The Delicate Art of Predicting The Weather

Every single human experiences weather. And most of us have opinions on the local weather forecaster — probably that they are always wrong.

Yet almost none of us have any idea how we predict the weather. It’s kind of crazy, when you think about it. How does it work? How do we know what the weather will be? And what does your local meteorologist see that you don’t?

Let’s find out.

* * *

When I turned 12, I wanted to be a meteorologist. That was as surprising to everyone I told about it then as it might sound to you right now. What kind of kid wants to be a meteorologist? It doesn’t have the money or prestige that being a surgeon or a nurse or a scientist might get you, nor does it have the sensational fame and fortune that comes with being a professional athlete, singer, or writer. Not to you, at least.

To 12-year-old me, though, being a weatherman was everything. My favorite local weatherman was more a celebrity to me than any NFL quarterback. I was one of the most religious commenters on his blog. I got so inspired that I created a blog of my own, on which I posted with the fervor of a diehard sports fan every time there was even the slightest chance that big fat flakes would fall across my lawn, our street, our city.

Later that year, I got the chance to meet that local meteorologist. This is the list of questions I wrote for him before we met:



I didn’t become a meteorologist. But I have remained enthralled by the weather for just about my entire life. And I’ve often wondered what it is about predicting the weather that makes it so interesting to me, still.

Is it because I love exploring the computer models that simulate the weather, drawing my own interpretation from thousands of iterations? Is it because I love it when the world gets so quiet you could damn near hear a snowflake fall, and then you do, and for the next few hours you can forget about everything except for what’s falling from the sky and what movie you’ll watch that night? Is it because I just wanted to be “not like the other kids” in middle school and found a hobby that’s stuck with me for life? I haven’t found any of those explanations quite convincing.

No, I think predicting the weather is so enchanting for one reason: predicting the weather is predicting the future. It is to possess an ability usually relegated to sci-fi, to answer a question historically replied to with the words, “only God knows that, son.”

Predicting the weather is, in a way, knowing what God knows.

Where does the weather come from?

Weather is the implementation of energy. 

On Earth, that energy comes from the sun. Despite what some random guy in the YouTube comments might tell you, our planet is not flat. And as a result, energy from the sun is not evenly distributed. And it’s this energy differential that is the primary driver of what you call weather. It creates temperature gradients across the Earth’s surface and atmosphere, and these drive variation in pressure, humidity, wind, and more. You can think of energy differential as the foundational force for the way weather works.

Weather doesn’t stop at energy differential, though. There are other forces at play, like the Coriolis Effect — a phenomenon caused by Earth’s rotation which affects atmospheric movements (like wind and ocean current direction). Ocean currents have a major impact on weather, too, as does our atmosphere’s composition. And then you have topographical features like mountains and valleys and oceans. There’s a lot more, but if we covered it here you’d probably feel more like you had just opened a 1,000-page science book at the library and would possibly stop reading this essay.

All of these things, and more, help to define how wind, precipitation, and temperature work in our world. Without them, it would be impossible for our weather to work the way it does. There are, naturally, millions of other elements, both on a macro- and micro-level, that impact what goes on outside your living-room window. Accurately predicting the weather means being able to model both the high-level and the granular, from major atmospheric patterns down to city- and neighborhood-specific topographical details.

The weird part isn’t that your local weatherman sometimes gets things wrong. The weird part is that weather forecasts actually work at all. In fact, they work most of the time.

How is that even possible?

Computer models that predict the weather

Ever had a great idea, but not the resources to pull it off? Then you’ve felt a fraction of what Lewis Fry Richardson felt in 1922 when he published Weather Prediction by Numerical Process.

Lewis, a mathematician and scientist who had been enamored by the weather all his life, had been growing increasingly fascinated by the idea that we might be able to predict the weather via mathematical equations.

The idea, Lewis thought, made sense: we use mathematical equations to model the most high-impact processes in our Earth’s climate, then extrapolate the findings of those equations to predict the weather.

It did make sense. Few scientists would have told you otherwise. What they also would have told you, though, is that the computing power required to do such a thing was straight out of science fiction. Predicting the weather isn’t simple napkin math; it requires billions of advanced calculations. Lewis recognized this, beginning his essay on the topic with the sentence,

“After so much hard reasoning, may one play with a fantasy?”

       
He went on to describe an illustrious hall with many individual computers, each calculating separate parts of individual equations, and passing those on to higher-ranking clerks who would pass them to the overseer, a conductor of sorts, who would make sure that all the correct calculations were happening at the right time. From here you would get the weather. (The process Lewis describes is actually similar to the one used in The Three-Body Problem, when Wei Cheng attempts to create one massive human computer.)

The idea sounded like science fiction in 1922 because it was. 

Thirty years later, though, Lewis Fry Richardson got to witness the first ever weather forecast, generated by ENIAC, the first modern computer. The forecasts looked something like this (a recreation from here):



They were, for the most part, horrible: most of the first ENIAC forecasts performed worse than a persistence forecast. (Ifyou’re wondering what a persistence forecast is, it’s when you assume the weather 24 hours from any given moment will be exactly the same as it is in that given moment.) It’s a simple control, and ENIAC performed worse.

Still, that humans had created a computer model capable of even trying to predict the weather was a milestone. The first iteration of anything usually isn’t very good. And ENIAC’s forecasts gave mathematicians, meteorologists, and computer scientists alike good reason to spend the next decades getting it right.

And they did (get it right, that is).

The magic of 2024’s weather models

Today, we have weather models that do things Lewis Fry Richardson would have relegated to the realm of science fiction, even in 1950. But they follow the same logic Lewis proposed more than one hundred years ago: billions of micro-calculations and tens of thousands of complex equations about the atmosphere, happening all at once, in order to produce a weather forecast.

It’s impressive, but I want you to understand just how impressive it is. As our example, let’s use the European model: the ECMWF, around since the late ‘70s and often considered the global standard in weather forecasting. The following chart, which shows accuracy over the years, explains why:


Sure, up and to the right. It looks good. What this chart is actually measuring is the accuracy of one of the bread-and-butter metrics whether people look at: 500 hPa heights. In plain English, a 500 hPa height forecast refers to how high you’d need to go at a specific point in the atmosphere to reach a specific pressure. This can tell you a lot about the general weather patterns in a given region—it’s a high-leverage predictor.

In the early 1980s, after decades of improvement, we still could only predict 500 hPa heights with ~80% accuracy about 3 or 4 days in advance. Not being able to predict 500 hPa heights with accuracy means not being to predict even generalweather patterns (like if it’s more likely to be rainy or sunny next week, on average), which is to say nothing of the hyper-specific forecasts that weather models are now capable of.

Anyway, by 2021, that number nearly doubled. We can predict 500 hPa heights pretty accurately about 7 days out now. Which is a miraculous improvement, especially when you consider that the difficulty of forecasting weather grows exponentially with time: the accuracy of the calculations you make about next Tuesday depend on the accuracy of your calculations for Monday. The further into the future you go, the more prior predictions you rely on. Chaos theory.

Computer models are magical. But the real art is in how they get used.

What do the weather people actually do?

I’m going to let you in on a little secret.

If you go to a site called Tropical Tidbits—or one of the hundreds of other free sites like it that exist—you can get access to, basically, the stuff that your local meteorologist uses when they create a weather forecast.

These sites, updated many times a day, show a whole buffet of weather forecast maps, based on numerical weather model output, for various regions of the world. Just look at the selections you might for just precipitation and moisture data if you wanted to look at GFS (the American weather model) forecasts for the western United States.




Click the other tabs on the left and you’d get detailed maps for other things, like:

• Surface temperature
• Upper-air temperature
• Surface wind speed
• Upper-air wind speed
• Snowfall
• Rainfall

And much more. These maps make it possible for a trained local meteorologist (or even an amateur) to make a reasonably accurate prediction about the weather in a specific place. All in a matter of minutes.

Want to see what the temperature somewhere in the Pacific Northwest will be on the afternoon of Friday, March 8th? A couple of clicks and you’re there.



Here’s another little secret: these sites are not the same as the raw output of numerical weather models. That stuff, in reality, is borderline incomprehensible—remember that numerical weather models are powerful computers that run billions of calculations. 

To understand how this works, picture famous conductor Gustavo Dudamel directing an orchestra for a new movie soundtrack. In this scenario, the sheet music is the raw numerical weather model output. To most people, it looks like a jumbled mess of black dots on paper. But, Gustavo uses his expertise to conduct a number of musicians, who, working together, translate the separate, jumbled mess of sheet music into something beautiful; something that transcends all technical knowledge; something that quite literally anyone could enjoy and understand.

Sites like Tropical Tidbits do what Dudamel and the orchestra do: they use algorithms to translate complex raw output into readable data.

There are varying levels to this—not all of the maps that professional meteorologists use are quite as pretty or easy-to-grok as the ones above—but that is, basically, the process of creating the weather maps professionals use.

Most weather apps stop right there

You probably use a weather app on a consistent basis. A plurality of people do. And a majority of people use weather apps or other non-human-input forecasts, like what you’d see on a weather site, like Accuweather or TWC.

The cool thing is, if you’ve read this far, you have a pretty good grasp on how your weather app is actually making forecasts about the weather. In general, weather apps (like the default one on your iPhone, if you have an iPhone) usually pull raw data directly from numerical weather models, like the GFS and ECMWF.

They then use a proprietary blend of preference weighting, done by the team of engineers who built the app, to translate that into a forecast. Some weather apps might prefer the GFS to the ECMWF, or they might prefer the GFS to the ECMWF in certain regions. The specific numbers you see are up to the way your specific weather app has been tuned (that’s why some weather apps often outperform others).

This works most of the time. If you’re planning a weekend trip in the spring, you want to know if there’s a good chance you’ll be ambushed by a rogue party of rain showers. If you’re heading to the beach in the summer, you want to know how little clothing you can get away with wearing. For these generalizations, weather apps work.

Yet, weather apps most often fail in the most important situations; weather events where important outcomes rest on the edge of a knife. They fail because the lack the most important thing for making a valuable weather forecast when forecasts really matter:

People.

The (human) art of predicting the weather

What is the point of a weather forecast?

The American Meteorological Society says that forecasts are designed to:

“Support decisions ranging from the routine (e.g., whether to hold an event) to important and urgent decisions to help protect life and property when threatened by hazardous behavior.”

I mostly agree with this definition, but think there is a simpler way to say it: weather forecasts are designed to positively influence human behavior. Weather forecasting is the one of the only high-resolution mediums through which people can accurately predict the future and make important decisions based on those predictions.

There is just one problem.

Give the average person access to a site like Tropical Tidbits or WeatherBell and they would likely struggle to make good decisions about the future behavior based on what they see—there is just so much contrasting information. Looking at numerical weather model output (from hundreds of models, like the GFS and ECMWF and GEM, which often contradict each other) and making reasonable decisions about human behavior is a big leap.

That is why we have human meteorologists. The job of a meteorologist as it relates to forecasting has two main components.

I: Interpreting the large and high-frequency amount of weather model data that is out there (usually within specific context, like “What do I think the high temperature will be on Thursday?”, or “How much will it rain on Saturday, and at what time of day”).

II: Translating those interpretations into words that the general public can use to make the best-informed decisions about their future actions.

This is far harder than it might sound, for reasons your local meteorologist probably does not have the time to explain on T.V. when a major winter storm is blowing in. The below will explain why, with a real-world example. Let’s wind back the clock.

A thought experiment set in the past

You are going to make a (hypothetical) forecast with me. Ready?

Imagine you are a meteorologist in Portland, Oregon, a city that is essentially crippled when more than an inch or two of snow falls. It is Sunday, January 9th, 2017, and you are starting to form the idea that something wintry is on the way. On Tuesday.

You have a few reasons to believe this.

Weather models, including the ECMWF and GFS, have been hinting at the possibility of a setup for snow. The 500 hPa charts (hopefully you remember this from earlier!) show an upper-air pattern conducive to relatively cold air making its way to the Portland metro area. At the same time, some of these same weather models are suggesting that a weak surface low pressure (read: precipitation) could spin up from the west-southwest.

All that to say, you’re thinking there might be a chance for some snow. When you check out the ECMWF snow forecast for Portland on Monday night, it looks like this for tomorrow evening:


Not bad. Portland might get a few inches of snow. We’re less than 24 hours out from the potential event. So you should feel pretty confident going on T.V. tonight and telling the city to prepare for a few inches of snow, and day or two-long shutdown. Right?

Then, on Tuesday morning, the day of the potential event, you look at a more niche regional model, the WRF-GFS (built by the team at UW, this is based on the GFS model).

Sleddable, if the sled is more skateboard-shaped and has wheels and is… Well, if it’s a skateboard designed for pavement. So actually not that sleddable.

Bad news. Just when you thought you could make a confident prediction—the snow is supposed to fall this afternoon and evening, for crying out loud!—the WRF-GFS tells you that, actually, just a trace of measurable snow is going to fall in the metro area.

If the WRF-GFS is right and you tell the city to prepare for a major snowstorm, you’ve just given advice that has negatively impacted the behavior of millions of people. Plans have been canceled, businesses have prepared to shut down, people have called in sick for work, and more. Apart from human annoyance, hundreds of thousands (or even millions) of dollars might be lost because of this change in behavior. And you would be, at least in part, to blame.

So why don’t you just tell people there isn’t a major storm incoming? Because, after looking at the WRF-GFS on the morning of the potential event, you look at the forecast from the RGEM, a Canadian model.



One foot of snow? Not only is that dramatically more snow than the other models are showing, but it would be catastrophic for the city if it came true. It would cost millions of dollars and trap people in their homes for days (Portlandis hilly and most people do not have snow-capable cars or tires). If you tell people to expect nothing and they wake up to more than one foot of snow, you have a lot of explaining to do.

Potential consequences start to add up. If you’re wondering what the harm could be in inaccurately forecasting a major snow event in a city that can’t handle snow, imagine an elderly couple that hears there’ll just be a light dusting and decides to postpone a badly-needed grocery trip to tomorrow. If it does snow more than forecasted, that elderly couple could be trapped at home, without food. That’s dangerous, however you want to draw it up. And even if they’re within walking distance of a grocery store, postponing the trip to an icy, snowy day could cause a damaging (or worse) fall. I’m catastrophizing here, but there genuinely are serious ramifications to consider when you make a weather forecast in a situation like this one.

Remember, you’re seeing all of this on the morning of the potential event.

Now you see why weather apps often fail (and mislead people) when they’re needed most. A weather app, if it’s even looking at all the inputs you are—and it likely is not—cannot properly communicate the nuance. If your weather app leaned towards the GFS, you might have seen a forecast for just a trace of snow (or even nothing) on January 10th, 2017. If it leaned towards the ECMWF, you might have seen 3 to 6”.

What would you say?

Here is what Mark Nelsen, chief meteorologist at KPTV, had to say on morning of the 10th, with all the data (and more) that you just saw:

“It appears that just about all of us in the metro area should wake up to a blanket of white Wednesday morning and snow should be falling too… When it’s mainly done, expect 1-4” of snow on the ground. [But] some of our models are wetter, which would be a real snowstorm late tonight and Wednesday AM. I think that’s less likely.”

This paragraph takes the mess of model data I just described above and condenses it into a few key takeaways:

• It is going to snow Tuesday evening and Wednesday (enough that the ground will be covered).
• It will snow 1-4”, which tends to be a shut-down-the-city amount of snow in Portland.
• There is a chance it will snow a lot more.

This forecast accomplishes the right behavioral outcome in a situation like this. A reader understands that they should probably cancel their plans, stay home, and prepare in the event that the storm is bigger than forecasted.

More importantly, as the casual viewer cancels their events based on this forecast, they don’t see the WRF-GFS, which is saying most people in Portland wake up to dry ground. The reader is also not seeing the shocking RGEM forecast, which would probably prompt panicked grocery store rushes across the city. If you think you’d have made a forecast like the one above, you did a pretty good job given the information and probabilities you had available.

That night, it snowed up to 15” in much of the metro area, far beyond any forecast:




Great success! (For kids who wanted school canceled.) The RGEM was right. There was a lot of moisture. It snowed a lot. Happy days for kids who had school. Everything was a blanket of clear, quiet white. Like this:

Why people (still) matter

It’s 2024 and everyone is saying AI’s going to take all our jobs, but meteorologists are used to this. People have been making predictions about the end of human weather forecasting for a long time; someone probably muttered the sentiment when the first ENIAC forecasts came out, all those years ago.

They aren’t completely wrong. Weather forecasting has come a long way, and it continues to get better. But a couple of things remain true. One is that weather models are often least reliable in the situations where we need them the most: the exact amount of precipitation amount in a close-call Portland snowstorm; the razor-thin margins of a potential hurricane track into South Florida; it is in these crucial situations that weather models most often fail us. Because while a weather app can tell you most of the time what the weather will feel like outside, that doesn’t cut it when everything stands on the edge of a knife. You simply don’t get enough context.

Far more important, though, is one fact: weather forecasting is about people.

As such, weather forecasting is just as much about understanding what the weather models are saying as it is communicating the potential outcomes in a way that will be actually useful for people. A simple “70% chance of snow”, like what most weather apps would’ve shown for Portland’s January 2017 event, does not cut it.

No, what you need in these situations is a person who not only understands weather models but who understands communication. Who knows not to cry wolf about the most extreme model outcomes, and who knows not to downplay an event when there is an extreme outcome. When they do their job well, human meteorologists make your life better in a way that computers do not.

Computer weather models are a marvel. It is shocking and beautiful that, less than 70 years after Lewis Fry Richardson’s sci-fi dream, we created functional and accurate weather models. But it is the people who interpret them, the meteorologists who balance the art of probability and communication and risk, who perform the most delicate art of them all.

* * *

Thanks to Mark Nelsen, Steve Spalding, and Jordan Gonen for reading and providing feedback on this essay.

---

Postscript: Loose ends

How can I forecast the weather for myself?

For free sites, I’d go to Tropical Tidbits and Pivotal Weather. Start playing around with the maps. It’s honestly pretty intuitive. If you’re trying to guess the snowfall in your city a couple days from now, for example, you’d want to look at all the available model runs you can find (from lots of different models!). Look for patterns and trends. Look at model runs over the last few days to see which direction things are trending.
It’s fun, and relatively easy.

I once turned $5 into $800 over a steady series of ~75 bets on the weather, a long time ago. Which means you can do it, too. If you want to go premium, a site like WeatherBell is good (this gets you access to a lot more, but it’s by no means necessary if you just want to play meteorologist for a day).

What’s all that nonsense I hear about forecasting percentages being a lie?

One of the general public’s favorite Neil deGrasse Tyson-isms when the subject of weather forecasting comes up is to say,“did you know that a 50% chance of rain doesn’t actually mean there’s a 50% chance it will rain?”

And this is, actually, true. It’s a weird concept called probabilistic forecasting, which is great sometimes and really terrible other times. Literally speaking, probabilistic forecasting is the percentage chance that any given spot in the forecast area will receive precipitation (or whatever the forecasted variable is). That sounds reasonable, until you realize that forecast areas can often be quite big—like an entire metro area!—and that the weather can change by neighborhood, to say nothing of the entire city.

So here are a few weird things that can happen with probabilistic forecasting:

• If a forecaster is 100% confident that it will rain in 50% of the forecast area, then the probabilistic forecast for the entire forecast area would be 50%. On television it would be no problem to explain this, but a weather app will not do that—hence confusion.

• If a forecaster is 70% confident that it will rain in 30% of the forecast area, then the probabilistic forecast would be 21% for that forecast area (.7 x .3).

• Another weird ramification is that if weather forecasters actually show the probability chart to their audience (likehow 538 would show an electoral chart), you get a lot of misinterpretations (like 538 did when they ran their model). “Oh my god, it might snow 12 inches tomorrow!” says someone who saw a 5% chance of that quantity on an official National Weather Service graphic. Again, this is where human input matters.

I’m not really a fan of probabilistic forecasting, but understand why it exists. It’s especially bad on weather apps, though, where most users don’t understand what the primary metric they use to determine their behavior even means.

Are all T.V. meteorologists the same?

They are not! There are two ‘genres’ of T.V. meteorologists, in my estimation.
On one hand, there are people who are primarily T.V. presenters, but are also interested in meteorology (or may have minored it in college, or something to that effect). These people aren’t hardcore, weather-geeky meteorologists, but they are good at presenting on T.V. and have a strong enough grasp of weather to talk about it on television.

On the other hand, you have what we could possibly call the ‘real’ meteorologists. These are people who studied atmospheric science or something similar in college, who are obsessed with the weather, who’d spend their waking hours scrolling through hundreds of gigabytes of numerical model output if they could. These are generally the people you should trust more to predict the weather. Some of them are great presenters, and so it’s a great combo when you can get a‘real’ meteorologist on T.V. who’s also a good presenter.

At lots of T.V. stations, you have both! There may be a Chief Meteorologist who’s quite geeky about all the weather stuff, and then there may be other meteorologists or weather presenters who aren’t quite as into it, but can competently talk about it on television.

(If you’re really curious about this stuff, you can always look it up for your preferred station.)

What about the other kinds of meteorologists?

You may have noticed I’ve focused this essay talking about the places most people get their weather forecasts from (appsand local T.V. stations). That’s because I thought it would be most interesting for you to read, but you should know that there are lots of other types of meteorologists.

The big one is the National Weather Service, which is the government agency that issues all the official Winter Storm Warning-type forecasts and alerts you see. They do make local forecasts (and their forecast discussions, which are publicly-available, are often quite interesting for weather nerds) but aren’t quite as popular as local meteorologists and apps are. Your average person probably doesn’t even know they can go to the NWS site to look at weather forecasts.

Then, of course, there are all kinds of private meteorologists and companies that specialize in certain things, especially those where lots of money is involved: like hurricane and other major natural disasters. There is a lot of money to be saved by accurately predicting major weather events, which means there’s a market for paying private meteorologists. This isn’t a huge market, though, but worth mentioning.

Finally, there are the kinds of meteorologists who actually make the weather models and study the climate at a more technical level. These people often have their head buried in weather model calculations—they typically don’t make forecasts.

Can I become a meteorologist?

You can! It’s one of those interesting job markets where you don’t actually need a formal education to be good at the job(frankly, with a few weeks of intense practice, you could forecast as well or better as the bottom ~70% of meteorologists in the United States), but most places that would hire you to be a meteorologist will want to see a degree.

More meteorologists graduate than meteorology jobs are available, so it’s a relatively competitive market. But you can do it. With AI, there’s likely going to be some kind of market for meteorologists at AI startups to help improve weather forecasting, but a background in computer science will help if that’s the path you take.
You can likely make ~$50k - $150k per year, depending where you work. I’m not actually in the meteorologist job market, but based on research, that seems to be the normal distribution of salaries for meteorologists.

Why is the weatherman (or woman) always wrong?

He (or she) isn’t. You might be, though. Google ‘confirmation bias’.

On a serious note, confirmation bias is quite literally why it seems like your local weatherperson is always wrong. Two points here:

• Meteorologists are right about the important stuff the vast majority of the time, and even when they are wrong, you don’t usually notice it. A high temperature forecast of 85 on a day it’s actually 83 is hardly noticeable.

• But when they are wrong and you do notice it, it can feel like a huge deal. A 2-degree miss on a high temperature in the summer is nothing, but a 2-degree miss during heavy precipitation in the winter could be the difference between cold rain and a shuts-down-the-city snowstorm.

So, you are wrong about them being wrong. They’re not (wrong). Usually not, at least.

---