WE鈥橵E all been there. Everything鈥檚 set for a great barbecue but when the day
arrives the weather forecast turns out to be as reliable as a mafia tax
return鈥攁nd down comes the rain.
Still, what can you expect? We all know weather forecasts aren鈥檛 100 per cent
reliable and we all know why. It鈥檚 the butterfly effect, isn鈥檛 it? That
notorious spin-off of chaos theory which says that the way the weather unfolds
is so sensitive to tiny fluctuations that a butterfly flapping its wings in
Kansas could trigger a typhoon in Singapore鈥攐r a downpour on your summer
party. With the weather being that sensitive to tiny changes, what hope have
forecasters got?
But now a small team of researchers is challenging this idea. David Orrell, a
mathematician at University College London, says it鈥檚 time to break the
butterfly鈥檚 spell and rouse meteorologists to a bright, new truth: weather
forecasts can be far, far better than they are right now. In fact, Orrell is
confident that we should be able to make near-perfect forecasts for at least 3
days ahead鈥攑erhaps even for more than a week. At the moment, forecasters
struggle to predict much beyond 24 hours with any accuracy.
Advertisement
No one is suggesting that chaos theory is wrong, or that forecasting is a
doddle. Just that the butterfly effect is not where the trouble lies. For
short-term forecasts鈥攅ven up to a week鈥攖he butterfly effect hardly
figures, say Orrell and his colleagues. The real problem, they say, is errors in
the meteorologist鈥檚 own mathematical models鈥攖he vast array of equations
they use to capture the physics behind the weather. It鈥檚 these errors that
dominate the picture and ruin the forecasts.
It鈥檚 a brave assertion, and it won鈥檛 go down well with many forecasters. Some
cynics suggest that the butterfly effect has served as a fig leaf for
meteorologists, conveniently hiding the embarrassing flaws in their models. For
everyone else, though, it鈥檚 wonderful news鈥攂ecause, unlike the butterfly
effect, a faulty mathematical model is something you can fix.
With help from Orrell鈥檚 new mathematical analysis, meteorologists are already
starting to pin down exactly where the problems lie and are devising clever ways
to fix them. Soon we may finally see the accurate predictions we鈥檝e always
longed for鈥攁 truly reliable forecast that farmers, pilots, sailors and
even the financial markets can trust. Something you can even stake your barbecue
on.
To most people, the notion that mathematical models are to blame for dodgy
forecasts may seem less than revolutionary, especially when you consider the
task that meteorologists face. To create any kind of forecast, they have to
build hugely complex models of the atmosphere, crammed with equations describing
the way the whole thing evolves. They plug in the starting values鈥攖he
current weather conditions鈥攁nd set the model running to calculate what the
next few days or weeks will bring.
Meteorologists would certainly be the last to claim that their models are
perfect. Even with today鈥檚 supercomputers, they admit they鈥檙e nowhere near
nailing every detail of the 5 million billion tonnes of air swirling over our
heads. But what they have come to believe is that any errors arising from their
equations are tiny compared to the humungous inaccuracies thrown up by the
butterfly effect. Chaos, they believe, always amplifies small errors in the data
into huge ones, rapidly making the forecasts wildly inaccurate.
Forecasters haven鈥檛 always thought that way. During the 1960s and 1970s,
people assumed that it was the models that were the source of forecast errors.
But then came Ed Lorenz, a meteorologist at the Massachusetts Institute of
Technology. He was the first to highlight the role of chaos in weather
forecasting鈥攁nd coined the term 鈥渂utterfly effect鈥. 鈥淐haos theory caused a
major change in thinking,鈥 says Orrell. 鈥淧eople then somehow started to think
that this was the major source of the trouble.鈥
It fitted right in with the zeitgeist. During the 1980s, newspapers and
magazines began carrying stories on chaos theory. James Gleick鈥檚 Chaos:
Making a new science was riding high in the best-sellers lists. And
according to Britain鈥檚 Meteorological Office, it was the butterfly effect that
was ultimately responsible for its failure to predict the Great Storm of 1987,
which struck southern England with winds of over 150 kilometres per hour.
Since chaos hit the scene, the vast majority of meteorology research has gone
into trying to gauge its impact on weather forecasts. It has led to the
emergence of the 鈥渆nsemble technique鈥 for predicting the weather, in which a
computer model generates families of 50 or more forecasts, each starting with
slightly different figures for temperature, pressure and the like. How strongly
these forecasts diverge from each other over time gives scientists a handle on
the current strength of the butterfly effect. It tells them how much confidence
they should place in the computer鈥檚 predictions, and what direction the weather
is most likely to take.
Oddly, however, the uncertainty estimate culled from ensemble techniques
still frequently fails to include what actually happens, even on three-day
forecasts. A day of sun and scattered showers, for example, might turn out to be
just a steady downpour instead. It鈥檚 as if there鈥檚 something else throwing the
forecasts off-track.
According to Orrell, that something is not chaos, it鈥檚 the model itself. And
he presents his evidence in a forthcoming paper in Nonlinear Processes in
Geophysics, co-authored by Lenny Smith of Oxford University and Tim Palmer
and Jan Barkmeijer, two meteorologists from the European Centre for Medium Range
Weather Forecasts near Reading.
The paper is the result of three years of research carried out by Orrell and
Smith at Oxford. Together, they developed a technique to reveal exactly where
the errors in the forecasts creep in. There were two main hurdles, says Orrell.
First, you have to separate the total error in the forecast into its constituent
parts鈥攖hat due to the model itself, and that caused by uncertainty in the
starting conditions plugged into the model. 鈥淪econd, you have to show how the
model error evolves with time, so that you can measure its relative importance,鈥
he says.
The difference between model error and starting-condition error鈥攚hich
is what the butterfly effect feeds on鈥攔eveals itself in a phenomenon
called 鈥渟hadowing鈥. Imagine that meteorologists have a perfect model of the
weather, capturing its every nuance, and they use it to produce a family of
forecasts. As these forecasts fall prey to the butterfly effect, their
predictions would diverge from the real weather in a very distinctive way,
spreading out but with the cloud of uncertainty still clustering around the
actual weather. In other words, the model鈥檚 family of forecasts would 鈥渟hadow鈥
the real weather conditions in the short term.
In reality, of course, even the best models have some flaws. And this, Orrell
and Smith found, makes forecast errors behave in a fundamentally different way.
Although the errors grow under the influence of the butterfly effect, the family
of forecasts no longer clusters around the true weather. The flaws in the model
mean the 鈥渟hadow鈥 drifts away from its expected position. Before long, there is
no overlap at all between what the model predicts and the real weather.
Orrell and Smith quickly realised that patterns of shadowing could test their
suspicion that model errors rival鈥攊f not surpass鈥攖he butterfly
effect. Orrell went further, and proved a mathematical theorem that shows
precisely how errors due to the model itself grow with time. This amounts to a
fingerprint that tells you when the model, rather than the butterfly, is to
blame for inaccurate forecasts.
As any textbook on chaos theory will tell you, the errors due to the
butterfly effect grow exponentially with time. Orrell showed that errors caused
by the model follow a completely different law: they will increase with the
square root of time鈥攇rowing relatively rapidly at first, before slowing
down to a much more sedate pace after a day or so.
As part of a long-standing collaboration, Orrell and Smith teamed up with
Palmer and Barkmeijer to look for the telltale sign of drift in forecasts
produced by the ECMWF Operational Model鈥攚idely regarded as one of the best
weather models in the world. If model error was dominating, it would betray
itself through forecasting errors that grow very rapidly early on before
levelling off. Then the error curve should start to bend upwards again as the
exponentially increasing butterfly effect begins to dominate.
When the team plotted its forecast errors against time, the curve followed
Orrell鈥檚 square-root law with amazing accuracy
(see Diagram): 鈥淚t certainly
didn鈥檛 look anything like the exponential curve expected for the butterfly
effect,鈥 says Orrell.
Further analysis revealed a remarkable fact. Around 90 per cent of all the
forecast errors for the first 72 hours could be pinned on flaws in the model
rather than the butterfly effect: 鈥淚t seems that the model errors account for
the large majority of the forecast error even as far out as five days.鈥
The team was taken aback by the speed with which the flaws in the Operational
Model drove it off-target. 鈥淲e estimate that its forecasts shadow reality for
only three to four hours,鈥 says Orrell. In other words, the flaws in the model
start to have an effect almost immediately. This, Orrell says, has important
implications for the ensemble techniques now widely used by meteorological
services. 鈥淭he assumption is that the basic model is pretty much perfect, so
only the initial-condition errors are important,鈥 says Orrell. 鈥淥ur results
suggest this isn鈥檛 the case. That would at least explain why these ensemble
methods seem to over-estimate the reliability of the final forecasts.鈥
Their conclusion, Orrell believes, may be extremely unwelcome in some
quarters of the meteorological community, but everyone else should be pleased.
Despite what chaos theorists have been telling us for years, the butterfly
effect needn鈥檛 crush all hopes of major improvements in forecasting鈥攂oth
in reliability and range: 鈥淭he weather may be chaotic,鈥 says Orrell, 鈥渂ut it鈥檚
not that chaotic.鈥
Over timescales of at least several days, flaws in the model are by far the
bigger source of forecasting error. This means that fixing them could bring big
payoffs. Get the models right and, according to Orrell, you could see almost
perfect three-day forecasts. It should even push the limits of reasonably
reliable forecasting well beyond the current limit of around 10 days or so. 鈥淚f
model error introduces large errors over the first few days, then the butterfly
effect will amplify those errors at longer times,鈥 says Orrell. 鈥淪o improving
the model will help nip those errors in the bud too.鈥
It鈥檚 a message that is already going down well with those with a big stake in
getting forecasts right such as Joseph Hrgovcic, head of research into
weather-risk management at Houston-based commodity broker Enron. 鈥淭he paper is
brilliantly done,鈥 he says. 鈥淭he implications of his results are wide-ranging.鈥
But sorting out the problems Orrell has uncovered is a daunting task. 鈥淚t is
not easy to deal with model error,鈥 says meteorologist Peter Houtekamer of
Canada鈥檚 environment ministry. 鈥淎fter all, we do not know what鈥檚 wrong with the
model.鈥 And ensemble forecasting remains crucial in order to see the effects of
uncertainty, says Smith, 鈥渆ven if it is blind to the effects of model error.鈥
Palmer insists that the ECMWF has been tackling model error for some time,
but he is adamant that getting more accurate data to feed into the models is
just as important. A 10-year international research programme entitled THORPEX
should help achieve that. This will use devices such as robot planes to gather
up-to-the-minute data from inaccessible regions of the globe.
Finding which of the possible model errors are most significant will be far
harder. Many atmospheric interactions are difficult to model鈥攍ike the
way the surface of the Earth creates turbulence in the atmosphere. Complex
things like cloud formation and dissipation are also vital, says Orrell. 鈥淏ut
they are one of the hardest to model.鈥 Here NASA鈥檚 GIFTS satellite could help.
Due for launch in 2004, it should provide high-quality atmospheric measurements.
鈥淢y own bet,鈥 says Orrell, 鈥渋s that errors will be biggest at low altitudes,
where there is plenty of data but the equations are hard to get right.鈥
Meteorologists must also rely on their equations to account for what the
weather is doing at scales much smaller than 40 kilometres. One solution, Palmer
suggests, may be to include a random-noise component in the equations to help
offset this.
David Stensrud from the National Severe Storms Lab in Norman, Oklahoma, has
already begun to use Orrell鈥檚 technique. He has found that errors seem to grow
fastest in the lower atmosphere, where turbulence is significant, just as Orrell
predicts. Stensrud now plans to configure a number of different models and use
the same techniques to pick out the versions that perform best.
Yet not all meteorologists are ready to wake from the butterfly鈥檚 spell. When
Orrell and his colleagues present their work at conferences, some are reluctant
to accept that there鈥檚 scope for dramatic improvements in forecasting. 鈥淭here鈥檚
still something of a consensus that model error is small, and it has proved hard
to convince some meteorologists otherwise,鈥 says Orrell.
Perhaps the community will slowly come around to the idea. Houtekamer already
sees signs that the butterfly is loosening its grip. 鈥淚 think we鈥檙e all now
discovering that model error is what limits the quality of our forecasts
and that we have to address it in order to make further progress.鈥
Orrell hopes that his findings will reinvigorate a field whose veterans put
up with endless criticism from a public that barely comprehends the scale of the
problem they tackle. 鈥淢odelling the Earth鈥檚 atmosphere has to be one of the most
difficult computational tasks ever attempted,鈥 he says. So having flawed models
is hardly something to be surprised at鈥攐r ashamed of.
-
More information at:
www.ucl.ac.uk/~ucesdor