Training AI models on historical weather patterns can turn them into accurate forecasters – but they may not be able to predict extreme events that don’t occur in their training data. This could be a growing issue as climate change drives more unprecedented weather.
“These models are good, but the question we have been asking is about events that are so rare and strong that these models haven’t seen anything like them before,” says at the University of Chicago.
Over the past few years, researchers have developed AI-powered weather models that are as accurate as conventional forecasts. Instead of calculating changes in the weather based on equations that describe the physics of the atmosphere, these deep learning models are trained to recognise statistical patterns amid historical weather and climate data. They are also far faster and more computationally efficient than traditional techniques.
Advertisement
However, an AI model’s predictive ability depends on the quantity and quality of its training data. This fact led experts to worry that AI weather models will struggle to forecast events that occur infrequently – such as the extreme disasters that cause the most damage.
“Generalisation to extremes is super important, and… you’d think AI models would be poor at it,” says at the University of Cambridge.
To precisely test this possibility, Hassanzadeh and his colleagues retrained a leading AI weather called FourCastNet using a dataset that excluded all instances of tropical cyclones stronger than a category 3 storm. “Instead of creating something hypothetical that doesn’t exist now, we just cut the training dataset,” says team member , also at the University of Chicago.
They found the resulting model was unable to predict the most powerful cyclones and hurricanes, events that the model trained on complete data had been able to capture. The researchers say this shows that, unlike a conventional physics-based model, the AI model can’t extrapolate stronger events from weaker ones.
That’s a concerning limitation as climate change shifts historical weather patterns across the planet. “Climate change supercharging these extreme events can increase the likelihood that these models do not do well,” says Hassanzadeh.
The researchers only tested one of the several advanced AI weather models out there, and they only looked at tropical cyclones. But they say this same inability to extrapolate extreme “grey swan” events from past data is likely to be a problem for all models predicting all types of rare weather, whether intense heatwaves or extreme precipitation.
“You don’t even need to invoke climate change to see that the AI models will be limited unless someone can produce a revised method that can extrapolate beyond the training dataset,” says at the University of Manchester, UK.
Encouragingly, however, when the training data included tropical cyclones in only the Pacific or Atlantic Ocean, the model was able to forecast powerful storms in both ocean basins. “It was very surprising,” says Hassanzadeh. “It generalises from one region to the other.” That means the models may be able to forecast an event that is record-breaking in one part of the world but has happened elsewhere. For instance, the researchers found in a that AI models could forecast 2024’s extreme rainfall in Dubai, predicting that unprecedented event based on extreme rain that occurred elsewhere in the world.
The researchers say there are a number of ways to improve future AI weather models’ view of grey swan events. For instance, developers could link AI and physics-based models to train them on extreme events or design better weather-specific learning algorithms.
“This is just the start of a greater assessment of the performance of AI systems and extreme weather, which is when we need them most,” says Turner.
PNAS
