New AI Model Shines During Hurricane Erin
Google’s DeepMind machine learning weather model tops other forecast models for Erin, but will it continue?
Google DeepMind, a London-based AI research lab, has been in the business of machine learning-based weather forecasting for several years, but back in June announced a new experimental AI model specific to tropical storms and hurricanes that would be evaluated in real-time this season by forecasters at the National Hurricane Center.
In its most important high-profile test to-date during Hurricane Erin, Google DeepMind showed great promise, besting many of our top conventional weather models and beating out the corrected consensus aids that have become the gold-standard of hurricane forecasting in recent seasons.
In verification data shared by former National Hurricane Center Branch Chief James Franklin on Monday, the Google DeepMind model had the best track and intensity forecasts for Erin for the first 3 days (72 hours) across all forecasts, and remained competitive with the best-performing track and intensity aids through five days across all forecasts.
While only one storm, the forecasts during Erin from Google’s machine learning-based hurricane prediction system are consistent with forecasts backtested by NHC for the 2022 to 2024 hurricane seasons which showed similar high-quality results.
Why AI?
The new machine learning-based weather models offer a huge time savings to conventional weather models that take hours upon hours to solve very complex governing equations of the atmosphere using expensive supercomputers. While the accuracy of traditional weather models has improved greatly in recent decades, they require vast computational resources and some of the most powerful computers on Earth to run.
The new class of AI weather models, rather than making a forecast through complex physical equations, take a shortcut by training models on historical weather data and recognizing weather patterns instead, exploiting faster and cheaper deep-learning hardware over larger supercomputers.
While there are tradeoffs to machine learning-based weather models, if they can prove to be as accurate as conventional physics-based weather models, we can leverage them to churn out faster and computationally less expensive forecasts.
AI blindspots
This hurricane season is the first with mature AI models and AI ensembles. Because we’re still in the early stages of evaluation, we need to be cautious leaning in too much just yet on this new method of hurricane prediction, which, like conventional forecast models, has its blindspots.
AI models use statistics to optimize skill, basically training the model to look best on paper using measures like root mean square error. While there’s nothing inherently wrong with this approach, it can cause real problems in certain circumstances with forecasts. For example, an AI forecast might completely miss rapid intensification because it’s trained to minimize errors, which will tend to smooth out the intensity forecast over several days, eliminating scenarios showing rapid intensification, even if the physics favor it.
Another example can be found with Erin last week. The root mean square error statistics often used by AI are based on something called the great‐circle distance – the shortest possible path between the forecast location and the actual location of the hurricane. The error from great-circle distance alone, however, can be misleading, since it overlooks important timing and landfall considerations better represented by what we call along- and cross-track errors.
In the case of Google DeepMind during Erin, it may have attempted to minimize the error in great-circle distance with its forecast, suffering larger along- and cross-track errors and missing the critical juncture when Erin turned north and out to sea.
In contrast, the gold-standard corrected consensus models like the Real-Time Hurricane Forecast Improvement Project Corrected Consensus Approach or HCCA, did a much better job catching the timing of the pivotal turn, even though it technically had a higher root mean square error than Google DeepMind.
Because AI-based weather models are trained on historical weather, they may also suffer during extreme weather events not previously documented in the training datasets. Physics-based models, on the other hand, have a much more complete understanding of the atmosphere and should capture even extreme outlier weather events even if they’ve not been documented before (think Sandy’s left hook in 2012).
That said, AI could be used in conjunction with physics-based weather models to capture processes we may not yet understand about the physical world (equations that aren’t built into our models) or processes that happen with such fine detail our conventional models can’t “see” them. Through methods like parameterization, we may be able to improve physically-based models with some processes AI catches solely by pattern recognition.
What we know so far about AI models for hurricane forecasting
While we’ve been looking at AI models for hurricane forecasting for a few years, we’re only now just beginning to evaluate them in a systematic, scientific way. Here’s what we know so far:
AI models are competitive with physics-based models for hurricane track, especially in the 3 to 7 day forecast window
So far, AI models have shown no consistent skill in forecasting hurricane intensity (which makes DeepMind’s forecast for Erin all the more impressive)
We’ve not yet evaluated the skill of AI in forecasting tropical cyclogenesis (the formation of tropical depressions, tropical storms, and hurricanes)
AI models, like physics-based models, suffer blindspots, largely through optimization strategies and an incomplete training dataset
No major development threats behind Fernand
Fernand got a big haircut overnight, losing all thunderstorm activity to blistering wind shear, as it accelerated northward over the open Atlantic.
Though NHC does mention the possibility of some very brief restrengthening as it passes over a pocket of warm waters in the next 24 hours, Fernand is forecast to lose its tropical characteristics by late Wednesday.
As we discussed in Monday’s newsletter, the Atlantic will tap the breaks for week or two behind Fernand. We may see some mischief along a stalled front off the east coast this weekend or early next week, but for now significant tropical development isn’t expected.
Wow - I'm going to have to reread the AI again, lol. There's layered learning going on...
Ty Mr. Lowry 😃
(& I sure am glad for the storm lull!)