Machine learning-based weather forecasts now available at kit-weather.de

For more than six decades, weather predictions based on numerical models solving physical equations have continuously improved and thus provided increasingly better information for day-to-day decision making. This "quiet revolution"[1] can be attributed to, among other things, a better observing system, better data assimilation, a better representation of physical processes in the model, and continuously growing computing resources. In the last three years, a paradigm shift has taken place, culminating in the last 12 months: Instead of research institutions and weather services, large technology companies such as Google, NVIDIA, Microsoft, and Huawei have developed weather prediction systems based on machine learning algorithms trained with decades of meteorological data. The astonishing thing is that these models achieve for some, especially larger-scale, meteorological fields a forecast quality that is similar or even higher than that of numerical weather prediction models run by the world's leading forecasting centers but at computing costs that are several orders of magnitude smaller

For IMK’s own website kit-weather.de, we now provide 7-day forecasts using the Pangu-Weather algorithm developed by Huawei [2]. This model was trained with so-called reanalysis data from the past 40 years provided by the European Centre for Medium-Range Weather Forecasts. Taking the operational analysis of the German Weather Service (DWD) as initial conditions, it now allows us to generate cost-effective and energy-saving forecasts. On the high-performance computers of KIT’s Steinbuch Centre for Computing, a global 7-day forecast using Pangu-Weather consumes 14 Watt-hours (a medium-sized electric car consumes this energy on 90 m of driving). In contrast, the energy consumption for a forecast with a state-of-the-art numerical weather prediction model would amount to 30,000 Watt-hours (an electric car consumes this energy on 200 km of driving). This estimate, however, does not take into account the energy needed to generate the training data, nor does it include the energy needed for the training itself. Still the development of purely data-driven weather forecasting systems is another revolution but faster and maybe less quiet than the first. It opens up many exciting research avenues, which will be explored in future projects at the IMK-TRO such as TEEMLEAP (A new TEstbed for Exploring Machine LEarning in Atmospheric Prediction) and the ERC-funded project ASPIRE (Advancing Sub-seasonal PredIctions at Reduced computational Effort).

[1] Bauer, P., Thorpe, A. & Brunet, G. The quiet revolution of numerical weather prediction. Nature 525, 47–55 (2015). https://doi.org/10.1038/nature14956

[2] Bi, K., Xie, L., Zhang, H., Chen, X., Gu, X., Tian, Q. Pangu-Weather: A 3D High-Resolution Model for Fast and Accurate Global Weather Forecast. (2022). https://arxiv.org/abs/2211.02556