Farewell to External AI Models

Summary

Machine learning (ML) models have become an increasingly important component of the global weather forecasting landscape. Over the past few years at ECMWF, several externally developed models have been run alongside the Integrated Forecasting System (IFS) in an experimental setting, allowing their performance to be assessed against established numerical methods. The upgrade of the IFS to Cycle 50r1 has provided a test of how robust these models are to changes in their underlying data. In this blog post, we discuss the impact of the IFS upgrade on the performance of the ML models run at ECMWF, and why this has prompted the decision to stop running external models in real time. The IFS and ML models The IFS is the global numerical weather prediction (NWP) model run operationally by ECMWF. It receives regular updates, with each upgrade incorporating new science, improving its performance. The IFS is also the backbone of the ERA5 reanalysis dataset, which is used to train ML models for global weather forecasting. ERA5 is based on IFS Cycle 41r2 – the operational cycle in 2016. ML models for global weather forecasting are generally trained on approximately 40 years of ERA5 data and, for some, fine-tuned on the latest version of the IFS. This is the case for the Artificial Intelligence Forecasting System (AIFS), ECMWF’s ML-based forecasting model that has been operational since February 2025. More precisely, AIFS v1.1 is fine-tuned on seven years of operational analysis spanning January 2016 to January 2023. A verification experiment Since 2023, ECMWF has run several external ML models in experimental mode. Forecasts from a selection of these models are routinely made available on openCharts, providing the weather community with quick and easy access to ML-based predictions. This effort predates the creation of the AIFS and has helped build a better understanding of the structure and performance of these new types of forecasts, which helped initiate the uptake of ML in global weather forecasting. At the beginning of 2026, the set of external ML models running daily at ECMWF comprised the following: Pangu-Weather (Huawei), GraphCast (Google DeepMind), Aurora (Microsoft Research), and FourCastNet (Nvidia). These models were run twice daily (at 0 and 12 UTC) and were initialised with the IFS operational analysis. Prior to the 50r1 upgrade, we conducted a verificatio…