Advancing Collaboration and Innovation in Weather Data Integration

This project aligns with ECMWF's initiative to enhance collaboration among Member and Co-Operating States. Established in December 2022, the pilot project focuses on adopting emerging technologies. ECMWF and Destination Earth will develop technology for data pipelines and end-to-end workflows to seamlessly integrate IFS data processing within Member States' internal workflows. Services like FDB and Aviso will be employed to ensure ECMWF's operational products are accessible, enabling the deployment of workflows that preprocess IFS data for distribution to Member States' operational infrastructure.

Delivering tailored advanced data pipeline blueprints for Member States

The project aims to develop blueprints for deploying data pipelines, leveraging advanced technologies such as Aviso, FDB, Earthkit-data, and Polytope. These blueprints will provide easy-to-deploy pipelines tailored to Member States' specific applications.

Key ECMWF Technologies:

• Aviso: A notification system used to set up services triggering workflows.
• FDB: A field database enabling semantic access to IFS production data.
• Earthkit-data: A Python framework facilitating access and management of GRIB data. Member states can develop Python applications (based on numpy/xarray) using Earthkit to access FDB's GRIB messages.
• Polytope: A service enabling efficient extraction of selected multidimensional regions from FDB.

Organization and Finance

MeteoSwiss leads the project coordination and development, leveraging its expertise in semantic storage (Gridefix) and cloud-native modernization efforts. MeteoSwiss commits 1 FTE for technical development and augments this with in-kind resources.