Contents area

ECMWF Pilot Project on Emerging Technologies (ELEMENT)

Project

The pilot project aims to encourage cooperation among ECMWF’s member states in emerging technologies to modernize and make data access more democratic, as well as to enhance distributed data computing capabilities for Numerical Weather Prediction (NWP). By pooling expertise, we are developing a blueprint workflow tailored for multiple member states, with the potential to utilize European Weather Cloud (EWC) resources, which allow applications to run close to the data. This workflow is designed with flexibility for deployment in member states' computing facilities or public cloud environments. In addition, there is a commitment to establishing continuous discussions within a dedicated forum focused on GPU and other accelerators, with an emphasis on adaptation strategies like Domain Specific Languages (DSLs).

Project start01.01.2024
Project end21.12.2025
RegionInternational
StatusCurrent projects
  • Measurement & forecasting systems
  • Research & cooperation

Footer

Top bar Navigation

Swiss federal authoritiesSwiss federal authorities

Advancing Collaboration and Innovation in Weather Data Integration

This project aligns with ECMWF's initiative to enhance collaboration among Member and Co-Operating States. Established in December 2022, the pilot project focuses on adopting emerging technologies. ECMWF and Destination Earth will develop technology for data pipelines and end-to-end workflows to seamlessly integrate IFS data processing within Member States' internal workflows. Services like FDB and Aviso will be employed to ensure ECMWF's operational products are accessible, enabling the deployment of workflows that preprocess IFS data for distribution to Member States' operational infrastructure.

Delivering tailored advanced data pipeline blueprints for Member States

The project aims to develop blueprints for deploying data pipelines, leveraging advanced technologies such as Aviso, FDB, Earthkit-data, and Polytope. These blueprints will provide easy-to-deploy pipelines tailored to Member States' specific applications.

Key ECMWF Technologies:

  • Aviso: A notification system used to set up services triggering workflows.
  • FDB: A field database enabling semantic access to IFS production data.
  • Earthkit-data: A Python framework facilitating access and management of GRIB data. Member states can develop Python applications (based on numpy/xarray) using Earthkit to access FDB's GRIB messages.
  • Polytope: A service enabling efficient extraction of selected multidimensional regions from FDB.

Organization and Finance

MeteoSwiss leads the project coordination and development, leveraging its expertise in semantic storage (Gridefix) and cloud-native modernization efforts. MeteoSwiss commits 1 FTE for technical development and augments this with in-kind resources.