Data preparation

In the MeteoSwiss data warehouse, measurement data are processed and systematically reviewed for users on a continuous basis. Measurement gaps are filled, additional parameters are calculated and corrections are made. This quality assurance process comprises different steps.

In the data warehouse (DWH), all measurement data are consolidated in order for them to be stored over the long term in a standardised form. Once stored in the DWH, the data then undergo various preparation processes. The older the measurement data, the more calculations, quality assurance checks and amendments that have been performed. Not all data series are subjected to the complete preparation process. Rather, the exact processing programme depends on how the data will be used. The most important processing steps include calculating additional parameters, filling measurement gaps and making corrections.

Calculation

  • Temporal: On the basis of values recorded every ten minutes, hourly, daily, monthly or annual values can be calculated.
  • Spatial: For a defined area, values are collected from the monitoring stations situated within it and used as a basis for an area value.
  • Additional parameters: Additional parameters such as the Foehn wind index, current weather codes, relative humidity and air pressure reduced to sea level are calculated.

Filling gaps

  • Manually: Certain gaps can be filled manually based on knowledge of the weather situation. The amended values are marked so that it remains clear at a later point how the values were calculated and to ensure they can still be scrutinised further down the line.
  • Automatically: Minor temporal measurement gaps at a station are automatically filled using an interpolation procedure. These values are also identified. The permitted gap size depends on the measurement parameter in question. In the case of air pressure, for example, lengthy gaps can be closed automatically. This is not the case for precipitation, however, as it varies more over space and time than air pressure does.

Correction of incorrect or implausible measurement values

The measurement data in the DWH is subjected to approximately 250 mathematically defined tests. On average, around 2% of values are either incorrect or dubious. Clearly incorrect values are automatically eliminated from the data series and, if possible, corrected manually. Measurement values deemed to be unlikely are marked automatically and checked manually at a later point.

  • Should a physically impossible value be recorded (e.g. a negative amount of precipitation), this clearly represents an error.
  • If certain climatological threshold values are exceeded (e.g. if a temperature of 25 °C is recorded in Zurich in February), this raises the suspicion of a measurement error.
  • Doubts are also raised if a parameter (e.g. temperature) remains constant over an extended period or changes erratically over a short space of time. However, such measurements are not necessarily incorrect: In the case of fog, for example, the temperature only changes minimally. On the other hand, should a Foehn wind be blowing, abrupt increases in temperature are possible.
  • If various parameters are inconsistent (e.g. if a station reports precipitation and sunshine at the same time), this indicates an error and the data is checked in more detail.

Long-term correction (homogenisation)

For selected and very significant measurement series, the data is homogenised a period of months or years after the measurements are taken. From these series of measurements, systematic measurement errors and leaps in data are eliminated and corrected. These may arise as a result of station relocations, measuring device errors or a change of measuring device. Following this treatment, the data is deemed to have achieved the highest quality level.