Skip to content

Data at ECMWF

ECMWF's Meteorological Archival and Retrieval System (MARS) enables you to explore and retrieve meteorological data in GRIB or NetCDF format. GRIB (General Regularly distributed Information in Binary form) is the WMO's format for binary gridded data and is designed for storing and distributing weather data. GRIB files are widely used in meteorological applications. NetCDF (Network Common Data Form) is a set of software libraries and self-describing, machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data.

Meteorological Archival and Retrieval System (MARS)

MARS request tree

How to make a MARS request?

What ECMWF says: here

General Regularly distributed Information in Binary form (GRIB)

GRIB is a binary format, and the data is packed to increase storage efficiency. GRIB messages are often concatenated together to form a GRIB file. GRIB files usually have the extension .grib, .grb or .gb.

Currently there are two different coding standards: GRIB edition 1 (commonly referred to as GRIB1) and GRIB edition 2 (GRIB2). The major differences are in the structure of the messages; in GRIB2, several variables are defined with more precision (e.g. in GRIB1, latitudes and longitudes are in milli-degrees while in GRIB2, they are in micro-degrees). Also in GRIB2, longitude values must lie between 0 and 360 degrees), the encoding of the parameter is very different, and in GRIB2 the description of the data is template/table based. Note that a GRIB file can contain a mix of GRIB1 and GRIB2 messages.

How to read GRIB files?

ECMWF provides and supports ecCodes. This software package has an Application Program Interface which makes ECMWF GRIB1 and GRIB2 files accessible from C, FORTRAN and Python programmes. ecCodes also provides a useful set of command line tools to give quick access to GRIB messages within the files.

Commonly used programs that can handle grib files:

  • Panoply by NASA
  • Metview by ECMWF
  • ecCodes by ECMWF
  • magics / metview by ECMWF
  • CDO
  • NCO
  • GrADS
  • idl

We have some examples of reading grib files:

it is also possible to convert grib files into netcdf files:

Convert GRIB to NetCDF
1
2
3
4
# load eccodes module or make grib_to_netcdf available to our PATH
module load eccodes
# convert to netcdf
grib_to_netcdf input.grb -o output.nc

grib_to_netcdf only works correctly when a single level coordinate is present in the GRIB file. Often the model output files have fields on multiple level types (ie. hybrid model levels and pressure levels).

Split Model level types
1
2
3
4
# check the contents of the grib file
grib_ls input.grb
# split by model lvel type
grib_copy input.grb output_[typeOfLevel].grb

more information can be found at ECMWF

Example of an efficient MARS request

It is not easy to write good MARS requests, because there are so many parameters.

A few things are important: 1. NetCDF does not handle well different times and steps. 2. Retrieving should loop by experiment, date, time and retrieve as much as possible 3. Check the catalogue first and do a "estimate download size" check. Look for the number of tapes. If it is one tape, then you are fine.

MARS keywords HRES Guide

MARS retrieval of HRES operational forecast
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
#!/bin/bash

# this example will filter the area of Europe (N/W/S/E) and interpolate the final fields to
# a 0.5x0.5 regular lat-lon grid (GRID=0.5/0.5)
AREA="73.5/-27/33/45"
GRID="0.5/0.5"

# fixed selection from the same block
PARAMS="130/131/132"
LEVELIST="127/128/129/130/131/132/133/134/135/136/137"
STEP="0/to/90/by/1"

TIMES="0000 1200"
YEAR="2017"
MONTH="04"

#date loop
for y in ${YEAR}; do
    for m in ${MONTH}; do
        #get the number of days for this particular month/year
        days_per_month=$(cal ${m} ${y} | awk 'NF {DAYS = $NF}; END {print DAYS}')

        for my_date in $(seq -w 1 ${days_per_month}); do
            my_date=${YEAR}${m}${my_date}

            #time loop
            for my_time in ${TIMES}; do
                cat << EOF > my_request_${my_date}_${my_time}.mars
RETRIEVE,
    CLASS      = OD,
    TYPE       = FC,
    STREAM     = OPER,
    EXPVER     = 0001,
    LEVTYPE    = ML,
    GRID       = ${GRID},
    AREA       = ${AREA},
    LEVELIST   = ${LEVELIST},
    PARAM      = ${PARAMS},
    DATE       = ${my_date},
    TIME       = ${my_time},
    STEP       = ${STEP},
    TARGET     = "oper_ml_${my_date}_${my_time}.grib"
EOF

            # request all times together
            mars my_request_${my_date}_${my_time}.mars
            if [ $? -eq 0 ]; then
                rm -f my_request_${my_date}_${my_time}.mars
            fi
            done
        done
    done
done
Last update: December 6, 2024
Created: February 1, 2024