Access to MARS¶
This Service will allow authorised users to retrieve and list MARS data from outside the ECMWF facilities. Users within ECMWF Member and Co-operating States may contact their Computing Representative to obtain access to MARS. All other users may request a username and password, following these instructions.
Accessing and retrieving files from MARS got a lot easier, with this package.
We need to install the web client for MARS and get credentials from ECMWf to allow the API to call MARS.
In [6]:
Copied!
!pip -q install --user ecmwf-api-client
!pip -q install --user ecmwf-api-client
You need an ECMWF account and login to get the login credentials for the web api of MARS.
Go to https://api.ecmwf.int/v1/key ( click login ) and then same the information to a file called $HOME/.ecmwfapirc
In [5]:
Copied!
%%writefile /mnt/users/staff/mmustermann/.ecmwfapirc
{
"url" : "https://api.ecmwf.int/v1",
"key" : "1234567890???????",
"email" : "max.mustermann@univie.ac.at"
}
%%writefile /mnt/users/staff/mmustermann/.ecmwfapirc
{
"url" : "https://api.ecmwf.int/v1",
"key" : "1234567890???????",
"email" : "max.mustermann@univie.ac.at"
}
Overwriting /mnt/users/staff/mblaschek/.ecmwfapirc
This is an example of a request. If you want to figure more exactly what you need it is best to browse the online catalog of MARS and then get exactly what you want.
Look here: Mars Catalogue or the Archive Catalogue (freely)
In [7]:
Copied!
from ecmwfapi import ECMWFService
server = ECMWFService("mars")
server.execute(
{
"class": "od",
"date": "20150101",
"expver": "1",
"levtype": "sfc",
"param": "167.128",
"step": "0/to/240/by/12",
"stream": "oper",
"time": "00",
"type": "fc"
},
"target.grib")
from ecmwfapi import ECMWFService
server = ECMWFService("mars")
server.execute(
{
"class": "od",
"date": "20150101",
"expver": "1",
"levtype": "sfc",
"param": "167.128",
"step": "0/to/240/by/12",
"stream": "oper",
"time": "00",
"type": "fc"
},
"target.grib")
2024-02-29 15:49:59 ECMWF API python library 1.6.3 2024-02-29 15:49:59 ECMWF API at https://api.ecmwf.int/v1 2024-02-29 15:49:59 Welcome Max Mustermann 2024-02-29 15:50:00 In case of problems, please check https://confluence.ecmwf.int/display/WEBAPI/Web+API+FAQ or contact servicedesk@ecmwf.int 2024-02-29 15:50:00 Request submitted 2024-02-29 15:50:00 Request id: 65e09998778174dbb7b3479f 2024-02-29 15:50:00 Request is submitted 2024-02-29 15:50:01 Request is queued 2024-02-29 15:50:07 Calling 'nice mars /tmp/20240229-1450/ff/tmp-_mars-BMsk2q.req' 2024-02-29 15:50:07 Forcing MIR_CACHE_PATH=/data/ec_coeff 2024-02-29 15:50:07 mars - WARN - 2024-02-29 15:50:07 mars - WARN - 2024-02-29 15:50:07 MIR environment variables: 2024-02-29 15:50:07 MIR_CACHE_PATH=/data/ec_coeff 2024-02-29 15:50:07 MIR_LSM_NAMED=1km.climate.v013 2024-02-29 15:50:07 Using MARS binary: /usr/local/apps/mars/versions/6.33.16.2/bin/mars.bin 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - Welcome to MARS 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - MARS Client build stamp: 20231005152332 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - MARS Client bundle version: 6.33.16.2 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - package mars-client version: 6.33.16 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - package mir version: 1.18.0 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - package odc version: 1.4.6 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - package fdb version: 5.11.23 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - package metkit version: 1.10.15 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - package eckit version: 1.24.4 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - package eccodes version: 2.32.0 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - Maximum retrieval size is 75.00 G 2024-02-29 15:50:07 retrieve,stream=oper,levtype=sfc,param=167.128,padding=0,step=0/to/240/by/12,expver=1,time=00,date=20150101,type=fc,class=odmars - INFO - 20240229.145002 - Automatic split on dates is on 2024-02-29 15:50:07 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - Processing request 1 2024-02-29 15:50:07 2024-02-29 15:50:07 RETRIEVE, 2024-02-29 15:50:07 CLASS = OD, 2024-02-29 15:50:07 TYPE = FC, 2024-02-29 15:50:07 STREAM = OPER, 2024-02-29 15:50:07 EXPVER = 0001, 2024-02-29 15:50:07 REPRES = GG, 2024-02-29 15:50:07 LEVTYPE = SFC, 2024-02-29 15:50:07 PARAM = 167.128, 2024-02-29 15:50:07 TIME = 0000, 2024-02-29 15:50:07 STEP = 0/12/24/36/48/60/72/84/96/108/120/132/144/156/168/180/192/204/216/228/240, 2024-02-29 15:50:07 DOMAIN = G, 2024-02-29 15:50:07 PADDING = 0, 2024-02-29 15:50:07 DATE = 20150101 2024-02-29 15:50:07 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - Web API request id: 65e09998778174dbb7b3479f 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - Requesting 21 fields 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - Setting SO_SNDBUF to 33554432 (32.00 M) 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - Current value is 8192 (8.00 K) 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - Setting SO_RCVBUF to 33554432 (32.00 M) 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - Current value is 43690 (42.67 K) 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - Calling mars on 'fdbprod', local port is 57169 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - Server task is 447 [ATOS FDB] 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - Retrieving from FDB [ATOS FDB] 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - Looking up FDB indexes: 0.00488 second elapsed, 0.003801 second cpu [ATOS FDB] 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - Calling mars on 'marsod-core', local port is 52683 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - Server task is 390 [marsod] 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - Request cost: 21 fields, 85.7978 Mbytes online, nodes: mvr004 mvr006 [marsod] 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - The efficiency of your requests in the last 12 hours is 100% [marsod] 2024-02-29 15:50:07 mars - INFO - 20240229.145002 - Transfering 89965512 bytes 2024-02-29 15:50:07 mars - INFO - 20240229.145004 - 21 fields retrieved from 'marsod' 2024-02-29 15:50:07 mars - INFO - 20240229.145004 - Request time: wall: 2 sec 2024-02-29 15:50:07 mars - INFO - 20240229.145004 - Visiting marsod: wall: 2 sec 2024-02-29 15:50:07 mars - INFO - 20240229.145004 - Read from network: 85.80 Mbyte(s) in 2 sec [53.45 Mbyte/sec] 2024-02-29 15:50:07 mars - INFO - 20240229.145004 - Writing to target file: 85.80 Mbyte(s) in < 1 sec [220.00 Mbyte/sec] 2024-02-29 15:50:07 mars - INFO - 20240229.145004 - Memory used: 40.92 Mbyte(s) 2024-02-29 15:50:07 mars - INFO - 20240229.145004 - No errors reported 2024-02-29 15:50:07 Process '['nice', 'mars', '/tmp/20240229-1450/ff/tmp-_mars-BMsk2q.req']' finished 2024-02-29 15:50:07 Request is complete 2024-02-29 15:50:07 Transfering 85.7978 Mbytes into target.grib 2024-02-29 15:50:07 From https://apps.ecmwf.int/api/streaming/private/blue/02/20240229-1450/d9/_mars-bol-webmars-private-svc-blue-003-4a73a881a8d5eead47db9eff2f9935a4-rDzHit.grib 2024-02-29 15:50:13 Transfer rate 14.1941 Mbytes/s 2024-02-29 15:50:13 Done
In [9]:
Copied!
!pip -q install --user cfgrib
!pip -q install --user cfgrib
In [1]:
Copied!
import xarray as xr
ds = xr.open_dataset('target.grib', engine='cfgrib')
import xarray as xr
ds = xr.open_dataset('target.grib', engine='cfgrib')
In [3]:
Copied!
print(ds)
print(ds)
<xarray.Dataset>
Dimensions: (step: 21, values: 2140702)
Coordinates:
number int64 ...
time datetime64[ns] ...
* step (step) timedelta64[ns] 0 days 00:00:00 ... 10 days 00:00:00
surface float64 ...
latitude (values) float64 ...
longitude (values) float64 ...
valid_time (step) datetime64[ns] ...
Dimensions without coordinates: values
Data variables:
t2m (step, values) float32 ...
Attributes:
GRIB_edition: 1
GRIB_centre: ecmf
GRIB_centreDescription: European Centre for Medium-Range Weather Forecasts
GRIB_subCentre: 0
Conventions: CF-1.7
institution: European Centre for Medium-Range Weather Forecasts
history: 2024-02-29T15:55 GRIB to CDM+CF via cfgrib-0.9.1...
In [ ]:
Copied!
Last update:
February 29, 2024
Created: February 29, 2024
Created: February 29, 2024