Author: Koen Hufkens
Date: 2023-01-19

# ADS functionality

#### Koen Hufkens

#### 2023-01-19

## Downloading Data Sets from Copernicus’s Atmosphere Data Store

Copernicus.eu provides a set of interesting data sets for research, education, and applied earth sciences on their Atmospheric Data Store (ADS) as part of the Copernicus Atmospheric Monitoring Service (CAMS). These data provide consistent information on the atmosphere anywhere in the world.

## Before Downloading Data

Before you will be able to download any data you need to get a free personal account.

- Register yourself for ADS services
- Terms and conditions

To retrieve data `ecmwfr` use `wf_set_key()`to add the login details to your local keyring (requires the R package `keyring`).

Once you are in possession of your personal user (namely your user ID and a secret key) `ecmwfr` allows to send requests to CDS and/or download the data.

## The Request Syntax

ADS data retrievals are based on a `list` object which specifies the data set to be downloaded. These definitions are called `requests` (for those who are familiar with mars: these are basically mars requests). A `request` defined the type of the `dataset`, the `variables` to be downloaded, the time period, output `format`, `target` location, a custom `area` extent, and other details.

The request syntax is available for a range of different CDS data sets. Check the ADS Dataset website to see a list of available datasets and to check whether API requests are allowed or not (go to Download Data, select some data, show request by clicking Show API Request (red button, bottom of page).

The conversion from a python based string to the list format can be automated if you use the RStudio based Addin. By selecting and using Addin -> python to list (or ‘Mars to list’) you dynamically convert queries copied from either ECMWF or CDS based services.

### A CAMS example

    # Specify the data set
    request <- list(
      date = "2003-01-01/2003-01-01",
      format = "netcdf",
      variable = "particulate_matter_2.5um",
      time = "00:00",
      dataset_short_name = "cams-global-reanalysis-eac4",
      target = "particulate_matter.nc"
    )

This request downloads global re-analysis data on particulate matter on January first 2003.

The data set as specified above can be downloaded calling the `wf_request` function:

    # Start downloading the data, the path of the file
    # will be returned as a variable (ncfile)
    file <- wf_request(
      user     = "2345",   # user ID (for authentification)
      request  = request,  # the request
      transfer = TRUE,     # download the file
      path     = "."       # store data in current working directory
      )

Depending on the request (the amount of data you are asking for) the request function may take a while! Please note: if you try to download larger amounts of data it is suggested to split the data sets, e.g., download year-by-year, or even month-by-month, if you are trying to download several variables/fields.

Once the retrieval has finished you should now be the owner of a NetCDF containing the requested information located in the current working directory, called `particulate_matter.nc`.

Quick check:

    # Open NetCDF file and plot the data
    # (trap read error on mac - if gdal netcdf support is missing)
    r <- try(terra::rast(ncfile))

    if(!inherits(r, "try-error")) {
      terra::plot(log(rotate(r)),
                  main = "CAMS reanalysis data (particulate matter 2.5u)",
                  col = rev(heat.colors(100)))
      maps::map("world", add = TRUE)
    }