Multi-Site simulations using run_multisite_LWFB90()

# Multi-Site simulations using `run_multisite_LWFB90()`

## Introduction

In the previous vignette ‘Multi-run simulations in LWFBrook90R’, we learned how to make multiple simulations using a set of variable model parameters using the function `run_multi_LWFB90()`. To simulate a set of different sites with different soil, climate and vegetation input, we can use the function `run_multisite_LWFB90()` that is the subject of this vignette.

    library(LWFBrook90R)
    library(data.table)
    data("slb1_meteo")
    data("slb1_soil")
    soil <- cbind(slb1_soil, hydpar_wessolek_tab(texture = slb1_soil$texture))

## List input for `soil`, `climate` and `param_b90`

The function `run_multisite_LWFB90()` runs through lists of `param_b90`, `climate`, and `soil`-objects, and evaluates the specified parameter sets for each of the soil/climate combinations. To demonstrate its usage, we define two parameter sets, that we want to run on three different sites (i.e. unique combinations of climate and soil). We include the two parameter sets in a list `parms_l`:

    parms_beech <- set_paramLWFB90(maxlai = 6)
    parms_spruce <- set_paramLWFB90(maxlai = 4.5, winlaifrac = 0.8)
    parms_l <- list(beech = parms_beech, spruce = parms_spruce)

We pretend that the three sites all have individual climates and soils, and set up lists for soil and climate input:

    soils_l <- list(soil1 = soil, soil2 = soil, soil3 = soil)
    climates_l <- list(clim1 = slb1_meteo, clim2 = slb1_meteo, clim3 = slb1_meteo)

Now we can run a small example:

    startdate <- as.Date("2002-06-01")
    enddate <- as.Date("2002-06-30")

    msite_run1 <- run_multisite_LWFB90(
      options_b90 = set_optionsLWFB90(startdate = startdate, enddate = enddate),
      param_b90 = parms_l,
      climate = climates_l,
      soil = soils_l,
      cores = 2)

The results are returned as a named list of single run objects, with their names being concatenated from the names of the input list entries holding the individual `param_b90`, `climate`, and `soil` input objects:

    str(msite_run1, max.level = 1)
    #> List of 6
    #>  $ clim1 soil1 beech :List of 5
    #>  $ clim1 soil1 spruce:List of 5
    #>  $ clim2 soil2 beech :List of 5
    #>  $ clim2 soil2 spruce:List of 5
    #>  $ clim3 soil3 beech :List of 5
    #>  $ clim3 soil3 spruce:List of 5

## Data management (ii): A function as `climate`-argument

The function `run_multisite_LWFB90()` can easily be set up to run a few dozens of sites with individual climate data. However, simulating thousands of sites can easily cause errors, because such a large list of `climate` data.frames might overload the memory of a usual desktop computer. Fortunately, it is possible to pass a function instead of a data.frame as `climate`-argument to `run_LWFB90()`. Such a function can be used to create the `climate`-data.frame from a file or database-connection within `run_LWFB90()` or `run_multisite_LWFB90()` on the fly.

For `run_LWFB90()`, we can simply provide arguments to the function via the `...`-placeholder. For `run_multisite_LWFB90()`, we need to pass arguments to a `climate`-function (possibly with individual values for individual site, e.g. a file name) via the `climate_args`-argument.

To demonstrate this mechanism, we write three files with climatic data to a temporary location, from where we will read them back in later:

    tdir <- tempdir()
    fnames <- paste0(tdir, "/clim", 1:3, ".csv")
    lapply(fnames, function(x) {
      write.csv(slb1_meteo[year(slb1_meteo$dates) == 2002,],
                file = x, row.names = FALSE)
    })

For testing, we perform a single run with `run_LWFB90()` and use the `fread` function from the ‘data.table’-package as `climate`-argument. The function reads text-files, and takes a `file` name as argument that we include in the call. It points to the first of our three climate files:

    srun <- run_LWFB90(
      options_b90 = set_optionsLWFB90(startdate = startdate, enddate = enddate),
      param_b90 = set_paramLWFB90(),
      soil = soil,
      climate = fread,
      file = fnames[1],
      rtrn.input = FALSE)

The same construct basically works with the function `run_multisite_LWFB90()`. The only difference to single-run simulations is that the arguments for the function have to be specified in a named list of lists with function arguments, one sub-list for each site. We set it up as follows:

    clim_args <- list(climfromfile1 = list(file = fnames[1]),
                      climfromfile2 = list(file = fnames[2]),
                      climfromfile3 = list(file = fnames[3]))

Now we call `run_multisite_LWFB90()`, and set up the function `fread` as `climate`-parameter. Our list of lists with individual arguments for `fread` is passed to the function via `climate_args`:

    msite_run2 <- run_multisite_LWFB90(
      options_b90 = set_optionsLWFB90(startdate = startdate, enddate = enddate),
      param_b90 = parms_l,
      soil = soils_l,
      climate = fread,
      climate_args = clim_args,
      cores = 2)

We simulated two parameter sets using three different climate/soil combinations:

    str(msite_run2, max.level = 1)
    #> List of 6
    #>  $ climfromfile1 soil1 beech :List of 5
    #>  $ climfromfile1 soil1 spruce:List of 5
    #>  $ climfromfile2 soil2 beech :List of 5
    #>  $ climfromfile2 soil2 spruce:List of 5
    #>  $ climfromfile3 soil3 beech :List of 5
    #>  $ climfromfile3 soil3 spruce:List of 5

The names of the climate used in the result names are now coming from the top-level names of our list `clim_args`, because we used a function as `climate`-argument. The function `fread` is evaluated directly within `run_multisite_LWFB90()`, and is not passed to `run_LWFB90()`, because otherwise it would have been evaluated for each single-run simulation. In this way, `fread` is evaluated only three times for in total six simulations which saves us some execution time, in case we want to simulate multiple parameter sets using the same climatic data.

## Multi-site simulation: Input from file, output to file

Now that we learned how to use a function as climate input, we can combine this input facility with an `output_fun` that writes the simulation results to a file. To do so, we extend our output function from the previous vignette ‘Multi-run simulations in LWFBrook90R’ so that it writes the aggregated results to a file in a specified directory. The file name is constructed from the names of the current soil, climate, and parameter object, which are passed automatically from `run_multisite_LWFB90()` to `run_LWFB90()` as character variables `soil_nm`, `clim_nm`, and `param_nm`. In this way, the names of currently processed input objects are accessible to `output_fun`-functions within `run_LWFB90()`.

    output_function <- function(x, tolayer, basedir = getwd(),
                                soil_nm, clim_nm, param_nm ) {
      # file-name
      filenm = file.path(basedir, paste(soil_nm, clim_nm, param_nm, sep = "_"))

      # aggregate SWAT
      swat_tran <- x$layer_output[which(nl <= tolayer),
                                 list(swat = sum(swati)),
                                 by  = list(yr, doy)]
      #add transpiration from EVAPDAY.ASC
      swat_tran$tran <- x$output$tran

      # get beginning and end of growing season from input parameters
      vpstart <- x$model_input$param_b90$budburstdoy
      vpend <- x$model_input$param_b90$leaffalldoy
      swat_tran <- merge(swat_tran,
                         data.frame(yr = unique(swat_tran$yr),
                                    vpstart, vpend), by = "yr")
      # mean swat and tran sum
      swattran_vp <- swat_tran[doy >= vpstart & doy <= vpend,
                list(swat_vp_mean = mean(swat), tran_vp_sum = sum(tran)), by = yr]

      write.csv(swattran_vp, file = paste0(filenm, ".csv"))
    }

Now we can run the simulations, with climate data coming from files, and the results being written to file our temporary directory `tdir`:

    msite_run3 <- run_multisite_LWFB90(
      options_b90 = set_optionsLWFB90(startdate = startdate, enddate = enddate),
      param_b90 = parms_l,
      soil = soils_l,
      climate = fread,
      climate_args = clim_args,
      rtrn_input = FALSE, rtrn_output = FALSE,
      output_fun = output_function,
      tolayer = 15,
      basedir = tdir,
      cores = 2)

After the simulation has finished, we can list the files and see that our attempt was successful:

    list.files(tdir, pattern = "csv")
    #> [1] "clim1.csv"                      "clim2.csv"
    #> [3] "clim3.csv"                      "soil1_climfromfile1_beech.csv"
    #> [5] "soil1_climfromfile1_spruce.csv" "soil2_climfromfile2_beech.csv"
    #> [7] "soil2_climfromfile2_spruce.csv" "soil3_climfromfile3_beech.csv"
    #> [9] "soil3_climfromfile3_spruce.csv"

We can also use database connection objects instead of files to read climate data and save simulation results. For the input of climate data, connection objects can be defined in advance, and passed directly to the `climate`-function. However, this does not work for `output_fun` in a parallel setting like in `run_multisite_LWFB90()` or `run_multi_LWFB90()`, because file or database connections in R are not exported to parallel workers. Connections therefore have to be set up (and closed again) within an `output_fun`-function.