# Temporal disaggregation and Benchmarking methods based on ‘JDemetra+’ 3.x

Abstract

The `rjd3bench` package provides a variety of methods for temporal disaggregation, interpolation, benchmarking, reconciliation and calendarization. It is part of the interface to the ‘JDemetra+’ 3.x time series analysis software and incorporates statistical methods described in the latest European Statistical System (ESS) guidelines on temporal disaggregation, benchmarking, and reconciliation (2018 edition). Most algorithms implemented in the package rely on an equivalent state‑space representation of the underlying model or mathematical problem, where computational efficiency is further improved by replacing matrix operations with functional forms, thereby enabling highly efficient estimates.

# Introduction

The methods implemented in the package `rjd3bench` intend to bridge the gap when there is a lack of high frequency time series or when there are temporal and/or contemporaneous inconsistencies between the high frequency series and the corresponding low frequency series. Although this can be an issue in any fields of research dealing with time series, methods of temporal disaggregation, interpolation, benchmarking, reconciliation and calendarization are often encountered in the production of official statistics. For example, National Accounts are often compiled according to two frequencies of production: annual series, the low frequency data, based on precise and detailed sources and quarterly series, the high frequency data, which usually rely on less accurate sources but give information on a timelier basis. In such case, the use of temporal disaggregation, benchmarking, and reconciliation methods can be used to achieve consistency between annual and quarterly national accounts over time.

The package `rjd3bench` is an interface to the highly efficient algorithms and modeling developed in the official ‘JDemetra+’ 3.x time series software. It provides a wide variety of methods, included those suggested in the ESS guidelines on temporal disaggregation, benchmarking and reconciliation (Eurostat, 2018).

# Set-up & Data

We illustrate the various methods using two datasets:

- The Retail dataset contains monthly figures over retail activity of various categories of goods and services from 1992 to 2010.
- The qna_data is a list of two datasets. The first data set ‘B1G_Y_data’ includes three annual benchmark series which are the Belgian annual value added on the period 2009-2020 in chemical industry (CE), construction (FF) and transport services (HH). The second data set ‘TURN_Q_data’ includes the corresponding quarterly indicators which are (modified) production indicators derived from VAT statistics and covering the period 2009Q1-2021Q4.

    library("rjd3bench")
    Retail <- rjd3toolkit::Retail
    qna_data <- rjd3bench::qna_data

# Temporal disaggregation and interpolation methods

Temporal disaggregation and interpolation are related to each other (and to benchmarking). They share similar properties and methods but they differ in their purpose.

Temporal disaggregation is usually associated with flow series. The purpose is to break down a low frequency time series into a higher frequency time series, where the low frequency series correspond to the sum or average of the corresponding higher frequency series.

Interpolation usually arises in the context of stock series. The purpose is to estimate missing values at time points between the known data points given by the low frequency series. For example, an annual series can typically corresponds to the fourth quarter or twelfth month of a quarterly or a monthly series and the purpose would be to obtain estimates for the other quarters or months.

For the Chow-Lin method and its variants, a separate function is considered for temporal disaggregation and interpolation. This is not the case of the other methods where the two are integrated in a single function.

Furthermore, there are raw version available for the functions `temporal_disaggregation()` and `temporal_interpolation()` related to Chow-Lin method and its variants. The functions `temporal_disaggregation_raw()` and `temporal_interpolation_raw()` enable the user to deal with atypical frequency data and with any frequency ratio. Note that for benchmarking, a function `denton_raw()` is also available.

## Chow-Lin, Fernandez and Litterman

Eurostat (2018) recommends the use of regression-based models for the purpose of temporal disaggregation. Among them, we retrieve the Chow-Lin method and its variants Fernandez and Litterman.

Let \(Y_T\), \(T=1,...,m\), and \(x_t\), \(t=1,...,n\), be, respectively the observed low frequency benchmark and the high-frequency indicator of an unknown high frequency variable \(y_t\). Chow-Lin, Fernandez and Litterman can be all expressed with the same equation, but with different models for the error term: \[ y_t = x_t\beta+u_t \] where

\(u_t = \phi u_{t-1} + \epsilon_t\), with \(|\phi| < 1\) (Chow-Lin),

\(u_t = u_{t-1} + \epsilon_t\) (Fernandez),

\(u_t = u_{t-1} + \phi(\Delta u_{t-1}) + \epsilon_t\), with \(|\phi| < 1\) (Litterman)

The temporal constraint is: \[ Y = Cy, \] where \(C = I_m \otimes c\), \(c\) is a row vector of size \(s\) which is the frequency ratio between the disaggregated/interpolated series and the low frequency benchmark. The distinction between temporal disaggregation and interpolation lies in the definition of this vector c:

- Temporal disaggregation: \(c = [1,1,...,1]\) for aggregation (e.g., flow variables) and \(c = [1/s,1/s,...,1/s]\) for average conversion (e.g., indexes)
- Interpolation: \(c = [0,0,...,1]\) when the low frequency series corresponds to the last value of the interpolated series, \(c = [1,0,...,0]\) when it’s the first value, etc. (e.g., stock variables)

While \(x_t\) is observed in high frequency, \(y_t\) is only observed in low frequency, and therefore the number of effective observations to estimate the parameters are the number of observations in the low-frequency benchmark.

The regression-based Chow-Lin method and its variants Fernandez and Litterman can be called with the functions `temporal_disaggregation()` or `temporal_interpolation()`. Those two functions require a ts object as input series and only deal with usual frequency conversion (i.e. annual to quarterly, annual to monthly or quarterly to monthly). Alternatively, the functions `temporal_disaggregation_raw()` and `temporal_interpolation_raw()` require a numeric vector as input series and extend the previous functions in a way that they can deal with atypical frequency series and with any frequency ratio.

The output of the functions `temporal_disaggregation()`, `temporal_interpolation()`, `temporal_disaggregation_raw()` and `temporal_interpolation_raw()` contains the most important information about the regression including the estimates of model coefficients and their covariance matrix, the decomposition of the disaggregated/interpolated series and information about the residuals. A print() and summary() functions can be applied on the output object. The plot() function, which displays the decomposition of the disaggregated/interpolated series between regression and smoothing effect, can be applied on the output object of the functions `temporal_disaggregation()` and `temporal_interpolation()`.

In practice, Chow-Lin and its variants Fernandez and Litterman are estimated based on an equivalent state space representation of the model. By default, for Fernandez and Litterman, a diffuse initialization is considered for the estimation of the initial values of the states so that those integrate the estimate of the constant term. The latter is thus ‘hidden’ and does not appear explicitly in the output. To make the estimates of the constant term visible - thus recovering the usual output from the classic formulation of the model - one option is to change the argument `zeroinitialization = TRUE` with `constant=TRUE` when calling the function.

    # Example 1: TD using Chow-Lin to disaggregate annual value added in construction sector using a quarterly indicator
    Y <- ts(qna_data$B1G_Y_data[, "B1G_FF"], frequency = 1, start = c(2009, 1))
    x <- ts(qna_data$TURN_Q_data[, "TURN_INDEX_FF"], frequency = 4, start = c(2009, 1))
    td <- rjd3bench::temporal_disaggregation(Y, indicators = x)

    y <- td$estimation$disagg # the disaggregated series
    print(td)
    #> Model: Ar1
    #>             coef         se         t
    #> const 2078.60371 234.756347  8.854302
    #> var-1   24.25466   1.994171 12.162783
    #>
    #> Use summary() for more details.
    #> Use plot() to see the decomposition of the disaggregated series.

    # Example 2: interpolation using Fernandez without indicator when the last value (default) of the interpolated series is the one consistent with the low frequency series.
    Y <- rjd3toolkit::aggregate(rjd3toolkit::Retail$RetailSalesTotal, 1)
    ti <- temporal_interpolation(Y, indicators = NULL, model = "Rw", freq = 4, nfcsts = 2)
    y <- ti$estimation$interp # the interpolated series

    # Example 3: TD of atypical frequency data using Fernandez with an offset of 1 period
    Y <- c(500,510,525,520)
    x <- c(97,
           98, 98.5, 99.5, 104, 99,
           100, 100.5, 101, 105.5, 103,
           104.5, 103.5, 104.5, 109, 104,
           107, 103, 108, 113, 110)
    td_raw <- temporal_disaggregation_raw(Y, indicators = x, startoffset = 1,  model = "Rw", freqratio = 5)
    y <- td_raw$estimation$disagg # the disaggregated series

    # Example 4: interpolation of atypical frequency data using Fernandez without offset, when the first value of the interpolated  series is the one consistent with the low frequency series.
    Y <- c(500,510,525,520)
    x <- c(490, 492.5, 497.5, 520, 495,
           500, 502.5, 505, 527.5, 515,
           522.5, 517.5, 522.5, 545, 520,
           535, 515, 540, 565, 550,
           560)
    ti_raw <- temporal_interpolation_raw(Y, indicators = x,  model = "Rw", freqratio = 5, obsposition = 1)
    y <- ti_raw$estimation$interp

## Model-based Denton

Denton method and variants are usually expressed in mathematical terms as a constrained minimization problem. For example, the widely used Denton proportional first difference (PFD) method is usually expressed as follows: \[ min_{y_t}\sum^n_{t=2}\biggl[\frac{y_t}{x_t}-\frac{y_{t-1}}{x_{t-1}}\biggr]^2 \] subject to the temporal constraint (flow variables) \[ \sum_{t} y_t = Y_T \] where \(y_t\) is the value of the estimate of the high frequency series at period t, \(x_t\) is the value of the high frequency indicator at period t and \(Y_T\) is the value of the low frequency series (i.e. the benchmark series) at period T.

Equivalently, the Denton PFD method can also be expressed as a statistical model considering the following state space representation

\[ \begin{aligned} y_t &= \beta_t x_t \\ \beta_{t+1} &= \beta_t + \varepsilon_t \qquad \varepsilon_t \sim {\sf NID}(0, \sigma^2_{\varepsilon}) \end{aligned} \]

where the temporal constraints are taken care of by considering a cumulated series \(y^c_t\) instead of the original series \(y_t\). Hence, the last high frequency period (for example, the last quarter of the year) is observed and corresponds to the value of the benchmark. The value of the other periods are initially defined as missing and estimated by maximum likelihood.

This alternative representation of Denton PFD method is interesting as it allows more flexibility. We might now include outliers - namely, level shift(s) in the Benchmark to Indicator ratio - that could otherwise induce undesirable wave effects. Outliers and their intensity are defined by changing the value of the innovation variances. There is also the possibility to freeze the disaggregated series at some specific period(s) or prior a certain date by fixing the high-frequency BI ratio(s). Following the principle of movement preservation inherent to Denton, the model-based Denton PFD method constitutes an interesting alternative for temporal disaggregation, interpolation and benchmarking. Here is a link to a presentation on the subject which include some comparison with the regression-based methods for temporal disaggregation.

The model-base Denton method can be applied with the `denton_modelbased()` function. The output of the `denton_modelbased()` function contains information about the disaggregated/interpolated series and the BI ratio as well as their respecting errors making it possible to construct confidence intervals. The print(), summary() and plot() functions can also be applied on the output object.The plot() function displays the disaggregated series and the BI ratio together with their respective 95% confidence interval.

    # Example: Use of model-based Denton for temporal disaggregation
    Y <- ts(qna_data$B1G_Y_data[, "B1G_FF"], frequency = 1, start = c(2009, 1))
    x <- ts(qna_data$TURN_Q_data[, "TURN_INDEX_FF"], frequency = 4, start = c(2009, 1))
    td_mbd <- denton_modelbased(Y, x, outliers = list("2020-01-01" = 100, "2020-04-01" = 100))

    y_mbd <- td_mbd$estimation$disagg
    print(td_mbd)
    #> Available output:
    #> [1] "disagg"   "edisagg"  "biratio"  "ebiratio"
    #>
    #> Use summary() for more details.
    #>  Use plot() to see the disaggregated series and BI ratio with their respective confidence interval.

## Autoregressive Distributed Lag (ADL) Models

Based on Proietti (2005), we consider a first order Autoregressive Distributed Lag model, or ADL(1,1), which takes the form:

\[ y_t = \phi y_{t-1} + m + gt + x_t'\beta_0 + x_{t-1}'\beta_1 + \varepsilon_t \qquad \varepsilon_t \sim {\sf NID}(0, \sigma^2) \qquad (1) \] subject to the temporal constraint (flow variables) \[ \sum_{t} y_t = Y_T \] where \(y_t\) is the value of the estimate of the high frequency series at period t, \(x_t\) is the value of the high frequency indicator at period t and \(Y_T\) is the value of the low frequency series (i.e. the benchmark series) at period T.

The ADL model nests the Chow-Lin model and its variants Fernandez and Litterman (for Litterman, the ADL model must be formulated in the first differences of the dependent and explanatory variables).

Recall the Chow-Lin model \[ y_t = x_t\beta + u_t \\ u_t = \phi u_{t-1} + \varepsilon_t \] Combine it into a single equation and substitute for \(u_{t-1}\) \[ y_t = x_t\beta + \phi (y_{t-1} - x_{t-1}\beta) + \varepsilon_t \] So, the ADL model corresponds to the Chow-Lin model if \[ \beta_1 = -\phi\beta_0 \qquad (2) \] and to the Fernandez model if we further assumes that \(\phi = 1\).

Recall that the Chow-Lin model relies on the strong assumption of that the disaggragated series \(y_t\) and the indicator(s) \(x_t\) are fully co-integrated (if not stationary). The main benefit of the ADL model over Chow-Lin is that it offers an extended modelling framework which accounts for the uncertainty about the existence of co-integration between \(y_t\) and \(x_t\). Hence, by nesting both the Chow-Lin and the Fernandez model, it prevents potential spurious regressions between \(y_t\) and \(x_t\).

Moreover, as explained in Proietti (2005, section 6.2), the ADL model can be an interesting option when the indicator is affected by a measurement error, since it can be showed that the model assumes that \(y_t\) is explained by a filtered version of \(x_t\), where the weights associated to \(x_t\) decline geometrically over time. Hence, in such case, the indicator’s excessive volatility won’t be reflected in the disaggregated series as it is with Chow-Lin; rather, it will be smoothed.

The ADL disaggregation method can be called with the `adl_disaggregation()` function. The ‘xar’ parameter allows you to set constraints on the coefficients of the lagged regression variables. As an alternative to the default value “FREE” (no constraint), setting the parameter to “SAME” corresponds to imposing the constraint (2). Note that those constraints, and therefore also the relevance of switching from an ADL model to a simpler Chow-Lin or Fernandez model, can be assessed by computing the Likelihood Ratio (LR) statistic (cfr. example below). Finally, you can set the parameter to “NONE”, which means an ADL(1,0) model is considered (no lag on \(x_t\)) which corresponds to the method suggested by Santos Silva and Cardoso (2001).

The output of the function `adl_disaggregation()` contains the most important information about the regression including the estimates of model coefficients and their covariance matrix, the disaggregated series and information about the residuals. A print(), summary() and plot() functions can be applied on the output object.

In practice, the ADL model is estimated based on an equivalent state space representation.

    # Example: Use of ADL models for temporal disaggregation

    Y <- ts(qna_data$B1G_Y_data[, "B1G_FF"], frequency = 1, start = c(2009, 1))
    x <- ts(qna_data$TURN_Q_data[, "TURN_INDEX_FF"], frequency = 4, start = c(2009, 1))

    ## 1. without constraints

    td_adl <- adl_disaggregation(Y, indicators = x, xar = "FREE")
    y <- td_adl$estimation$disagg # the disaggregated series
    print(td_adl)
    #> Model: FREE
    #>        coef        se         t
    #> 1 591.96748 58.844898 10.059793
    #> 2  38.85308  6.334725  6.133349
    #> 3 -31.75200  6.194977 -5.125442
    #>
    #> Use summary() for more details.
    #> Use plot() to see the decomposition of the disaggregated series.

    ## 2. with constraints

    ### b1 = -phi * b0 (~ Chow-Lin)
    td_adl_constr_1 <- adl_disaggregation(Y, indicators = x, xar = "SAME")

    ### phi = 1 and b1 = -b0 (~ Fernandez)
    td_adl_constr_2 <- adl_disaggregation(Y, constant = FALSE, indicators = x, xar = "SAME", phi = 1, phi.fixed = TRUE)

    ### b1 = 0 (~ Santos Silva-Cardoso)
    td_adl_constr_3 <- adl_disaggregation(Y, indicators = x, xar = "NONE")

    ## LR test for assessing constraint(s)
    LR_stat <- -2 * (td_adl_constr_1$likelihood$ll - td_adl$likelihood$ll) # -> H0 not rejected. Chow-Lin specification is supported here.

## Reverse regression

Bournay and Laroque (1979) proposed an alternative regression-based approach for temporal disaggregation. Unlike previous models, where the target series (the disaggregated or interpolated series) was defined as the dependent variable, this approach flips the regression and treats the high-frequency indicator as the dependent variable and the target series as the independent variable.

Let \(Y_T\), \(T=1,...,m\), and \(x_t\), \(t=1,...,n\), be, respectively the observed low frequency benchmark and a single high-frequency indicator of the unknown high frequency target variable \(y_t\). The model is defined as:

\[ x_t = a + by_t + u_t \\ u_t = \phi u_{t-1} + \varepsilon_t \] subject to the temporal constraint (flow variables) \[ \sum_{t} y_t = Y_T \] The choice of which variable is dependent and which is independent is far from arbitrary. It changes the assumptions and the interpretation of the model, and the outcome may be very different too. In particular, if the fit between the benchmark and the indicator is globally poor, the smoothing part of the Chow-Lin model usually becomes more influential (if constant > 0), resulting in a disaggregated series that is smoother, meaning that the indicator’s movements are dampened. Conversely, under the reverse model above, we can have a high estimate for parameter \(a\) and a low estimate for \(b\), which may result in a disaggregated series being more volatile, effectively amplifying the indicator’s movements.

The reverse regression method can be called with the `temporaldisaggregationI()` function. The output of the function `temporaldisaggregationI()` contains the disaggregated series as well as the estimates of the model coefficients. A print(), summary() and plot() functions can be applied on the output object.

    Y <- ts(qna_data$B1G_Y_data[, "B1G_FF"], frequency = 1, start = c(2009, 1))
    x <- ts(qna_data$TURN_Q_data[, "TURN_INDEX_FF"], frequency = 4, start = c(2009, 1))
    td_rv <- temporaldisaggregationI(Y, indicator = x)

    y_rv <- td_rv$estimation$disagg # the disaggregated series
    print(td_rv)
    #>       coef
    #> a -72.0977
    #> b   0.0385
    #>
    #> Use summary() for more details.
    #> Use plot() to visualize the disaggregated series.

    # Comparison with Chow-Lin
    td_cl <- temporal_disaggregation(Y, indicators = x)
    y_cl <- td_cl$estimation$disagg
    stats::ts.plot(y_rv, y_cl, gpars=list(col=c("red", "blue"), xaxt="n", main="Reverse regression vs Chow-Lin"))
    legend("topleft",c("Reverse regression", "Chow-Lin"), lty = c(1,1), col=c("red", "blue"))

# Benchmarking methods

The benchmarking problem arises when time series data for the same target variable are measured at two different frequencies with different levels of accuracy. Typically, the high frequency series is less reliable than the low frequency series, referred to as the benchmark. Thus, benchmarking is the process of adjusting the high frequency series to make it consistent with the more reliable low frequency series.

As for the temporal disaggregation/interpolation method Chow-Lin and its variants, a raw version of the `denton()` benchamrking method is made available to the user. The function `denton_raw()` enables the user to deal with atypical frequency data and with any frequency ratio.

## Denton

Denton methods relies on the principle of movement preservation. There exist several variants corresponding to different definitions of movement preservation: additive first difference (AFD), proportional first difference (PFD), additive second difference (ASD), proportional second difference (PSD).

The most widely used is the Denton PFD variant. Let \(Y_T\), \(T=1,...,m\), and \(x_t\), \(t=1,...,n\), be, respectively the temporal benchmarks and the high-frequency preliminary values of an unknown target variable \(y_t\). The objective function of the Denton PFD method is as follows (considering the small modification suggested by Cholette to deal with the starting conditions of the problem): \[ min_{y_t}\sum^n_{t=2}\biggl[\frac{y_t}{x_t}-\frac{y_{t-1}}{x_{t-1}}\biggr]^2 \] This objective function is minimized subject to the temporal aggregation constraints \(\sum_{t\epsilon T} y_t = Y_T\), \(T=1,...,m\) (flows variables). In other words, the benchmarked series is estimated in such a way that the “Benchmark-to-Indicator” ratio \(\frac{y_t}{x_t}\) remains as smooth as possible, which is often of key interest in benchmarking.

In the literature (see for example Di Fonzo and Marini, 2011), Denton PFD is generally considered as a good approximation of the GRP method, meaning that it preserves the period-to-period growth rates of the preliminary series. It is also argued that in many applications, Denton PFD is more appropriate than GRP method as it deals with a linear problem which is computationally easier, and does not suffer from the issues related to time irreversibility and singular objective function when \(y_t\) approaches 0 (see Daalmans et al, 2018).

Denton methods can be called with the `denton()` function which can deal with usual frequency conversion (i.e. annual to quarterly, annual to monthly or quarterly to monthly). Alternatively, the `denton_raw()` function requires a numeric vector as input series, but extends the `denton()` function in a way that it can deal with atypical frequency series and with any frequency ratio. The `denton()` and `denton_raw()` functions return the high frequency series benchmarked with the Denton method.

    # Example 1: use of Denton method for benchmarking
    Y <- ts(qna_data$B1G_Y_data[, "B1G_HH"], frequency = 1, start = c(2009, 1))

    y_den0 <- denton(t = Y, nfreq = 4) # denton PFD without high frequency series

    x <- y_den0 + rnorm(n = length(y_den0), mean = 0, sd = 10)
    y_den1 <- denton(s = x, t = Y) # denton PFD (= the default)
    y_den2 <- denton(s = x, t = Y, d = 2, mul = FALSE) # denton ASD

    # Example 2: use of of Denton method for benchmarking atypical frequency data
    Y <- c(500,510,525,520)
    x <- c(97, 98, 98.5, 99.5, 104,
           99, 100, 100.5, 101, 105.5,
           103, 104.5, 103.5, 104.5, 109,
           104, 107, 103, 108, 113,
           110)

    y_denraw <- denton_raw(x, Y, freqratio = 5) # for example, x and Y could be annual and quiquennal series respectively

## Growth rate preservation (GRP)

GRP explicitly preserves the period-to-period growth rates of the preliminary series.

Let \(Y_T\), \(T=1,...,m\), and \(x_t\), \(t=1,...,n\), be, respectively the temporal benchmarks and the high-frequency preliminary values of an unknown target variable \(y_t\). Cauley and Trager(1981) consider the following objective function:

\[ f(x) = \sum_{t=2}^{n}\left(\frac{y_t}{y_{t-1}} - \frac{x_t}{x_{t-1}}\right)^2 \] and look for values \(y_t^*\), \(t=1,...,n\), which minimize it subject to the temporal aggregation constraints \(\sum_{t\epsilon T} y_t = Y_T\), \(T=1,...,m\) (flows variables). In other words, the benchmarked series is estimated in such a way that its temporal dynamics; as expressed by the growth rates \(\frac{y_t^*}{y_{t-1}^*}\), \(t=2,...,n\), be “as close as possible” to the temporal dynamics of the preliminary series, where the “distance” from the preliminary growth rates \(\frac{x_t}{x_{t-1}}\) is given by the sum of the squared differences. (Di Fonzo, Marini, 2011)

The objective function considered by Cauley and Trager is a natural measure of the movement of a time series and as one would expect, it is usually slightly better than the Denton PFD method at preserving the movement of the series (Di Fonzo, Marini, 2011). However, unlike the Denton PFD method which deals with a linear problem, GRP solves a more difficult nonlinear problem. Furthermore, the GRP method suffers from a couple of drawbacks, which are time irreversibility and potential singularities in the objective function when \(y_{t-1}\) approaches to 0, which could lead to undesirable results (see Daalmans et al, 2018).

The standard objective function for GRP considered by Cauley and Trager and defined above means that we apply the benchmarking forward. Alternatively, we could apply it backward, which means performing the benchmarking on the reversed time series. As previsouly mentionned, this is not equivalent when using GRP method. As altenatives, Daalmans et al (2018) proposed two other objective functions for GRP (symmetric GRP and logarithmic GRP) which are “time symmetric”.

Backward GRP: \[ f(x) = \sum_{t=2}^{n}\left(\frac{y_{t-1}}{y_t} - \frac{x_{t-1}}{x_t}\right)^2 \] Symmetric GRP: \[ f(x) = \frac{1}{2} \sum_{t=2}^{n}\left(\frac{y_t}{y_{t-1}} - \frac{x_t}{x_{t-1}}\right)^2 + \frac{1}{2} \sum_{t=2}^{n}\left(\frac{y_{t-1}}{y_t} - \frac{x_{t-1}}{x_t}\right)^2 \] Logarithmic GRP: \[ f(x) = \sum_{t=2}^{n}\left(log\left(\frac{y_t}{y_{t-1}}\right) - log\left(\frac{x_t}{x_{t-1}}\right) \right)^2 \]

The GRP method, corresponding to the method of Cauley and Trager, using the solution proposed by Di Fonzo and Marini (2011), can be called with the `grp()` function. An alternative objective function as those suggested by Daalmans et al (2018) can also be considered. The `grp()` function returns the high frequency series benchmarked with the GRP method.

    # Example: use GRP method for benchmarking
    Y <- ts(qna_data$B1G_Y_data[, "B1G_HH"], frequency = 1, start = c(2009, 1))
    y_den0 <- denton(t = Y, nfreq = 4)
    x <- y_den0 + rnorm(n = length(y_den0), mean = 0, sd = 10)

    y_grpf <- grp(s = x, t = Y)
    y_grpl <- grp(s = x, t = Y, objective = "Log")

## Cubic splines

Cubic splines are piecewise cubic functions that are linked together in a way to guarantee smoothness at data points. Additivity constraints are added for benchmarking purpose and sub-period estimates are derived from each spline. When a sub-period indicator (or a preliminary series) is used, cubic splines are no longer drawn based on the low frequency data but the Benchmark-to-Indicator (BI ratio) is the one being smoothed. Sub-period estimates are then simply the product between the smoothed high frequency BI ratio and the indicator.

The method can be called through the `cubicspline()` function. The `cubicspline()` function returns the high frequency series benchmarked with cubic spline method.

    # Example: use cubic splines for benchmarking
    y_cs1 <- cubicspline(t = Y, nfreq = 4) # without high frequency series (smoothing)

    x <- y_cs1 + rnorm(n = length(y_cs1), mean = 0, sd = 10)
    y_cs2 <- cubicspline(s = x, t = Y) # with a high frequency preliminary series to benchmark

## Cholette method

Cholette method is based on a benchmarking methodology developed at Statistics Canada. It is a generalized method relying on the principle of movement preservation that encompasses other benchmarking methods. The Denton method (both the AFD and PFD variants), as well as the naive pro-rating method, emerge as particular cases of the Cholette method.

Let \(Y_T\), \(T=1,...,m\), and \(x_t\), \(t=1,...,n\), be, respectively the temporal benchmarks and the high-frequency preliminary values of an unknown target variable \(y_t\). The objective function of the Cholette method is as follows (Quenneville et al, 2006):

\[ f(x) = (1-\rho^2) \left(\frac{x_1 - y_1}{|x_1|^\lambda}\right)^2 + \sum_{t=2}^{n}\left[\left(\frac{x_t - y_t}{|x_t|^\lambda}\right) - \rho \left(\frac{x_{t-1} - y_{t-1}}{|x_{t-1}|^\lambda}\right)\right]^2 \] This objective function is minimized subject to the temporal aggregation constraints \(\sum_{t\epsilon T} y_t = Y_T\), \(T=1,...,m\) (flows variables). The method is driven by a couple of parameters:

- The adjustment model parameter \(\lambda\), \(\lambda \in \mathbb{R}\). Set \(\lambda = 0\) for an additive benchmarking model and \(\lambda = 1\) for a proportional benchmarking model. Finally, set \(\lambda = 0.5\) with \(\rho = 0\), for the naive pro-rating method.
- The smoothing parameter \(\rho\), \(0 \leq \rho \leq 1\). \(\rho\) determines the degree of movement preservation. When \(\lambda = 1\), the closer \(\rho\) is to 1, the smoother will be the ratios of the benchmarks to the corresponding totals in the preliminary series and the better the movement of the latter will be preserved.

Cholette method also provides for the possibility of considering a bias correction factor, which is the expected discrepancy between the benchmarks and the high-frequency preliminary series. The additive and multiplicative bias correction factor are estimated respectively as:

\[ \begin{aligned} b_a &= \frac{\sum_{T=1}^{m}{Y_T} - \sum_{T=1}^{m}{\sum_{t\epsilon T}x_t}}{m} \\ b_m &= \frac{\sum_{T=1}^{m}{Y_T}}{\sum_{T=1}^{m}{\sum_{t\epsilon T}x_t}} \end{aligned} \] If a bias correction factor is considered, the preliminary series is re-scaled in the objective function above: \(x_t^*\) replaces \(x_t\), where \(x_t^*=b_a+x_t\) in the additive case and \(x_t^*=b_m \times x_t\) in the multiplicative case. The rationale for considering a bias correction factor with this method is provided in Dagum and Cholette (2006, Ch. 6). It mainly impacts the observations at the end of the series that are not covered by a benchmark. In particular, when \(\rho < 1\), the Benchmark-to-Indicator ratios (BI ratios) at the end of the series converge to the bias correction factor. By default, no bias is considered, meaning that we do not expected a systematic bias between the benchmarks and the preliminary series (\(b_a = 0\) or \(b_m = 1\)).

Cholette method has been widely used to benchmark seasonally adjusted series to annual totals derived from the raw series. For this purpose, Quenneville et al (2006) argues that an undesirable feature of the Denton PFD method is that it repeats the last BI ratio for the observations at the end of the series that are not covered by a benchmark. For observations without a benchmark, the best estimate of the BI-ratio is the estimated value of the bias; so, repeating the last value is not appropriate. Instead, to obtain a smooth transition from this last BI-ratio to the bias, one can set \(\rho < 1\). For observations with a benchmark, the BI-ratios are closer to those obtained with the Denton PFD method (\(\rho = 1\)) and smoother when \(\rho \to 1\). As a pragmatic benchmarking method routinely applicable to large numbers of seasonal time series Dagum and Cholette (2006) recommend the proportional benchmarking method (\(\lambda = 1\)) with a value of \(\rho = 0.90\) for monthly series and \(\rho = 0.90^3= 0.729\) for quarterly series. An alternative would be to estimate the autocorrelation structure of the error instead of using those default values.

Cholette method can be called with the `cholette()` function. The `cholette()` function returns the high frequency series benchmarked with the Cholette method.

In practice, the benchmarked series is estimated based on an equivalent state space representation of the Cholette method described above.

    # Example: use Cholette method for benchmarking
    Y <- ts(qna_data$B1G_Y_data[, "B1G_HH"], frequency = 1, start = c(2009, 1))
    xn <- c(denton(t = Y, nfreq = 4) + rnorm(n = length(Y)*4, mean = 0, sd = 10), 5750, 5800)
    x <- ts(xn, start = start(Y), frequency = 4)

    y_cho1 <- cholette(s = x, t = Y, rho = 0.729, lambda = 1, bias = "Multiplicative")  # proportional benchmarking
    y_cho2 <- cholette(s = x, t = Y, rho = 0.729, lambda = 1) # proportional benchmarking with no bias (assuming bm=1)
    y_cho3 <- cholette(s = x, t = Y, rho = 0.729, lambda = 0, bias = "Additive")  # additive benchmarking
    y_cho4 <- cholette(s = x, t = Y, rho = 1, lambda = 1) # Denton PFD
    y_cho5 <- cholette(s = x, t = Y, rho = 0, lambda = 0.5) # pro-rating

# Reconciliation and multivariate temporal disaggregation

## Multivariate Cholette

This is a multivariate extension of the Cholette benchmarking method which can be used for the purpose of reconciliation. While standard benchmarking methods consider one target series at a time, reconciliation techniques aim to restore consistency in a system of time series with regards to both contemporaneous and temporal constraints. Reconciliation techniques are typically needed when the total and its components are estimated independently (the so-called direct approach). The multivariate Cholette method relies on the principle of movement preservation and encompasses other reconciliation methods such as the multivariate Denton method.

Let

- \(Y_{i,T}\), \(T=1,...,m\), \(i=1,...,I\), be the set of temporal benchmarks
- \(z_{k,t}\), \(t=1,...,n\), \(k=1,...,K\), be the set of contemporaneous constraints
- \(x_{i,t}\) be the high-frequency preliminary values of the set of the unknown target variables \(y_{i,t}\).

The objective function of the multivariate Cholette method is: \[ f(x) = (1-\rho^2) \sum_{i=1}^{I}\left(\frac{x_{i,1} - y_{i,1}}{|x_{i,1}|^\lambda}\right)^2 + \sum_{i=1}^{I}\sum_{t=2}^{n}\left[\left(\frac{x_{i,t} - y_{i,t}}{|x_{i,t}|^\lambda}\right) - \rho \left(\frac{x_{i,t-1} - y_{i,t-1}}{|x_{i,t-1}|^\lambda}\right)\right]^2 \] This objective function is minimized subject to

- the temporal aggregation constraints \(\sum_{t\epsilon T} y_{i,t} = Y_{i,T}\), and
- the contemporaneous constraints given by \(\sum_{j\epsilon J_k}\omega_{k,j}x_{j,t} = z_{k,t}\).

Note that consistency between the temporal aggregation constraints and the contemporaneous constraints is required (see the consistency check in the example).

The method may also be applied in the absence of temporal aggregation constraints. The contemporaneous constraints are then imposed by altering the dynamic movements of the series as little as possible. Conversely, the absence of contemporaneous constraint is less relevant, as this is just equivalent to applying the univariate Cholette method to each of the preliminary series separately. In addition to binding contemporaneous constraints, non-binding constraints can also specified by modifying their formulation in the `ccvector` argument. For example, instead of imposing a binding condition such as \(x_1+x_2+x_3=z_1\), one may express a non-binding constraint as \(x_1+x_2+x_3-z_1=0\), which allows the \(z_1\) series to be adjusted in the same manner as \(x_1\), \(x_2\) and \(x_3\) (weights may also be introduced when needed).

As in the univariate case, the multivariate Cholette method is driven by a couple of parameters:

- The adjustment model parameter \(\lambda\), \(\lambda \in \mathbb{R}\) makes the trade‑off between an additive and a proportional reconciliation model, and therefore determines how strongly the relative size of each series influences the distribution of discrepancies. Setting \(\lambda = 0\) yields a purely additive model, while values of \(\lambda\) close to 1 approximate a proportional model in which larger series absorb a larger share of the adjustment. Although \(\lambda = 1\) is also possible, it should be used with caution in a multivariate context. This is because contemporaneous constraints, combined with the fact that pure movement preservation specifies nothing about the level of the individual reconciled series, may sometimes produce substantial differences in level between the preliminary and the benchmarked series. This is especially true in the absence of temporal constraints where strong movement preservation should not be pursued during reconciliation. Finally, as in the univariate case, the naive pro-rating method corresponds to setting \(\lambda = 0.5\) with \(\rho = 0\).

- The smoothing parameter \(\rho\), \(0 \leq \rho \leq 1\). \(\rho\) determines the degree of movement preservation. When \(\lambda = 1\), the closer \(\rho\) is to 1, the smoother will be the ratios of the benchmarks to the corresponding totals in the preliminary series and the better the movement of the latter will be preserved.

The multivariate Cholette method can be called with the `multivariatecholette()` function. The `multivariatecholette()` function returns a list of benchmarked series, fulfilling both the contemporary and the temporal constraints (if any).

In practice, the benchmarked series are estimated based on an equivalent state space representation of the multivariate Cholette method described above.

    # Example: use the multivariate Cholette method for reconciliation
    x1 <- ts(c(7, 7.2, 8.1, 7.5, 8.5, 7.8, 8.1, 8.4), frequency = 4, start = c(2010, 1))
    x2 <- ts(c(18, 19.5, 19.0, 19.7, 18.5, 19.0, 20.3, 20.0), frequency = 4, start = c(2010, 1))
    x3 <- ts(c(1.5, 1.8, 2, 2.5, 2.0, 1.5, 1.7, 2.0), frequency = 4, start = c(2010, 1))

    z <- ts(c(27.1, 29.8, 29.9, 31.2, 29.3, 27.9, 30.9, 31.8), frequency = 4, start = c(2010, 1))

    Y1 <- ts(c(30.0, 30.6), frequency = 1, start = c(2010, 1))
    Y2 <- ts(c(80.0, 81.2), frequency = 1, start = c(2010, 1))
    Y3 <- ts(c(8.0, 8.1), frequency = 1, start = c(2010, 1))

    ### check consistency between temporal and contemporaneous constraints
    lfs <- cbind(Y1,Y2,Y3)
    as.numeric(rowSums(lfs) - stats::aggregate.ts(z)) # should all be 0
    #> [1] 0 0

    data_list <- list(x1 = x1, x2 = x2, x3 = x3, z = z, Y1 = Y1, Y2 = Y2, Y3 = Y3)
    tc <- c("Y1 = sum(x1)", "Y2 = sum(x2)", "Y3 = sum(x3)") # temporal constraints
    cc <- c("z = x1+x2+x3") # (binding) contemporaneous constraint
    cc_nb <- c("0 = x1+x2+x3-z") # non-binding contemporaneous constraint

    rec1 <- multivariatecholette(xlist = data_list, tcvector = tc, ccvector = cc, rho = .5, lambda = .5) # trade-off values for rho and lambda
    print(rec1)
    #> $x1
    #>          Qtr1     Qtr2     Qtr3     Qtr4
    #> 2010 7.051902 7.371871 8.069296 7.506931
    #> 2011 7.916967 6.956146 7.572586 8.154301
    #>
    #> $x2
    #>          Qtr1     Qtr2     Qtr3     Qtr4
    #> 2010 18.55737 20.59774 19.80615 21.03874
    #> 2011 19.19172 19.27605 21.37229 21.35994
    #>
    #> $x3
    #>          Qtr1     Qtr2     Qtr3     Qtr4
    #> 2010 1.490728 1.830389 2.024550 2.654333
    #> 2011 2.191317 1.667802 1.955124 2.285758
    rec2 <- multivariatecholette(xlist = data_list, tcvector = tc, ccvector = cc, rho = 1) # Denton
    rec3 <- multivariatecholette(xlist = data_list, tcvector = tc, ccvector = cc, rho = 0.729) # Cholette
    rec4 <- multivariatecholette(xlist = data_list, tcvector = NULL, ccvector = cc) # no temporal constraints
    rec5 <- multivariatecholette(xlist = data_list, tcvector = tc, ccvector = cc_nb) # non-binding contemporaneous constraint

# Calendarization

Time series data do not always coincide with calendar periods (e.g., fiscal years starting in March-April or retail data collected in non-monthly intervals). Calendarization is the process of transforming the values of a flow time series observed over varying time intervals into values that cover given calendar intervals such as month, quarter or year.

The calendarization process involves two steps:

- Temporal disaggregation of the observed data into daily values using or not an indicator
- Aggregation of the resulting daily values into the desired calendar reference periods.

Based on the paper from Quenneville et al (2012), the temporal disaggregation step is performed by considering a state-space representation of the Denton proportional first difference (PFD) method. Recall the objective function of the (modified) Denton PFD method \[ min_{y_t}\sum^n_{t=2}\biggl[\frac{y_t}{x_t}-\frac{y_{t-1}}{x_{t-1}}\biggr]^2 \] which is minimized under the temporal aggregation constraints \[ \sum_{t\epsilon l} y_t = Y_l \] \(Y_l\), \(l=1,...,q\), are the observed values to be distributed and \(x_t\), \(t=1,...,n\) are the daily indicator values that represent the daily movement of the unknown target variable \(y_t\). In the absence of such information, a constant indicator (say, a vector of 1) is used instead.

The calendarization process can be called with the `calendarization()` function. By default, a constant indicator is considered which means that the daily level of activity is assumed to be constant. To include an indicator into the disaggregation process, the function parameter `dailyweights` has to be filled. If provided, the daily indicator values (or weights) should reflect the daily level of activity that may be varying in function for instance of seasonality, trading day or other calendar effects such as public holidays.

The `calendarization()` function returns a list with the final aggregated results (after running the two steps process described above) and their associated errors, as well as the disaggregated daily values (after running the first step only) and their associated errors.

    # Example of calendarization (from Quenneville et al (2012))

    ## Observed data
    obs <- list(
        list(start = "2009-02-18", end = "2009-03-17", value = 9000),
        list(start = "2009-03-18", end = "2009-04-14", value = 5000),
        list(start = "2009-04-15", end = "2009-05-12", value = 9500),
        list(start = "2009-05-13", end = "2009-06-09", value = 7000))

    ## calendarization in absence of daily indicator values (or weights)
    cal_1 <- calendarization(obs, 12, end = "2009-06-30", dailyweights = NULL, stde = TRUE)

    ym_1 <- cal_1$rslt
    eym_1 <- cal_1$erslt
    yd_1 <- cal_1$days
    eyd_1 <- cal_1$edays
    print(cal_1)
    #> $rslt
    #>           Feb      Mar      Apr      May      Jun
    #> 2009 4012.553 7370.539 7842.402 9330.171 6399.954
    #>
    #> $erslt
    #>            Feb       Mar       Apr       May       Jun
    #> 2009 0.1758821 0.2712289 0.2517507 0.2588054 0.9227178
    #>
    #> $start
    #> [1] "2009-02-18"
    #>
    #> $days
    #>   [1] 372.6235 372.2312 371.4466 370.2697 368.7005 366.7390 364.3852 361.6391
    #>   [9] 358.5008 354.9701 351.0471 346.7318 342.0242 336.9244 331.4322 325.5477
    #>  [17] 319.2709 312.6019 305.5405 298.0868 290.2409 282.0026 273.3720 264.3492
    #>  [25] 254.9340 245.1266 234.9268 224.3348 213.3504 203.2359 193.9911 185.6162
    #>  [33] 178.1111 171.4757 165.7102 160.8145 156.7886 153.6325 151.3463 149.9298
    #>  [41] 149.3831 149.7062 150.8992 152.9619 155.8945 159.6968 164.3690 169.9110
    #>  [49] 176.3228 183.6044 191.7558 200.7770 210.6680 221.4288 233.0594 245.5598
    #>  [57] 258.9301 271.5329 283.3684 294.4365 304.7372 314.2706 323.0366 331.0352
    #>  [65] 338.2664 344.7303 350.4268 355.3559 359.5176 362.9119 365.5389 367.3985
    #>  [73] 368.4907 368.8156 368.3731 367.1632 365.1859 362.4412 358.9292 354.6498
    #>  [81] 349.6030 343.7889 337.2073 329.8584 321.7421 313.9157 306.3792 299.1325
    #>  [89] 292.1757 285.5087 279.1317 273.0444 267.2471 261.7396 256.5220 251.5943
    #>  [97] 246.9564 242.6084 238.5502 234.7820 231.3036 228.1150 225.2164 222.6075
    #> [105] 220.2886 218.2595 216.5203 215.0710 213.9115 213.0419 212.4622 212.1723
    #> [113] 212.1723 212.1723 212.1723 212.1723 212.1723 212.1723 212.1723 212.1723
    #> [121] 212.1723 212.1723 212.1723 212.1723 212.1723 212.1723 212.1723 212.1723
    #> [129] 212.1723 212.1723 212.1723 212.1723 212.1723
    #>
    #> $edays
    #>   [1] 0.02755654 0.02585358 0.02421679 0.02265858 0.02119317 0.01983673
    #>   [7] 0.01860719 0.01752375 0.01660579 0.01587096 0.01533271 0.01499758
    #>  [13] 0.01486308 0.01491698 0.01513844 0.01550048 0.01597297 0.01652522
    #>  [19] 0.01712775 0.01775322 0.01837661 0.01897502 0.01952716 0.02001275
    #>  [25] 0.02041181 0.02070393 0.02086731 0.02087768 0.02070666 0.02039086
    #>  [31] 0.01995956 0.01943922 0.01885468 0.01822994 0.01758875 0.01695495
    #>  [37] 0.01635260 0.01580565 0.01533732 0.01496886 0.01471803 0.01459732
    #>  [43] 0.01461245 0.01476162 0.01503571 0.01541933 0.01589252 0.01643244
    #>  [49] 0.01701480 0.01761482 0.01820774 0.01876884 0.01927325 0.01969535
    #>  [55] 0.02000799 0.02018138 0.02018138 0.02000799 0.01969535 0.01927325
    #>  [61] 0.01876884 0.01820774 0.01761482 0.01701480 0.01643244 0.01589252
    #>  [67] 0.01541933 0.01503571 0.01476162 0.01461245 0.01459732 0.01471803
    #>  [73] 0.01496886 0.01533732 0.01580565 0.01635260 0.01695495 0.01758875
    #>  [79] 0.01822994 0.01885468 0.01943922 0.01995956 0.02039086 0.02070666
    #>  [85] 0.02087768 0.02086731 0.02070393 0.02041181 0.02001275 0.01952716
    #>  [91] 0.01897502 0.01837661 0.01775322 0.01712775 0.01652522 0.01597297
    #>  [97] 0.01550048 0.01513844 0.01491698 0.01486308 0.01499758 0.01533271
    #> [103] 0.01587096 0.01660579 0.01752375 0.01860719 0.01983673 0.02119317
    #> [109] 0.02265858 0.02421679 0.02585358 0.02755654 0.02931489 0.03097358
    #> [115] 0.03254786 0.03404942 0.03548750 0.03686954 0.03820161 0.03948877
    #> [121] 0.04073528 0.04194476 0.04312033 0.04426469 0.04538020 0.04646895
    #> [127] 0.04753276 0.04857327 0.04959196 0.05059015 0.05156901 0.05252964
    #> [133] 0.05347301

    ## calendarization in presence of daily indicator values (or weights)
    x <- rep(c(1.0, 1.2, 1.8 , 1.6, 0.0, 0.6, 0.8), 19)
    cal_2 <- calendarization(obs, 12, end = "2009-06-30", dailyweights = x, stde = TRUE)

    ym_2 <- cal_2$rslt
    eym_2 <- cal_2$erslt
    yd_2 <- cal_2$days
    eyd_2 <- cal_2$edays

# References

Bournay J., Laroque G. (1979). Reflexions sur la methode d’elaboration des comptes trimestriels. Annales de l’Insee, n°36, pp.3-30.

Causey, B., and Trager, M.L. (1981). Derivation of Solution to the Benchmarking Problem: Trend Revision. Unpublished research notes, U.S. Census Bureau, Washington D.C. Available as an appendix in Bozik and Otto (1988).

Chamberlin, G. (2010). Temporal disaggregation. ONS Economic & Labour Market Review.

Di Fonzo, T., and Marini, M. (2011). A Newton’s Method for Benchmarking Time Series according to a Growth Rates Preservation Principle. IMF WP/11/179.

Daalmans, J., Di Fonzo, T., Mushkudiani, N., Bikker, R. (2018). Growth Rates Preservation (GRP) temporal benchmarking: Drawbacks and alternative solutions. Survey Methodology, June 2018 Vol.44, No.1, pp. 43-60 Statistics Canada, Catalogue No. 12-001-X.

Dagum, E. B., and Cholette, P. A. (2006): Benchmarking, Temporal Distribution and Reconciliation Methods of Time Series. Springer-Verlag, New York, Lecture notes in Statistics.

Eurostat (2018). ESS guidelines on temporal disaggregation, benchmarking and reconciliation. 2018 edition.

Proietti, P. (2005). Temporal Disaggregation by State Space Methods: Dynamic Regression Methods Revisited. Working papers and Studies, European Commission, ISSN 1725-4825.

Quenneville, B., Fortier S., Chen Z.-G., Latendresse E. (2006). Recent Developments in Benchmarking to Annual Totals in X12-ARIMA and at Statistics Canada. Statistics Canada, Working paper of the Time Series Research and Analysis Centre.

Quenneville, B., Picard F., Fortier S. (2012). Calendarization with interpolating splines and state space models. Statistics Canada, Appl. Statistics (2013) 62, part 3, pp 371-399.

Quilis, EM. (2018). Temporal disaggregation of economic time series - The view from the trenches. Statistica Neerlandica, Wiley.

Santos Silva, J., Cardoso, F.N. (2001). The Chow-Lin method using dynamic models. Economic Modelling, 18 (2). pp. 269-280.