# Metafrontier Malmquist Productivity Index

    library(metafrontier)

## Motivation

Standard Malmquist productivity indices measure productivity change over time by decomposing it into efficiency change and technical change. When firms operate under different technologies, however, this decomposition misses an important dimension: changes in the technology gap between a group’s frontier and the global best practice.

The metafrontier Malmquist TFP index of O’Donnell, Rao, and Battese (2008) extends the standard index with a three-way decomposition that separates within-group dynamics from cross-group technology convergence or divergence.

## The three-way decomposition

For a firm observed at times \(s\) and \(t\), the metafrontier Malmquist index decomposes as:

\[M^* = TEC \times TGC \times TC^*\]

where:

- TEC (Technical Efficiency Change) \(= TE^{group}_t / TE^{group}_s\) measures whether the firm moved closer to or further from its own group’s frontier.

- TGC (Technology Gap Change) \(= TGR_t / TGR_s\) measures whether the group’s frontier moved closer to or further from the metafrontier. A value above 1 indicates the group is catching up technologically.

- TC* (Metafrontier Technical Change) measures the shift of the global production possibility frontier itself.

Values above 1 indicate improvement; below 1 indicate deterioration.

## Simulating panel data

The `simulate_metafrontier()` function generates cross-sectional data. For panel data, we call it repeatedly with varying parameters to create time-varying technology gaps:

    set.seed(42)
    panels <- lapply(1:4, function(t) {
      sim <- simulate_metafrontier(
        n_groups = 2,
        n_per_group = 50,
        beta_meta = c(1.0, 0.5, 0.3),
        tech_gap = c(0, 0.3 + 0.03 * t),  # G2 falls behind over time
        sigma_u = c(0.2, 0.3),
        sigma_v = 0.15,
        seed = 42 + t
      )
      sim$data$time <- t
      sim$data$id <- seq_len(nrow(sim$data))
      sim$data
    })
    panel_data <- do.call(rbind, panels)

    table(panel_data$group, panel_data$time)
    #>
    #>       1  2  3  4
    #>   G1 50 50 50 50
    #>   G2 50 50 50 50

In this simulation, Group G1 operates at the metafrontier (zero technology gap), while G2 has an increasing gap over time. We would expect TGC < 1 for G2 (falling behind) and TGC \(\approx\) 1 for G1.

## Computing the index

    malm <- malmquist_meta(
      log_y ~ log_x1 + log_x2,
      data = panel_data,
      group = "group",
      time = "time",
      orientation = "output",
      rts = "crs"
    )

    malm
    #>
    #> Metafrontier Malmquist TFP Index
    #> ================================
    #> Orientation:  output
    #> RTS:          crs
    #> Groups:       G1, G2
    #> Periods:      1 -> 2 -> 3 -> 4
    #> Observations: 300
    #>
    #> Mean decomposition (M* = TEC x TGC x TC*):
    #>   MPI  = 1.085
    #>   TEC  = 1.123
    #>   TGC  = 1.127
    #>   TC*  = 0.9818

## Detailed results

The `summary()` method provides group-level and period-level breakdowns:

    summary(malm)
    #>
    #> Metafrontier Malmquist TFP Index Summary
    #> =========================================
    #>
    #> Call:
    #> malmquist_meta(formula = log_y ~ log_x1 + log_x2, data = panel_data,
    #>     group = "group", time = "time", orientation = "output", rts = "crs")
    #>
    #> Orientation: output
    #> RTS:         crs
    #> Groups:      G1, G2
    #> Periods:     1 -> 2 -> 3 -> 4
    #> Observations: 300
    #>
    #> Overall means:
    #>   MPI  = 1.085
    #>   TEC  = 1.123
    #>   TGC  = 1.127
    #>   TC*  = 0.9818
    #>
    #> --- Three-Way Decomposition by Group ---
    #> M* = TEC x TGC x TC*
    #>
    #> Group: G1 (n = 150 )
    #>    MPI    TEC    TGC     TC
    #> 1.0726 1.2388 0.9833 0.9874
    #>
    #> Group: G2 (n = 150 )
    #>    MPI    TEC    TGC     TC
    #> 1.0962 1.0082 1.2703 0.9766
    #>
    #> --- By Period ---
    #>
    #> Period 1 -> 2
    #>    MPI    TEC    TGC     TC
    #> 1.1697 1.0360 1.3314 0.9044
    #>
    #> Period 2 -> 3
    #>    MPI    TEC    TGC     TC
    #> 1.0470 1.5288 1.0757 0.6282
    #>
    #> Period 3 -> 4
    #>    MPI    TEC    TGC     TC
    #> 1.0364 0.8056 0.9732 1.4224
    #>
    #> --- Technology Gap Ratios ---
    #>
    #> Group: G1
    #>   Mean TGR (from): 0.987
    #>   Mean TGR (to):   0.9687
    #>   Mean TGC:        0.9833
    #>
    #> Group: G2
    #>   Mean TGR (from): 0.7011
    #>   Mean TGR (to):   0.8234
    #>   Mean TGC:        1.27

## Interpreting the decomposition

The main results table contains one row per firm per consecutive period pair:

    head(malm$malmquist, 10)
    #>    id group period_from period_to       MPI       TEC       TGC        TC
    #> 1   1    G1           1         2 2.5584365 2.7812965 1.0000000 0.9198719
    #> 2   2    G1           1         2 1.6371319 2.2260475 0.8271005 0.8891826
    #> 3   3    G1           1         2 0.5330948 0.6482012 0.9164504 0.8973991
    #> 4   4    G1           1         2 1.0177377 1.1385028 0.9851380 0.9074124
    #> 5   5    G1           1         2 2.6412214 2.3940135 1.0000000 1.1032609
    #> 6   6    G1           1         2 1.3723118 1.5701415 1.0000000 0.8740052
    #> 7   7    G1           1         2 0.6848389 0.8406678 1.0000000 0.8146368
    #> 8   8    G1           1         2 1.7332279 1.8166178 1.0000000 0.9540961
    #> 9   9    G1           1         2 1.0869101 1.1994245 1.0000000 0.9061930
    #> 10 10    G1           1         2 1.3215333 1.7610854 0.8459739 0.8870349

Each row reports:

Column | Meaning
--- | ---
`MPI` | Metafrontier Malmquist TFP index (\(M^* = TEC \times TGC \times TC^*\))
`TEC` | Within-group efficiency change
`TGC` | Technology gap change
`TC` | Metafrontier technical change

The identity can be verified:

    m <- malm$malmquist
    complete <- complete.cases(m[, c("MPI", "TEC", "TGC", "TC")])
    all.equal(m$MPI[complete], m$TEC[complete] * m$TGC[complete] * m$TC[complete])
    #> [1] TRUE

## Within-group vs metafrontier Malmquist

The object also stores the standard within-group Malmquist decomposition and the metafrontier-level decomposition:

    # Within-group: MPI_group = EC_group x TC_group
    head(malm$group_malmquist)
    #>   id group period_from period_to MPI_group  EC_group  TC_group
    #> 1  1    G1           1         2        NA 2.7812965        NA
    #> 2  2    G1           1         2  1.757668 2.2260475 0.7895912
    #> 3  3    G1           1         2  0.556865 0.6482012 0.8590929
    #> 4  4    G1           1         2  1.018773 1.1385028 0.8948361
    #> 5  5    G1           1         2  2.641221 2.3940135 1.1032609
    #> 6  6    G1           1         2  1.325650 1.5701415 0.8442872

    # Metafrontier: MPI_meta = EC_meta x TC_meta
    head(malm$meta_malmquist)
    #>   id group period_from period_to  MPI_meta   EC_meta   TC_meta
    #> 1  1    G1           1         2 2.5584365 2.7812965 0.9198719
    #> 2  2    G1           1         2 1.6371319 1.8411649 0.8891826
    #> 3  3    G1           1         2 0.5330948 0.5940443 0.8973991
    #> 4  4    G1           1         2 1.0177377 1.1215823 0.9074124
    #> 5  5    G1           1         2 2.6412214 2.3940135 1.1032609
    #> 6  6    G1           1         2 1.3723118 1.5701415 0.8740052

The within-group index captures only efficiency change and frontier shift within the group. The metafrontier index additionally accounts for whether the group is converging toward or diverging from the global best practice.

## Technology gap dynamics

The TGR at each period endpoint is stored in the `tgr` component:

    tgr_df <- malm$tgr

    # Mean TGR by group and period
    aggregate(cbind(TGR_from, TGR_to) ~ group, data = tgr_df, FUN = mean)
    #>   group  TGR_from    TGR_to
    #> 1    G1 0.9869568 0.9686967
    #> 2    G2 0.7010526 0.8233954

For G2, we expect TGR to decline over time (increasing technology gap). The TGC column confirms this:

    aggregate(TGC ~ group, data = tgr_df, FUN = mean)
    #>   group       TGC
    #> 1    G1 0.9832828
    #> 2    G2 1.2702686

## Returns to scale assumptions

The `rts` argument controls the DEA technology assumption. Under variable returns to scale, scale effects are netted out:

    malm_vrs <- malmquist_meta(
      log_y ~ log_x1 + log_x2,
      data = panel_data,
      group = "group",
      time = "time",
      rts = "vrs"
    )

    # Compare mean MPI under CRS vs VRS
    data.frame(
      CRS = colMeans(malm$malmquist[, c("MPI", "TEC", "TGC", "TC")],
                     na.rm = TRUE),
      VRS = colMeans(malm_vrs$malmquist[, c("MPI", "TEC", "TGC", "TC")],
                     na.rm = TRUE)
    )
    #>           CRS      VRS
    #> MPI 1.0847509 1.018219
    #> TEC 1.1234751 1.004931
    #> TGC 1.1267757 1.014285
    #> TC  0.9818377 1.005744

## Using real-world panel data

The `plm` package provides `Produc`, a panel of 48 US states over 1970–1986 with a built-in `region` grouping variable. This is a natural candidate for metafrontier Malmquist analysis:

    library(plm)
    data("Produc", package = "plm")

    malm_us <- malmquist_meta(
      gsp ~ pc + emp,
      data = Produc,
      group = "region",
      time = "year",
      rts = "crs"
    )
    summary(malm_us)

Similarly, `sfaR::utility` provides electric utility data with a binary `regu` variable (regulated vs. deregulated) over 1986–1996:

    library(sfaR)
    data("utility", package = "sfaR")

    malm_util <- malmquist_meta(
      y ~ k + labor + fuel,
      data = utility,
      group = "regu",
      time = "year",
      rts = "vrs"
    )
    summary(malm_util)

## References

- O’Donnell, C.J., Rao, D.S.P. and Battese, G.E. (2008). Metafrontier frameworks for the study of firm-level efficiencies and technology ratios. Empirical Economics, 34(2), 231–255.

- Battese, G.E., Rao, D.S.P. and O’Donnell, C.J. (2004). A metafrontier production function for estimation of technical efficiencies and technology gaps for firms operating under different technologies. Journal of Productivity Analysis, 21(1), 91–103.