# Methodological Details

#### Drew Dimmery (ddimmery@univie.ac.at)

#### Edward Kennedy (edward@stat.cmu.edu)

#### July 29, 2025

Abstract

In this paper, we introduce the `tidyhte` package for estimation of heterogeneous treatment effects (HTE) from observational or experimental data. This package implements the methods of Kennedy (2020) and presents them through a tidy-style user-facing API. The design principles undergirding this package are (1) the APIs should be tidy-friendly, (2) Analyses should be easy to replicate with minor changes, (3) specifying complex ensembles for the nuisance functions should be straightforward, and (4) sensible diagnostics should be easily accessible. Plotting and formatting of the results are left for the end-user to customize.

# Summary

This document details how `tidyhte` constructs estimates of heterogeneous treatment effects. It will highlight a variety of the features of the package and discuss the mathematics which undergird them.

After a brief introduction to the methods of HTE estimation of Kennedy (2020), the structure will generally follow the estimation API of `tidyhte`: it will begin by discussing the creation of cross-validation folds, then highlight nuisance function estimation, proceed to the construction of “pseudo outcomes”, a concept from Kennedy (2020), and conclude by demonstrating the calculation of a few varieties of Quantities of Interest: the actual statistics which are desired by the end-user.

# Preliminaries

## Problem Setting

Our data is defined by the triple \(Z_i = (X_i, A_i, Y_i)\), with \(X \in \mathbb{R}^d\), \(Y \in \mathbb{R}\) and \(A \in \{0, 1\}\). Define the following nuisance functions: \[ \pi(x) \equiv \mathbb{P}(A = 1 \mid X = x) \\ \mu_a(x) \equiv \mathbb{E}(Y \mid X = x, A = a) \]

Heterogeneous treatment effects are defined as the difference in conditional expectations under treatment and control, \({\tau(x) \equiv \mu_1(x) - \mu_0(x)}\). Throughout, we will maintain the following assumptions:

1. Consistency: \(Y_i = Y_i(A_i)\)

2. No Unmeasured Confounding: \(A \perp (Y(1), Y(0)) \mid X\)

3. Positivity: \(\epsilon \leq \pi \leq 1 - \epsilon\) with probability \(1\)

Under these assumptions, \(\tau(x) = \mathbb{E}[Y(1) - Y(0) \mid X = x]\).

For the remainder of this paper, we will use a semi-simulated running example based on the `penguins` dataset of Horst, Hill, and Gorman (2020). We will imagine a randomly assigned nutritional intervention for which we wish to measure the causal effect on body mass. We also have a lower variance measurement of the average food consumed per day over an observation period. Gentoo penguins gain weight by this intervention on average, while vice-versa for Adelie penguins. The average change in weight for Chinstrap penguins is zero.

    penguins <- within(penguins, {
      id <- 1:n
      propensity_score <- 0.5
      treatment <- rbinom(n, 1, propensity_score)
      tau <- 0.2 * (species == "Gentoo") - 0.2 * (species == "Adelie") + rnorm(n, sd = 0.05)
      food_consumed_g <- rnorm(n, 500, 5) * (1 + tau * treatment)
      body_mass_g <- body_mass_g * (1 + tau * treatment)
    })

## Overview of Method

We begin by introducing the following “DR-Learner” algorithm:

DR-learner algorithm. This consists of three main steps:

1. Given: estimates of nuisance functions trained elsewhere: \(\left(\hat{\pi}, \hat{\mu}_0, \hat{\mu}_1\right)\)
2. Construct “pseudo-outcome: This quantity is a transformation of the provided nuisance function estimators. \[ \hat{\psi}(Z) = \frac{A - \hat{\pi}(X)}{\hat{\pi}(X)(1 - \hat{\pi}(X))} \left( Y - \hat{\mu}_{A}(X) \right) + \hat{\mu}_{1}(X) - \hat{\mu}_{0}(X) \]
3. Second-stage regression: Construct a smoothing model over the transformed pseudo-outcome. \[ \hat{\tau}_{dr}(x) = \hat{\mathbb{E}}[\hat{\psi}(Z) \mid X = x] \]

This algorithm is written assuming estimates of nuisance functions are trained on separate data and, therefore, can be treated as fixed. The easiest way to do this is with a sample-splitting procedure, after which results are averaged across splits.

The crucial result, Theorem 2 of Kennedy (2020), shows that the error of the second stage regression will match the error of an oracle regression of the true individual treatment effects on covariates \(X\) up to a factor which depends on the product of the errors of the two estimated nuisance functions (that is, the errors in \(\pi\) and in \(\mu\)).

When the second-stage regression is simple, like subgroup averages within cells defined by \(X\), the DR-Learner inherits unbiasedness and efficiency like an AIPW estimator (Robins, Rotnitzky, and Zhao 1995; Van der Laan, Laan, and Robins 2003; Tsiatis 2006; Tsiatis et al. 2008; Chernozhukov et al. 2018).

The approximation results of Kennedy (2020) applied to regressions on this transformed outcome allow for a lot of flexibility in quantities of interest that may be estimated. We will use this fact to allow the estimation of a variety of models as if they were estimated on the individual causal effects themselves: for example, variable importance measures operate off the assumption that various second-stage regressions will be accurate.

## Principles

The `tidyhte` package is premised on the idea of breaking up the analysis of HTEs into a few distinct parts, which can then be mixed together as desired. The first step is to define a configuration object (or recipe) describing at a high-level how HTE estimation should be performed. During estimation, the specific variables of interest are indicated. This design allows for repeating very similar analyses multiple times with very little overhead. Instances when this might be useful are when the user wishes to explore heterogeneity across a variety of outcomes, or when there are a variety of treatment contrasts of particular interest. Each of these analyses will tend to share common features: similar classes of models will be included in the ensembles for nuisance estimation, for example.

Also important is that the methods provided support the usage of common

### Clustered data

A common feature of real-world data is that treatment may be clustered. In other words, if one unit receives treatment, there may be other units who are then also more likely to receive treatment. A common example of this sort of design might be when villages or townships are assigned to treatment, but measurement occurs at the individual level. As discussed by Abadie et al. (2023), this design implies that standard errors should be clustered at the level at which treatment was assigned. The `tidyhte` package supports clustering as a first class citizen, and resulting estimates will all receive the proper statistical uncertainty estimates based around this clustering. In practice, the end-user simply specifies the individual unit-id when constructing cross-validation splits (`make_splits()`), and all subsequent analyses take this into account.

### Population weights

Another common feature of designs used in practice is that they come from samples that do not perfectly represent the larger populations from which they are drawn. The most common solution to this problem is through the use of weights to make the sample more closely resemble the population. This, too, is supported by `tidyhte`, by simply specifying weights when models are estimated (`produce_plugin_estimates()`). Downstream analyses will then take weights into account appropriately.

# Recipe API

In order to build up definitions around how HTE should be estimated, `tidyhte` provides an API to progressively build up a configuration object. A `basic_config` function creates a bare-bones configuration consisting of only linear models for the respective nuisance functions and a number of diagnostics.

    cfg <- basic_config() %>%
        add_known_propensity_score("propensity_score") %>%
        add_outcome_model("SL.glmnet", alpha = c(0.0, 1.0)) %>%
        add_moderator("Stratified", species, island, sex, year) %>%
        add_moderator("KernelSmooth", bill_length_mm, bill_depth_mm, flipper_length_mm) %>%
        add_vimp(sample_splitting = FALSE)

Since the subject of interest is an experimental intervention, the propensity score is known. Using this known propensity score provides unbiasedness for many quantities of interest (although this may leave some efficiency on the table CITE). In this case, in addition to the (default) linear model included in the SuperLearner ensemble, we add an elastic-net regression (Zou and Hastie 2005). We sweep over a variety of mixing parameters between LASSO and ridge, throwing each of these models into the ensemble. This means that SuperLearner will perform model selection and averaging to identify the best hyper-parameter values (Van der Laan, Polley, and Hubbard 2007). Furthermore, we define all of the moderators of interest and how their results should be collected and displayed. Discrete moderators will just take stratified averages at each level of the moderator, while continuous moderators will use local polynomial regression (Fan and Gijbels 2018; Calonico, Cattaneo, and Farrell 2019). Finally, we add a variable importance measure from Williamson et al. (2021).

After the configuration is completed, it can be attached to the dataset.

    penguins %<>% attach_config(cfg)

# Cross-validation

The first step in an analysis of heterogeneous treatment effects following this procedure is to define how to construct splits to be used for cross-validation. `tidyhte` accomplishes this by using blocking methods to construct lower-variance splits than a purely randomized splitting procedure would entail. Existing methods for generating splits often provide options for stratifying based on a binary outcome to ensure there is variance in the outcome in all splits, even when the outcome is very sparse. For instance, `SuperLearner::SuperLearner.CV.control` provides such an option.

The appropriate function in `tidyhte` is `make_splits`. This function takes in a dataframe and determines a set of splits which accord with the provided unit identifier and number of splits. If any covariates are provided to this function, it will include them in a blocking design for constructing randomized splits using the methods of Higgins, Sävje, and Sekhon (2016) as implemented in the `quickblock` package.

There are a few relevant methodological notes about how this blocking for cross-validation strata works. In short, blocks are constructed which are sized to be at least as large as the number of splits to use. These blocks are constructed to minimize the within-block distance between units (where distances are Euclidean based on the provided covariates). When more than one row shares an identifier, blocking is performed on the average covariate value within each identifier. Rows with the same identifier are then assigned to the same split. Precise details on the construction of blocks may be found in `higgins2016improving`. Within each block, a vector of split IDs is constructed which has a marginal distribution as close to the uniform distribution over splits as possible (up to integer division errors). This set of split IDs is then randomly permuted within blocks.

Consistent with tidy semantics, the original dataframe is returned, but with the addition of a column `.split_id` representing these newly constructed splits, and a few attributes for bookkeeping (e.g. the column name of the identifier). Since the object returned is the same as what was passed in, this makes for easy chaining of commands using `dplyr`.

    penguins %<>% make_splits(id, species, sex, flipper_length_mm, .num_splits = 3)

    ## `num_splits` must be even if VIMP is requested as a QoI. Rounding up.

    ## `quickblock` is not installed, so falling back to un-stratified CV.

    ## Dropped 11 of 344 rows (3.2%) through listwise deletion.

Note that `tidyhte` gracefully handles missing data via listwise deletion. More advanced imputation methods are not yet supported.

# Nuisance function estimation

Estimation of nuisance functions such as the propensity score and the outcome regression are typically handled by the SuperLearner library. Specifying a full array of models with diverse hyperparameters is much simplified through the `tidyhte` API. To specify a cross-validated learner using SuperLearner syntax requires substantially more boilerplate:

    learners <- create.Learner(
        "SL.glmnet",
        tune = list(
            alpha = c(0.05, 0.15, 0.2, 0.25, 0.5, 0.75)
        ),
        detailed_names = TRUE,
        name_prefix = paste0("SLglmnet")
    )

    CV.SuperLearner(label, covariates, SL.library = learners$names)

In contrast, the `tidyhte` Recipe API requires only the following one line:

    add_outcome_model(cfg, "SL.glmnet", alpha = c(0.0, 0.25, 0.5, 0.75, 1.0))

    penguins %<>% produce_plugin_estimates(
      # outcome
      food_consumed_g,
      # treatment
      treatment,
      # covariates
      species, island, sex, year, bill_length_mm, bill_depth_mm, flipper_length_mm
    )

# Pseudo-outcome construction

Once nuisance functions are estimated, it is simply a matter of combining these results together into the appropriate pseudo-outcome. For typical HTE estimation, the pseudo-outcome of interest is the one analyzed by Kennedy (2020): the uncentered influence function of the average treatment effect (Robins, Rotnitzky, and Zhao 1995).

    penguins %<>% construct_pseudo_outcomes(food_consumed_g, treatment)

# Quantities of interest

Finally, it comes to the most glamorous part of the analysis, when effects are estimated and put into charts.

The design of `tidyhte` chooses to leave the charting to the end-user, but merely returns a tidy tibble with all of the requested quantities of interest. The package focuses on a few types of quantities of interest: - Marginal Conditional Average Treatment Effects (MCATEs): the standard Conditional Average Treatment Effects (CATEs) in the literature, in which all covariates except one are marginalized over, providing one average effect for one level of a single covariate. This is in contrast to a “Partial” CATE, in which other variables are “controlled for” in some way. This does not provide a satisfying causal interpretation without assumptions of joint randomization of treatment and covariates. - Variable Importance (VIMP): Using Williamson et al. (2021), `tidyhte` calculates how much each moderator contributes to the overall reduction in mean-squared-error in a joint model of the heterogeneous effects. - Diagnostics: A wide variety of diagnostics are provided for all models fit as part of the `tidyhte` estimation process. These include single-number summaries like mean-squared-error or AUC, as well as entire receiver operating characteristic curves and coefficients in the SuperLearner ensembles.

    penguins %>%
      estimate_QoI(species, island, sex, year, bill_length_mm, bill_depth_mm, flipper_length_mm) ->
      results

The resulting tibble provides looks like the following:

    results

estimand | term | value | level | estimate | std_error
--- | --- | --- | --- | --- | ---
MCATE | species | NA | Adelie | -97.6015410 | 3.0244596
MCATE | species | NA | Gentoo | 101.0159746 | 3.8757733
MCATE | species | NA | Chinstrap | -6.7478462 | 5.4682440
MCATE | island | NA | Torgersen | -94.4444275 | 6.0631662
MCATE | island | NA | Biscoe | 47.5695000 | 7.5933983
MCATE | island | NA | Dream | -48.8025850 | 5.5723696
MCATE | sex | NA | male | -6.9821036 | 7.7849620
MCATE | sex | NA | female | -9.1805099 | 7.2607727
MCATE | year | 2007.00000 | NA | -12.3711334 | 9.4102451
MCATE | year | 2008.00000 | NA | 1.4002176 | 9.3750740
MCATE | year | 2009.00000 | NA | -13.4339776 | 8.8758959
MCATE | bill_length_mm | 35.70000 | NA | -101.4951715 | 4.1807930
MCATE | bill_length_mm | 35.90000 | NA | -102.7355707 | 4.1760523
MCATE | bill_length_mm | 36.00000 | NA | -103.4886263 | 4.1683476
MCATE | bill_length_mm | 36.20000 | NA | -105.0963642 | 4.1524989
MCATE | bill_length_mm | 36.36727 | NA | -106.3818040 | 4.1427218
MCATE | bill_length_mm | 36.50000 | NA | -107.3639819 | 4.1362700
MCATE | bill_length_mm | 36.67091 | NA | -108.4157163 | 4.1203853
MCATE | bill_length_mm | 36.87273 | NA | -109.4799235 | 4.1229524
MCATE | bill_length_mm | 37.14909 | NA | -110.8619435 | 4.1499920
MCATE | bill_length_mm | 37.30000 | NA | -111.4057401 | 4.1864570
MCATE | bill_length_mm | 37.57818 | NA | -112.0034005 | 4.2170951
MCATE | bill_length_mm | 37.68000 | NA | -112.0034057 | 4.1888788
MCATE | bill_length_mm | 37.78182 | NA | -111.8051248 | 4.1512724
MCATE | bill_length_mm | 37.88364 | NA | -111.4393904 | 4.1239732
MCATE | bill_length_mm | 38.10000 | NA | -110.0018568 | 4.0883197
MCATE | bill_length_mm | 38.18727 | NA | -109.1507608 | 4.0941187
MCATE | bill_length_mm | 38.47818 | NA | -105.9105734 | 4.2103762
MCATE | bill_length_mm | 38.60000 | NA | -104.2915889 | 4.2955480
MCATE | bill_length_mm | 38.80000 | NA | -101.4342473 | 4.4689948
MCATE | bill_length_mm | 38.90000 | NA | -99.9272072 | 4.5693312
MCATE | bill_length_mm | 39.00000 | NA | -98.2709272 | 4.6770654
MCATE | bill_length_mm | 39.20000 | NA | -94.7896545 | 4.9128270
MCATE | bill_length_mm | 39.50000 | NA | -89.3907649 | 5.2980279
MCATE | bill_length_mm | 39.60000 | NA | -87.5793353 | 5.4315143
MCATE | bill_length_mm | 39.60000 | NA | -87.5793353 | 5.4315143
MCATE | bill_length_mm | 39.70000 | NA | -85.7620432 | 5.5645036
MCATE | bill_length_mm | 39.82182 | NA | -83.5326891 | 5.7194742
MCATE | bill_length_mm | 40.20000 | NA | -76.1258232 | 6.1893781
MCATE | bill_length_mm | 40.32182 | NA | -73.6497522 | 6.3381528
MCATE | bill_length_mm | 40.60000 | NA | -67.8472592 | 6.6760657
MCATE | bill_length_mm | 40.61455 | NA | -67.5279206 | 6.6935823
MCATE | bill_length_mm | 40.81636 | NA | -63.0101369 | 6.9221794
MCATE | bill_length_mm | 40.90000 | NA | -61.0759813 | 7.0113433
MCATE | bill_length_mm | 41.10000 | NA | -56.1909922 | 7.2134525
MCATE | bill_length_mm | 41.10000 | NA | -56.1909922 | 7.2134525
MCATE | bill_length_mm | 41.14727 | NA | -55.0176295 | 7.2564510
MCATE | bill_length_mm | 41.40000 | NA | -48.6708731 | 7.4788708
MCATE | bill_length_mm | 41.52727 | NA | -45.2878653 | 7.5857834
MCATE | bill_length_mm | 41.85818 | NA | -35.7053834 | 7.8192078
MCATE | bill_length_mm | 42.13091 | NA | -27.4967304 | 7.9845839
MCATE | bill_length_mm | 42.33273 | NA | -21.4124209 | 8.0703689
MCATE | bill_length_mm | 42.50000 | NA | -16.3497006 | 8.1312467
MCATE | bill_length_mm | 42.70000 | NA | -10.2458220 | 8.1850934
MCATE | bill_length_mm | 42.83818 | NA | -6.0716514 | 8.2148666
MCATE | bill_length_mm | 43.14000 | NA | 2.9302749 | 8.2552029
MCATE | bill_length_mm | 43.20000 | NA | 4.6936628 | 8.2612901
MCATE | bill_length_mm | 43.34364 | NA | 8.8643206 | 8.2626845
MCATE | bill_length_mm | 43.50000 | NA | 13.3572191 | 8.2625240
MCATE | bill_length_mm | 43.89455 | NA | 23.8041290 | 8.2191545
MCATE | bill_length_mm | 44.24727 | NA | 32.5904134 | 8.1375456
MCATE | bill_length_mm | 44.90000 | NA | 47.1512711 | 7.8843537
MCATE | bill_length_mm | 45.10000 | NA | 50.9729028 | 7.7921983
MCATE | bill_length_mm | 45.20000 | NA | 52.8257130 | 7.7460599
MCATE | bill_length_mm | 45.20000 | NA | 52.8257130 | 7.7460599
MCATE | bill_length_mm | 45.30000 | NA | 54.5460542 | 7.6939865
MCATE | bill_length_mm | 45.46000 | NA | 57.0627013 | 7.6059763
MCATE | bill_length_mm | 45.50000 | NA | 57.6479612 | 7.5833797
MCATE | bill_length_mm | 45.60000 | NA | 59.0724845 | 7.5273076
MCATE | bill_length_mm | 45.70000 | NA | 60.4474101 | 7.4725290
MCATE | bill_length_mm | 45.80000 | NA | 61.6852109 | 7.4144991
MCATE | bill_length_mm | 46.00000 | NA | 63.9071814 | 7.2856199
MCATE | bill_length_mm | 46.10000 | NA | 64.9540465 | 7.2213269
MCATE | bill_length_mm | 46.20000 | NA | 65.9345715 | 7.1588974
MCATE | bill_length_mm | 46.37455 | NA | 67.4232180 | 7.0523582
MCATE | bill_length_mm | 46.40000 | NA | 67.6104400 | 7.0367293
MCATE | bill_length_mm | 46.50000 | NA | 68.3007598 | 6.9768976
MCATE | bill_length_mm | 46.58000 | NA | 68.7817211 | 6.9302013
MCATE | bill_length_mm | 46.70000 | NA | 69.4118406 | 6.8552309
MCATE | bill_length_mm | 46.80000 | NA | 69.7998084 | 6.7886170
MCATE | bill_length_mm | 46.90000 | NA | 70.1159181 | 6.7222648
MCATE | bill_length_mm | 47.20000 | NA | 70.6758974 | 6.5516901
MCATE | bill_length_mm | 47.48909 | NA | 70.4375659 | 6.3887280
MCATE | bill_length_mm | 47.50000 | NA | 70.4169628 | 6.3830678
MCATE | bill_length_mm | 47.69273 | NA | 69.7402591 | 6.2859239
MCATE | bill_length_mm | 48.10000 | NA | 67.3609711 | 6.1559992
MCATE | bill_length_mm | 48.39273 | NA | 64.8435271 | 6.1172928
MCATE | bill_length_mm | 48.49818 | NA | 63.7597028 | 6.1154557
MCATE | bill_length_mm | 48.60000 | NA | 62.6641695 | 6.1182781
MCATE | bill_length_mm | 48.70182 | NA | 61.5724160 | 6.1274485
MCATE | bill_length_mm | 49.00000 | NA | 58.8944555 | 6.1481376
MCATE | bill_length_mm | 49.10000 | NA | 58.1112764 | 6.1616283
MCATE | bill_length_mm | 49.20727 | NA | 57.3831459 | 6.1824471
MCATE | bill_length_mm | 49.40909 | NA | 55.9264697 | 6.2425300
MCATE | bill_length_mm | 49.51091 | NA | 55.1149003 | 6.2847812
MCATE | bill_length_mm | 49.61273 | NA | 54.2526178 | 6.3374313
MCATE | bill_length_mm | 49.80000 | NA | 52.6414202 | 6.4471679
MCATE | bill_length_mm | 50.00000 | NA | 51.1712609 | 6.5839790
MCATE | bill_length_mm | 50.00000 | NA | 51.1712609 | 6.5839790
MCATE | bill_length_mm | 50.12000 | NA | 50.3714042 | 6.6778965
MCATE | bill_length_mm | 50.22182 | NA | 49.7545147 | 6.7612413
MCATE | bill_length_mm | 50.42364 | NA | 48.6297256 | 6.9435650
MCATE | bill_length_mm | 50.50000 | NA | 48.3115668 | 7.0159955
MCATE | bill_length_mm | 50.62727 | NA | 47.7922408 | 7.1423411
MCATE | bill_length_mm | 50.80000 | NA | 47.2171824 | 7.3314712
MCATE | bill_length_mm | 50.83091 | NA | 47.1593517 | 7.3631766
MCATE | bill_length_mm | 51.03273 | NA | 46.7274563 | 7.5709957
MCATE | bill_length_mm | 51.30000 | NA | 45.8726816 | 7.8475523
MCATE | bill_length_mm | 51.33636 | NA | 45.7392750 | 7.8881391
MCATE | bill_length_mm | 51.57636 | NA | 44.8152600 | 8.1810784
MCATE | bill_length_mm | 52.00000 | NA | 43.2345464 | 8.8251736
MCATE | bill_depth_mm | 13.90000 | NA | 102.2326405 | 6.9632426
MCATE | bill_depth_mm | 13.96182 | NA | 101.0877754 | 6.4888892
MCATE | bill_depth_mm | 14.10000 | NA | 99.2432566 | 5.6206059
MCATE | bill_depth_mm | 14.20000 | NA | 98.8787890 | 5.2327017
MCATE | bill_depth_mm | 14.20000 | NA | 98.8787890 | 5.2327017
MCATE | bill_depth_mm | 14.30000 | NA | 99.2318484 | 5.0441899
MCATE | bill_depth_mm | 14.40000 | NA | 99.1676753 | 5.0155298
MCATE | bill_depth_mm | 14.47273 | NA | 98.6834689 | 5.0538849
MCATE | bill_depth_mm | 14.50000 | NA | 98.5747942 | 5.0766810
MCATE | bill_depth_mm | 14.50000 | NA | 98.5747942 | 5.0766810
MCATE | bill_depth_mm | 14.60000 | NA | 98.1741852 | 5.2317838
MCATE | bill_depth_mm | 14.60000 | NA | 98.1741852 | 5.2317838
MCATE | bill_depth_mm | 14.78182 | NA | 98.1925173 | 5.6613555
MCATE | bill_depth_mm | 14.88364 | NA | 98.0159885 | 6.0193721
MCATE | bill_depth_mm | 15.00000 | NA | 97.6368602 | 6.5180430
MCATE | bill_depth_mm | 15.00000 | NA | 97.6368602 | 6.5180430
MCATE | bill_depth_mm | 15.00000 | NA | 97.6368602 | 6.5180430
MCATE | bill_depth_mm | 15.10000 | NA | 96.6576283 | 6.9709574
MCATE | bill_depth_mm | 15.20000 | NA | 94.7442070 | 7.4577639
MCATE | bill_depth_mm | 15.29455 | NA | 92.5049697 | 7.9410625
MCATE | bill_depth_mm | 15.30000 | NA | 92.3694654 | 7.9682704
MCATE | bill_depth_mm | 15.49818 | NA | 83.8886028 | 8.9220107
MCATE | bill_depth_mm | 15.60000 | NA | 78.3713434 | 9.4255887
MCATE | bill_depth_mm | 15.70000 | NA | 72.3430064 | 9.8114085
MCATE | bill_depth_mm | 15.70364 | NA | 72.1147823 | 9.8230312
MCATE | bill_depth_mm | 15.80000 | NA | 65.8267606 | 10.0883906
MCATE | bill_depth_mm | 15.90000 | NA | 58.4446434 | 10.2276914
MCATE | bill_depth_mm | 15.90909 | NA | 57.6838061 | 10.2281955
MCATE | bill_depth_mm | 16.00000 | NA | 49.7514834 | 10.2075726
MCATE | bill_depth_mm | 16.10000 | NA | 40.5958334 | 10.1746539
MCATE | bill_depth_mm | 16.10000 | NA | 40.5958334 | 10.1746539
MCATE | bill_depth_mm | 16.20000 | NA | 31.4716880 | 10.1400210
MCATE | bill_depth_mm | 16.30000 | NA | 21.8104569 | 10.1049033
MCATE | bill_depth_mm | 16.40000 | NA | 12.0796012 | 10.1108222
MCATE | bill_depth_mm | 16.50000 | NA | 2.9369283 | 10.0842382
MCATE | bill_depth_mm | 16.60000 | NA | -5.3967870 | 9.9434344
MCATE | bill_depth_mm | 16.60000 | NA | -5.3967870 | 9.9434344
MCATE | bill_depth_mm | 16.70000 | NA | -13.1938694 | 9.7040687
MCATE | bill_depth_mm | 16.80000 | NA | -20.6908275 | 9.4570377
MCATE | bill_depth_mm | 16.90000 | NA | -28.5200375 | 9.2676175
MCATE | bill_depth_mm | 17.00000 | NA | -36.0673354 | 9.0552313
MCATE | bill_depth_mm | 17.00000 | NA | -36.0673354 | 9.0552313
MCATE | bill_depth_mm | 17.00000 | NA | -36.0673354 | 9.0552313
MCATE | bill_depth_mm | 17.00000 | NA | -36.0673354 | 9.0552313
MCATE | bill_depth_mm | 17.10000 | NA | -42.9811964 | 8.7908804
MCATE | bill_depth_mm | 17.10000 | NA | -42.9811964 | 8.7908804
MCATE | bill_depth_mm | 17.20000 | NA | -49.3054333 | 8.5133952
MCATE | bill_depth_mm | 17.20000 | NA | -49.3054333 | 8.5133952
MCATE | bill_depth_mm | 17.30000 | NA | -54.7229655 | 8.2374783
MCATE | bill_depth_mm | 17.30000 | NA | -54.7229655 | 8.2374783
MCATE | bill_depth_mm | 17.30000 | NA | -54.7229655 | 8.2374783
MCATE | bill_depth_mm | 17.50000 | NA | -63.2805473 | 7.6892999
MCATE | bill_depth_mm | 17.50000 | NA | -63.2805473 | 7.6892999
MCATE | bill_depth_mm | 17.60000 | NA | -66.5135923 | 7.3703504
MCATE | bill_depth_mm | 17.65818 | NA | -68.0137019 | 7.1720825
MCATE | bill_depth_mm | 17.80000 | NA | -70.8577581 | 6.7119253
MCATE | bill_depth_mm | 17.80000 | NA | -70.8577581 | 6.7119253
MCATE | bill_depth_mm | 17.80000 | NA | -70.8577581 | 6.7119253
MCATE | bill_depth_mm | 17.90000 | NA | -72.6652651 | 6.4423247
MCATE | bill_depth_mm | 17.90000 | NA | -72.6652651 | 6.4423247
MCATE | bill_depth_mm | 17.90000 | NA | -72.6652651 | 6.4423247
MCATE | bill_depth_mm | 17.97091 | NA | -73.7084158 | 6.2865123
MCATE | bill_depth_mm | 18.00000 | NA | -74.1371876 | 6.2247369
MCATE | bill_depth_mm | 18.10000 | NA | -74.8246048 | 6.0527094
MCATE | bill_depth_mm | 18.10000 | NA | -74.8246048 | 6.0527094
MCATE | bill_depth_mm | 18.10000 | NA | -74.8246048 | 6.0527094
MCATE | bill_depth_mm | 18.20000 | NA | -74.5109418 | 5.9492791
MCATE | bill_depth_mm | 18.28182 | NA | -74.3437371 | 5.9314729
MCATE | bill_depth_mm | 18.30000 | NA | -74.3344190 | 5.9347189
MCATE | bill_depth_mm | 18.40000 | NA | -74.0829887 | 5.9780680
MCATE | bill_depth_mm | 18.48727 | NA | -73.6293648 | 6.0363607
MCATE | bill_depth_mm | 18.50000 | NA | -73.5586596 | 6.0487677
MCATE | bill_depth_mm | 18.50000 | NA | -73.5586596 | 6.0487677
MCATE | bill_depth_mm | 18.50000 | NA | -73.5586596 | 6.0487677
MCATE | bill_depth_mm | 18.60000 | NA | -72.8630056 | 6.1216814
MCATE | bill_depth_mm | 18.60000 | NA | -72.8630056 | 6.1216814
MCATE | bill_depth_mm | 18.60000 | NA | -72.8630056 | 6.1216814
MCATE | bill_depth_mm | 18.70000 | NA | -71.8332803 | 6.1974441
MCATE | bill_depth_mm | 18.70000 | NA | -71.8332803 | 6.1974441
MCATE | bill_depth_mm | 18.80000 | NA | -70.4284770 | 6.2936190
MCATE | bill_depth_mm | 18.80000 | NA | -70.4284770 | 6.2936190
MCATE | bill_depth_mm | 18.90000 | NA | -68.5821796 | 6.4699829
MCATE | bill_depth_mm | 18.90000 | NA | -68.5821796 | 6.4699829
MCATE | bill_depth_mm | 18.90000 | NA | -68.5821796 | 6.4699829
MCATE | bill_depth_mm | 19.00000 | NA | -66.6711602 | 6.7111429
MCATE | bill_depth_mm | 19.00000 | NA | -66.6711602 | 6.7111429
MCATE | bill_depth_mm | 19.00000 | NA | -66.6711602 | 6.7111429
MCATE | bill_depth_mm | 19.10000 | NA | -64.3957144 | 6.9657114
MCATE | bill_depth_mm | 19.12000 | NA | -63.8639508 | 7.0269676
MCATE | bill_depth_mm | 19.20000 | NA | -60.9094836 | 7.2469231
MCATE | bill_depth_mm | 19.32364 | NA | -57.4071794 | 7.6220474
MCATE | bill_depth_mm | 19.40000 | NA | -55.7903016 | 7.8953027
MCATE | bill_depth_mm | 19.50000 | NA | -54.0195854 | 8.2691850
MCATE | bill_depth_mm | 19.50000 | NA | -54.0195854 | 8.2691850
MCATE | bill_depth_mm | 19.60000 | NA | -52.5044732 | 8.6413269
MCATE | bill_depth_mm | 19.70000 | NA | -50.8069926 | 9.0981814
MCATE | bill_depth_mm | 19.80000 | NA | -49.0884272 | 9.6186280
MCATE | bill_depth_mm | 19.90000 | NA | -48.2422289 | 10.0597659
MCATE | bill_depth_mm | 20.00000 | NA | -47.7016897 | 10.4419900
MCATE | bill_depth_mm | 20.00000 | NA | -47.7016897 | 10.4419900
MCATE | flipper_length_mm | 181.00000 | NA | -84.0161967 | 5.2326472
MCATE | flipper_length_mm | 182.00000 | NA | -84.0967412 | 5.0683455
MCATE | flipper_length_mm | 183.63636 | NA | -85.2348662 | 5.0037976
MCATE | flipper_length_mm | 184.00000 | NA | -85.6844092 | 4.9890108
MCATE | flipper_length_mm | 184.00000 | NA | -85.6844092 | 4.9890108
MCATE | flipper_length_mm | 185.00000 | NA | -86.6140376 | 4.9772330
MCATE | flipper_length_mm | 185.00000 | NA | -86.6140376 | 4.9772330
MCATE | flipper_length_mm | 185.00000 | NA | -86.6140376 | 4.9772330
MCATE | flipper_length_mm | 186.00000 | NA | -86.7576997 | 4.9669797
MCATE | flipper_length_mm | 186.00000 | NA | -86.7576997 | 4.9669797
MCATE | flipper_length_mm | 187.00000 | NA | -86.6186035 | 4.9487053
MCATE | flipper_length_mm | 187.00000 | NA | -86.6186035 | 4.9487053
MCATE | flipper_length_mm | 187.00000 | NA | -86.6186035 | 4.9487053
MCATE | flipper_length_mm | 187.00000 | NA | -86.6186035 | 4.9487053
MCATE | flipper_length_mm | 187.00000 | NA | -86.6186035 | 4.9487053
MCATE | flipper_length_mm | 188.00000 | NA | -85.9258479 | 4.9009817
MCATE | flipper_length_mm | 188.00000 | NA | -85.9258479 | 4.9009817
MCATE | flipper_length_mm | 189.00000 | NA | -84.7604577 | 4.8837210
MCATE | flipper_length_mm | 189.00000 | NA | -84.7604577 | 4.8837210
MCATE | flipper_length_mm | 189.94545 | NA | -83.4630925 | 4.8942677
MCATE | flipper_length_mm | 190.00000 | NA | -83.3734436 | 4.8971922
MCATE | flipper_length_mm | 190.00000 | NA | -83.3734436 | 4.8971922
MCATE | flipper_length_mm | 190.00000 | NA | -83.3734436 | 4.8971922
MCATE | flipper_length_mm | 190.00000 | NA | -83.3734436 | 4.8971922
MCATE | flipper_length_mm | 190.00000 | NA | -83.3734436 | 4.8971922
MCATE | flipper_length_mm | 190.00000 | NA | -83.3734436 | 4.8971922
MCATE | flipper_length_mm | 191.00000 | NA | -81.3312360 | 4.9423130
MCATE | flipper_length_mm | 191.00000 | NA | -81.3312360 | 4.9423130
MCATE | flipper_length_mm | 191.00000 | NA | -81.3312360 | 4.9423130
MCATE | flipper_length_mm | 191.00000 | NA | -81.3312360 | 4.9423130
MCATE | flipper_length_mm | 191.14545 | NA | -80.9695900 | 4.9461808
MCATE | flipper_length_mm | 192.00000 | NA | -78.6370777 | 5.0163755
MCATE | flipper_length_mm | 192.00000 | NA | -78.6370777 | 5.0163755
MCATE | flipper_length_mm | 193.00000 | NA | -75.4185163 | 5.1216394
MCATE | flipper_length_mm | 193.00000 | NA | -75.4185163 | 5.1216394
MCATE | flipper_length_mm | 193.00000 | NA | -75.4185163 | 5.1216394
MCATE | flipper_length_mm | 193.00000 | NA | -75.4185163 | 5.1216394
MCATE | flipper_length_mm | 193.27273 | NA | -74.4850894 | 5.1592443
MCATE | flipper_length_mm | 194.00000 | NA | -71.9847866 | 5.3040104
MCATE | flipper_length_mm | 195.00000 | NA | -67.9721533 | 5.6568389
MCATE | flipper_length_mm | 195.00000 | NA | -67.9721533 | 5.6568389
MCATE | flipper_length_mm | 195.00000 | NA | -67.9721533 | 5.6568389
MCATE | flipper_length_mm | 195.00000 | NA | -67.9721533 | 5.6568389
MCATE | flipper_length_mm | 195.00000 | NA | -67.9721533 | 5.6568389
MCATE | flipper_length_mm | 195.00000 | NA | -67.9721533 | 5.6568389
MCATE | flipper_length_mm | 196.00000 | NA | -63.0110839 | 6.0989182
MCATE | flipper_length_mm | 196.00000 | NA | -63.0110839 | 6.0989182
MCATE | flipper_length_mm | 196.00000 | NA | -63.0110839 | 6.0989182
MCATE | flipper_length_mm | 197.00000 | NA | -57.1171892 | 6.5831259
MCATE | flipper_length_mm | 197.00000 | NA | -57.1171892 | 6.5831259
MCATE | flipper_length_mm | 197.00000 | NA | -57.1171892 | 6.5831259
MCATE | flipper_length_mm | 197.52727 | NA | -53.7767360 | 6.8537436
MCATE | flipper_length_mm | 198.00000 | NA | -50.4871677 | 7.1160691
MCATE | flipper_length_mm | 198.00000 | NA | -50.4871677 | 7.1160691
MCATE | flipper_length_mm | 199.00000 | NA | -42.6560338 | 7.6791550
MCATE | flipper_length_mm | 199.00000 | NA | -42.6560338 | 7.6791550
MCATE | flipper_length_mm | 200.00000 | NA | -33.6497856 | 8.2009771
MCATE | flipper_length_mm | 200.63636 | NA | -27.1191953 | 8.5347171
MCATE | flipper_length_mm | 201.00000 | NA | -22.9367473 | 8.7235768
MCATE | flipper_length_mm | 201.67273 | NA | -15.7080780 | 8.9713835
MCATE | flipper_length_mm | 202.00000 | NA | -12.2278195 | 9.0432388
MCATE | flipper_length_mm | 203.00000 | NA | -1.2741143 | 9.2491815
MCATE | flipper_length_mm | 204.45455 | NA | 16.2334839 | 9.6154355
MCATE | flipper_length_mm | 205.74545 | NA | 30.6578533 | 9.4887529
MCATE | flipper_length_mm | 207.76364 | NA | 52.6901778 | 8.5393664
MCATE | flipper_length_mm | 208.00000 | NA | 55.0811100 | 8.3766626
MCATE | flipper_length_mm | 208.00000 | NA | 55.0811100 | 8.3766626
MCATE | flipper_length_mm | 209.00000 | NA | 64.5369424 | 7.8231111
MCATE | flipper_length_mm | 209.00000 | NA | 64.5369424 | 7.8231111
MCATE | flipper_length_mm | 210.00000 | NA | 73.5437696 | 7.3263284
MCATE | flipper_length_mm | 210.00000 | NA | 73.5437696 | 7.3263284
MCATE | flipper_length_mm | 210.00000 | NA | 73.5437696 | 7.3263284
MCATE | flipper_length_mm | 210.00000 | NA | 73.5437696 | 7.3263284
MCATE | flipper_length_mm | 210.92727 | NA | 81.3520194 | 6.8749029
MCATE | flipper_length_mm | 212.00000 | NA | 89.0229426 | 6.4090240
MCATE | flipper_length_mm | 212.00000 | NA | 89.0229426 | 6.4090240
MCATE | flipper_length_mm | 212.98182 | NA | 94.5071838 | 6.1081718
MCATE | flipper_length_mm | 213.00000 | NA | 94.5946264 | 6.1038495
MCATE | flipper_length_mm | 214.00000 | NA | 99.2413474 | 5.8400140
MCATE | flipper_length_mm | 214.00000 | NA | 99.2413474 | 5.8400140
MCATE | flipper_length_mm | 215.00000 | NA | 102.7838554 | 5.6215167
MCATE | flipper_length_mm | 215.00000 | NA | 102.7838554 | 5.6215167
MCATE | flipper_length_mm | 215.00000 | NA | 102.7838554 | 5.6215167
MCATE | flipper_length_mm | 215.00000 | NA | 102.7838554 | 5.6215167
MCATE | flipper_length_mm | 216.00000 | NA | 104.7073428 | 5.5120379
MCATE | flipper_length_mm | 216.00000 | NA | 104.7073428 | 5.5120379
MCATE | flipper_length_mm | 217.00000 | NA | 105.8188237 | 5.6487617
MCATE | flipper_length_mm | 217.18182 | NA | 106.0126538 | 5.7085062
MCATE | flipper_length_mm | 218.00000 | NA | 106.7292937 | 5.8758048
MCATE | flipper_length_mm | 219.00000 | NA | 107.0753353 | 6.1383574
MCATE | flipper_length_mm | 219.00000 | NA | 107.0753353 | 6.1383574
MCATE | flipper_length_mm | 220.00000 | NA | 107.3051373 | 6.3824910
MCATE | flipper_length_mm | 220.00000 | NA | 107.3051373 | 6.3824910
MCATE | flipper_length_mm | 220.29091 | NA | 107.3434214 | 6.4468056
MCATE | flipper_length_mm | 221.00000 | NA | 107.4646758 | 6.5949082
MCATE | flipper_length_mm | 222.00000 | NA | 107.3483437 | 6.8482332
MCATE | flipper_length_mm | 222.00000 | NA | 107.3483437 | 6.8482332
MCATE | flipper_length_mm | 223.00000 | NA | 106.5251739 | 7.0552770
MCATE | flipper_length_mm | 224.00000 | NA | 104.2704023 | 7.2682986
MCATE | flipper_length_mm | 225.00000 | NA | 101.8649689 | 7.3184353
VIMP | species | NA | NA | 0.0712735 | 0.0142229
VIMP | island | NA | NA | 0.0000000 | 0.0011779
VIMP | sex | NA | NA | 0.0000000 | 0.0005214
VIMP | year | NA | NA | 0.0000000 | 0.0009405
VIMP | bill_length_mm | NA | NA | 0.0020426 | 0.0021568
VIMP | bill_depth_mm | NA | NA | 0.0000000 | 0.0012858
VIMP | flipper_length_mm | NA | NA | 0.0002417 | 0.0015137
MSE | food_consumed_g | NA | Control Response | 27.1199595 | 2.3646324
MSE | food_consumed_g | NA | Treatment Response | 783.2769673 | 90.2786749
SL risk | SL.glm_All | NA | Control Response | 28.6118060 | 0.6913950
SL risk | custom_SL.glmnet_0_All | NA | Control Response | 26.9653481 | 0.4276785
SL risk | custom_SL.glmnet_1_All | NA | Control Response | 26.9462735 | 0.4947490
SL risk | SL.glm_All | NA | Treatment Response | 840.5589154 | 16.3885080
SL risk | custom_SL.glmnet_0_All | NA | Treatment Response | 910.1614653 | 24.3456084
SL risk | custom_SL.glmnet_1_All | NA | Treatment Response | 804.5305186 | 24.3270504
SL coefficient | SL.glm_All | NA | Control Response | 0.1950071 | 0.0948603
SL coefficient | custom_SL.glmnet_0_All | NA | Control Response | 0.1707645 | 0.1707645
SL coefficient | custom_SL.glmnet_1_All | NA | Control Response | 0.6342284 | 0.2281237
SL coefficient | SL.glm_All | NA | Treatment Response | 0.1291916 | 0.0780640
SL coefficient | custom_SL.glmnet_0_All | NA | Treatment Response | 0.0000000 | 0.0000000
SL coefficient | custom_SL.glmnet_1_All | NA | Treatment Response | 0.8708084 | 0.0780640
SATE | NA | NA | NA | -8.0714040 | 5.3185760

The `estimand` column denotes the class of Quantities of Interest to be estimated, and include such values as “MCATE” and “VIMP”. The `term` column denotes the covariate being referred to (if relevant) in the results. For example, a calculated MCATE refers to a particular covariate, indicated by this column. The columns `value` and `level` refer to quantities which indicate a more granular division within a particular term, with the former representing a numeric value while the latter indicates a categorical one. For example, in the case of the MCATE, if the covariate is discrete, the value at which the MCATE was calculated would be in the `level` column, while if it were continuous, it would be in the `value` column. Each of these columns is, therefore, type-stable. The final two columns for `estimate` and `std_error` are self explanatory. The details on their calculation lie with the particular Quantity of Interest requested.

## Plotting MCATEs

It’s then a simple matter to plot results using code like the following:

    filter(results, estimand == "MCATE", is.na(value)) %>%
    ggplot(aes(level, estimate)) +
    geom_point() +
    geom_linerange(aes(ymin = estimate - 1.96 * std_error, ymax = estimate + 1.96 * std_error)) +
    geom_hline(yintercept = 0, linetype = "dashed") +
    coord_flip() +
    facet_wrap(~term, scales = "free_y")

    filter(results, estimand == "MCATE", is.na(level)) %>%
    ggplot(aes(value, estimate)) +
    geom_line() +
    geom_ribbon(
      aes(ymin = estimate - 1.96 * std_error, ymax = estimate + 1.96 * std_error),
      alpha = 0.5
    ) +
    geom_hline(yintercept = 0, linetype = "dashed") +
    scale_x_continuous("Covariate value") +
    scale_y_continuous("CATE") +
    coord_flip() +
    facet_wrap(~term, scales = "free_y")

## Plotting diagnostics

Similarly, it’s easy to plot diagnostic information:

    filter(results, estimand == "SL risk") %>%
    ggplot(aes(reorder(term, estimate), estimate)) +
    geom_point() +
    geom_linerange(
      aes(ymin = estimate - 1.96 * std_error, ymax = estimate + 1.96 * std_error),
      alpha = 0.5
    ) +
    geom_hline(yintercept = 0, linetype = "dashed") +
    scale_x_discrete("") +
    scale_y_continuous("Risk") +
    facet_wrap(~level) +
    coord_flip()

    filter(results, estimand == "SL coefficient") %>%
    ggplot(aes(reorder(term, estimate), estimate)) +
    geom_point() +
    geom_linerange(
      aes(ymin = estimate - 1.96 * std_error, ymax = estimate + 1.96 * std_error),
      alpha = 0.5
    ) +
    geom_hline(yintercept = 0, linetype = "dashed") +
    scale_x_discrete("") +
    scale_y_continuous("Coefficient") +
    facet_wrap(~level) +
    coord_flip()

## Plotting VIMP

And finally, we can examine the variable importance measure of Williamson et al. (2021) when applied to a joint model of the pseudo-outcome:

    filter(results, estimand == "VIMP") %>%
    ggplot(aes(reorder(term, estimate), estimate)) +
    geom_point() +
    geom_linerange(
      aes(ymin = estimate - 1.96 * std_error, ymax = estimate + 1.96 * std_error),
      alpha = 0.5
    ) +
    geom_hline(yintercept = 0, linetype = "dashed") +
    scale_x_discrete("") +
    scale_y_continuous("Reduction in R²") +
    coord_flip()

# Replicating analyses

One of the biggest benefits of `tidyhte` is its ability to repeat similar analyses.

As an example, to perform the same analysis used above on a different outcome would require the following code:

    penguins %<>% produce_plugin_estimates(
      # outcome
      body_mass_g,
      # treatment
      treatment,
      # covariates
      species, island, sex, year, bill_length_mm, bill_depth_mm, flipper_length_mm
    ) %>%
    construct_pseudo_outcomes(body_mass_g, treatment) %>%
    estimate_QoI(
      species, island, sex, year, bill_length_mm, bill_depth_mm, flipper_length_mm
    ) -> results_mass

All the same quantities can be easily plotted for this outcome as well, and results may be joined together conveniently.

    results_all <- bind_rows(
      results %>% mutate(outcome = "food_consumed_g"),
      results_mass %>% mutate(outcome = "body_mass_g")
    )

By allowing the user to flexibly compose HTE estimators, it drastically reduces the amount of work necessary for a typical HTE analysis which by nature tends to involve multiple moderators, models and outcomes.

# Conclusion

This paper has introduced the concepts underlying the `tidyhte` package and given examples as to how the package can be used. In general, the package is written in a sufficiently general way that some features can be added with relatively little work. For instance, adding new plugin models is as simple as providing some information on configuration as well as standardizing train / predict methods. This is similarly true for providing new ways to summarize CATEs for plotting, which is handled in much the same way. This makes it relatively easy for methodologists to fit in their preferred fitting methods and thereby extending `tidyhte`’s functionality.

# Acknowledgements

We gratefully acknowledge the collaboration with the US 2020 Facebook and Instagram Election Project for being a testbed for the initial versions of `tidyhte`, particularly Pablo Barberá at Meta. We received no financial support for this software project.

# References

Abadie, Alberto, Susan Athey, Guido W Imbens, and Jeffrey M Wooldridge. 2023. “When Should You Adjust Standard Errors for Clustering?” The Quarterly Journal of Economics 138 (1): 1–35.
Calonico, Sebastian, Matias D Cattaneo, and Max H Farrell. 2019. “Nprobust: Nonparametric Kernel-Based Estimation and Robust Bias-Corrected Inference.” arXiv Preprint arXiv:1906.00198.
Chernozhukov, Victor, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. 2018. “Double/Debiased Machine Learning for Treatment and Structural Parameters.” The Econometrics Journal 21 (1): C1–68.
Fan, Jianqing, and Irene Gijbels. 2018. Local Polynomial Modelling and Its Applications: Monographs on Statistics and Applied Probability 66. Routledge.
Higgins, Michael J, Fredrik Sävje, and Jasjeet S Sekhon. 2016. “Improving Massive Experiments with Threshold Blocking.” Proceedings of the National Academy of Sciences 113 (27): 7369–76.
Horst, Allison M, Alison Presmanes Hill, and Kristen B Gorman. 2020. “Allisonhorst/Palmerpenguins: V0.1.0.” Zenodo. https://doi.org/10.5281/zenodo.3960218.
Kennedy, Edward H. 2020. “Optimal Doubly Robust Estimation of Heterogeneous Causal Effects.” arXiv Preprint arXiv:2004.14497.
Robins, James M, Andrea Rotnitzky, and Lue Ping Zhao. 1995. “Analysis of Semiparametric Regression Models for Repeated Outcomes in the Presence of Missing Data.” Journal of the American Statistical Association 90 (429): 106–21.
Tsiatis, Anastasios A. 2006. “Semiparametric Theory and Missing Data.”
Tsiatis, Anastasios A, Marie Davidian, Min Zhang, and Xiaomin Lu. 2008. “Covariate Adjustment for Two-Sample Treatment Comparisons in Randomized Clinical Trials: A Principled yet Flexible Approach.” Statistics in Medicine 27 (23): 4658–77.
Van der Laan, Mark J, MJ Laan, and James M Robins. 2003. Unified Methods for Censored Longitudinal Data and Causality. Springer Science & Business Media.
Van der Laan, Mark J, Eric C Polley, and Alan E Hubbard. 2007. “Super Learner.” Statistical Applications in Genetics and Molecular Biology 6 (1).
Williamson, Brian D, Peter B Gilbert, Marco Carone, and Noah Simon. 2021. “Nonparametric Variable Importance Assessment Using Machine Learning Techniques.” Biometrics 77 (1): 9–22.
Zou, Hui, and Trevor Hastie. 2005. “Regularization and Variable Selection via the Elastic Net.” Journal of the Royal Statistical Society Series B: Statistical Methodology 67 (2): 301–20.