
<!-- README.md is generated from README.Rmd. Please edit that file -->

# adjustr <a href="https://corymccartan.com/adjustr/"><img src="man/figures/logo.png" align="right" height="138" /></a>

<!-- badges: start -->

[![R-CMD-check](https://github.com/CoryMcCartan/adjustr/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/CoryMcCartan/adjustr/actions/workflows/R-CMD-check.yaml)
[![Codecov test
coverage](https://codecov.io/gh/CoryMcCartan/adjustr/branch/main/graph/badge.svg)](https://app.codecov.io/gh/CoryMcCartan/adjustr?branch=main)
<!-- badges: end -->

Sensitivity analysis is a critical component of a good modeling
workflow. Yet as the number and power of Bayesian computational tools
has increased, the options for sensitivity analysis have remained
largely the same: compute importance sampling weights manually, or fit a
large number of similar models, dramatically increasing computation
time. Neither option is satisfactory for most applied modeling.

**adjustr** is an R package which aims to make sensitivity analysis
faster and easier, and works with Bayesian models fitted with
[Stan](https://mc-stan.org). Users provide a series of alternate
sampling specifications, and the package uses Pareto-smoothed importance
sampling to estimate the posterior under each specification. The package
also provides functions to summarize and plot how posterior quantities
change across specifications.

The package provides a simple interface that makes it as easy as
possible for researchers to try out various adjustments to their Stan
models, without needing to write any specific Stan code or even
recompile or rerun their model.

The package works by parsing Stan model code, so everything works best
if the model was written by the user. Models made using **brms** may in
principle be used as well. Models made using **rstanarm** are
constructed using more complex model templates, and cannot be used.

## Installation

Install from CRAN with:

``` r
install.packages("adjustr")
```

Or install the development version from GitHub:

``` r
# install.packages("remotes")
remotes::install_github("CoryMcCartan/adjustr")
```

## Getting Started

The basic **adjustr** workflow is as follows:

1.  Use `make_spec()` to specify the set of alternative model
    specifications you’d like to fit.

2.  Use `adjust_weights()` to calculate importance sampling weights
    which approximate the posterior of each alternative specification.

3.  Use `summarize()` and `spec_plot()` to examine posterior quantities
    of interest for each alternative specification, in order to assess
    the sensitivity of the underlying model.

To illustrate, the package lets us do the following:

``` r
extract_samp_stmts(eightschools_m)
#> Sampling statements for model 2c8d1d8a30137533422c438f23b83428:
#>   parameter   eta ~ std_normal()
#>   data        y ~ normal(theta, sigma)

make_spec(eta ~ student_t(0, 1, df), df=1:10) %>%
    adjust_weights(eightschools_m) %>%
    summarize(wasserstein(mu))
#> # A tibble: 11 x 5
#>       df .samp                     .weights      .pareto_k `wasserstein(mu)`
#>    <int> <chr>                     <list>            <dbl>             <dbl>
#>  1     1 eta ~ student_t(df, 0, 1) <dbl [1,000]>     1.02              0.928
#>  2     2 eta ~ student_t(df, 0, 1) <dbl [1,000]>     1.03              0.736
#>  3     3 eta ~ student_t(df, 0, 1) <dbl [1,000]>     0.915             0.534
#>  4     4 eta ~ student_t(df, 0, 1) <dbl [1,000]>     0.856             0.411
#>  5     5 eta ~ student_t(df, 0, 1) <dbl [1,000]>     0.826             0.341
#>  6     6 eta ~ student_t(df, 0, 1) <dbl [1,000]>     0.803             0.275
#>  7     7 eta ~ student_t(df, 0, 1) <dbl [1,000]>     0.782             0.234
#>  8     8 eta ~ student_t(df, 0, 1) <dbl [1,000]>     0.753             0.195
#>  9     9 eta ~ student_t(df, 0, 1) <dbl [1,000]>     0.736             0.166
#> 10    10 eta ~ student_t(df, 0, 1) <dbl [1,000]>     0.721             0.151
#> 11    NA <original model>          <dbl [1,000]>  -Inf                 0
```

The tutorial
[vignette](https://corymccartan.com/adjustr/articles/adjustr.html) walks
through a full sensitivity analysis for this 8-schools example. Smaller
examples are also included in the package
[documentation](https://corymccartan.com/adjustr/reference/index.html).

## References

Vehtari, A., Simpson, D., Gelman, A., Yao, Y., & Gabry, J. (2024).
Pareto smoothed importance sampling. *Journal of Machine Learning
Research*, 25(72), 1–58. <https://doi.org/10.48550/arXiv.1507.02646>
