
<!-- README.md is generated from README.Rmd. Please edit that file -->

# midr <img src="man/figures/logo.png" align="right" height="138"/>

<!-- badges: start -->

[![R-CMD-check](https://github.com/ryo-asashi/midr/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/ryo-asashi/midr/actions/workflows/R-CMD-check.yaml)
[![CRAN
status](https://www.r-pkg.org/badges/version/midr)](https://CRAN.R-project.org/package=midr)

<!-- badges: end -->

The goal of ‘midr’ is to provide a model-agnostic method for
interpreting and explaining black-box predictive models by creating a
globally interpretable surrogate model. The package implements ‘Maximum
Interpretation Decomposition’ (MID), a functional decomposition
technique that finds an optimal additive approximation of the original
model. This approximation is achieved by minimizing the squared error
between the predictions of the black-box model and the surrogate model.
The theoretical foundations of MID are described in Iwasawa & Matsumori
(2026) \[Forthcoming\], and the package itself is detailed in [Asashiba
et al. (2025)](https://arxiv.org/abs/2506.08338).

## Installation

You can install the released version of midr from
[CRAN](https://cran.r-project.org/) with:

``` r
install.packages("midr")
```

and the development version from [GitHub](https://github.com/) with:

``` r
# install.packages("devtools")
devtools::install_github("ryo-asashi/midr")
```

## Examples

In the following example, we fit a random forest model to the `Boston`
dataset included in ISLR2, and then attempt to interpret it using the
functions of midr.

``` r
# load required packages
library(midr)
library(ggplot2)
library(gridExtra)
library(ISLR2)
library(ranger)
theme_set(theme_midr("xy"))
# split the Boston dataset
data("Boston", package = "ISLR2")
set.seed(42)
idx <- sample(nrow(Boston), nrow(Boston) * .75)
train <- Boston[ idx, ]
valid <- Boston[-idx, ]
# fit a random forest model
rf <- ranger(medv ~ ., train, mtry = 5)
preds_rf <- predict(rf, valid)$predictions
cat("RMSE: ", weighted.loss(valid$medv, preds_rf))
#> RMSE:  3.351362
```

The first step is to create a MID model as a global surrogate of the
target model using `interpret()`.

``` r
# fit a two-dimensional MID model
mid <- interpret(medv ~ .^2, train, rf, lambda = .1)
mid
#> 
#> Call:
#> interpret(formula = medv ~ .^2, data = train, model = rf, lambda = 0.1)
#> 
#> Model Class: ranger
#> 
#> Intercept: 22.446
#> 
#> Main Effects:
#> 12 main effect terms
#> 
#> Interactions:
#> 66 interaction terms
#> 
#> Uninterpreted Variation Ratio: 0.016248
preds_mid <- predict(mid, valid)
cat("RMSE: ", weighted.loss(preds_rf, preds_mid))
#> RMSE:  1.106667
cat("RMSE: ", weighted.loss(valid$medv, preds_mid))
#> RMSE:  3.306324
```

To visualize the main and interaction effects of the variables, apply
`ggmid()` or `plot()` to the fitted MID model.

``` r
# visualize the main and interaction effects of the MID model
grid.arrange(
  ggmid(mid, "lstat") +
    labs(title = "Feature Effects",
         subtitle = "Main Effect of 'lstat'"),
  ggmid(mid, "dis") +
    labs(title = "", subtitle = "Main effect of 'dis'"),
  ggmid(mid, "lstat:dis") +
    labs(subtitle = "Interaction Effect of 'lstat:dis'"),
  ggmid(mid, "lstat:dis", main.effects = TRUE, type = "compound") +
    labs(subtitle = "Total Effect of 'lstat:dis'")
)
```

<img src="man/figures/README-ggmid-1.png" width="100%" />

``` r
# visualize all main effects
grid.arrange(grobs = mid.plots(mid), nrow = 3)
```

<img src="man/figures/README-ggmid-2.png" width="100%" />

`mid.importance()` helps to compute and compare the importance of main
and interaction effects.

``` r
# visualize the MID importance of the component functions
imp <- mid.importance(mid)
grid.arrange(nrow = 1L,
  ggmid(imp, fill = "steelblue", max.nterms = 20) +
    theme(legend.position = "none") +
    labs(title = "Feature Effect Importance",
         subtitle = "Bar Plot"),
  ggmid(imp, "heatmap", linewidth=.25, color="gray85") +
    theme(legend.position = "none") +
    labs(title = "", subtitle = "Heat Map")
)
```

<img src="man/figures/README-mid_importance-1.png" width="100%" />

`mid.breakdown()` provides a way to analyze individual predictions by
decomposing the differences between the intercept and the predicted
value into variable effects.

``` r
# visualize the MID breakdown of the model predictions
bd1 <- mid.breakdown(mid, data = train, row = 1L)
bd9 <- mid.breakdown(mid, data = train, row = 9L)
grid.arrange(nrow = 1L,
  ggmid(bd1, fill = "steelblue") +
    theme(legend.position = "none") +
    labs(title = "Additive Breakdown of Predictions",
         subtitle = "Waterfall Plot for Row 1"),
  ggmid(bd9, theme = "shap", size = 3, type = "dotchart") +
    theme(legend.position = "none") +
    labs(title = "", subtitle = "Dot Chart for Row 9")
)
```

<img src="man/figures/README-mid_breakdown-1.png" width="100%" />

`mid.conditional()` can be used to compute the ICE curves (Goldstein et
al. 2015) of the fitted MID model, as well as the breakdown of the ICE
curves by main and interaction effects.

``` r
# visualize the ICE curves of the MID model
ice <- mid.conditional(mid, "lstat", max.nsamples = 100)
grid.arrange(
  ggmid(ice, color = "steelblue") +
    labs(title = "Individual Conditional Expectation",
         subtitle = "ICE Plot of 'lstat'"),
  ggmid(ice, "centered", color = "steelblue") +
    labs(title = "", subtitle = "c-ICE Plot of 'lstat'"),
  ggmid(ice, term = "lstat:dis", var.color = dis) +
    labs(subtitle = "Interaction Effect with 'dis'"),
  ggmid(ice, term = "lstat:age", var.color = age) +
    labs(subtitle = "Interaction Effect with 'age'")
)
```

<img src="man/figures/README-mid_conditional-1.png" width="100%" />

## References

\[1\] Iwasawa, H. & Matsumori, Y. (2026). “A Functional Decomposition
Approach to Maximize the Interpretability of Black-Box Models”.
\[Forthcoming\]

\[2\] Asashiba, R., Kozuma, R. & Iwasawa, H. (2025). “midr: Learning
from Black-Box Models by Maximum Interpretation Decomposition”.
<https://arxiv.org/abs/2506.08338>

\[3\] Goldstein, A., Kapelner, A., Bleich, J., & Pitkin, E. (2015).
“Peeking Inside the Black Box: Visualizing Statistical Learning With
Plots of Individual Conditional Expectation”. *Journal of Computational
and Graphical Statistics*, *24*(1), 44–65.
<https://doi.org/10.1080/10618600.2014.907095>
