%% Journal of Open Research Software Latex template -- Created By Stephen Bonner and John Brennan, Durham Universtiy, UK.

\documentclass{josr}



%% revised text
\newcommand{\new}[1]{{%\color{blue}
#1}}

%% packages
\usepackage{natbib}

\usepackage{subfig}
\usepackage{amsmath}
\usepackage{tikz}
\usetikzlibrary{shapes,arrows,positioning}

\usepackage{booktabs}

%% Nice inline code
\makeatletter
\newcommand*{\textalltt}{}
\DeclareRobustCommand*{\textalltt}{%
  \begingroup
    \let\do\@makeother
    \dospecials
    \catcode`\\=\z@
    \catcode`\{=\@ne
    \catcode`\}=\tw@
    \verbatim@font\@noligs
    \@vobeyspaces
    \frenchspacing
    \@textalltt
}
\newcommand*{\@textalltt}[1]{%
    #1%
  \endgroup
}
\makeatother


%% Set the header information
\pagestyle{fancy}
\definecolor{mygray}{gray}{0.6}
\renewcommand\headrule{}
\rhead{\footnotesize 3}
\rhead{\textcolor{gray}{UP JORS software Latex paper template version 0.1}}

\begin{document}
\hypersetup{pageanchor=false}

<<setup, echo=FALSE, message=FALSE>>=
library("knitr")
knitr::opts_chunk$set(cache = TRUE, message = FALSE, fig.align = "center")

# knit_hooks$set(inline = function(x) {
#   if (is.numeric(x)) return(knitr:::format_sci(x, 'latex'))
#   highr:::hi_latex(x)
# })

library("dplyr")
library("stringr")

knit_hooks$set(inline = function(x) {
  if (is.numeric(x)) return(knitr:::format_sci(x, 'latex'))
  highr:::hi_latex(x) %>%  str_replace("hlopt", "textalltt")
  })
@



{\bf Software paper for submission to the Journal of Open Research Software} \\

%To complete this template, please replace the blue text with your own. The paper has three main sections: (1) Overview; (2) Availability; (3) Reuse potential. \\

%Please submit the completed paper to: editor.jors@ubiquitypress.com

\rule{\textwidth}{1pt}

\section*{(1) Overview}

\vspace{0.5cm}

\section*{Title}
model4you: An R package for personalised treatment effect estimation


\section*{Paper Authors}
1. Seibold, Heidi; 2. Zeileis, Achim; 3. Hothorn, Torsten



\section*{Paper Author Roles and Affiliations}
1. Software, Conceptualization, Writing – original draft; Institute for Medical
Information Processing, Biometry, and Epidemiology, LMU Munich

2. Conceptualization, Writing – review and editing; Department of Statistics, Faculty of Economics and Statistics,
University of Innsbruck

3. Conceptualization, Writing – review and editing; Biostatistics Department, Epidemiology, Biostatistics \&
Prevention Institute, University of Zurich



\section*{Abstract}
Typical models estimating treatment effects assume that the treatment effect is
the same for all individuals.  Model-based recursive partitioning allows to
relax this assumption and to estimate stratified treatment effects (model-based
trees) or even personalised treatment effects (model-based forests).  With
model-based trees one can compute treatment effects for different strata of
individuals. The strata are found in a data-driven fashion and depend on
characteristics of the individuals.  Model-based random forests allow for a
similarity estimation between individuals in terms of model parameters (e.g.~
intercept and treatment effect). The similarity measure can then be used to
estimate personalised models.  The R package \emph{model4you} implements these
stratified and personalised models \new{in the setting with two randomly
assigned treatments} with a focus on ease of use and interpretability so that
clinicians and other users can take the model they usually use for the
estimation of the average treatment effect and with a few lines of code get a
visualisation that is easy to understand and interpret.

%\textcolor{blue}{A short (ca. 100 word) summary of the software being described: what problem the software addresses, how it was implemented and architected, where it is stored, and its reuse potential.}

\section*{Keywords}

personalised medicine; subgroup analysis; model-based recursive partitioning;
unbiased trees; treatment effect; random forest


\section*{Introduction}

%\textcolor{blue}{An overview of the software, how it was produced, and the research for which it has been used, including references to relevant research articles. A short comparison with software which implements similar functionality should be included in this section. }
Studies in various fields randomly assign individuals to one of two groups with
different exposure and then measure a response. For example, in clinical trials
patients are assigned to one of two treatment groups where usually one
treatment group receives a new treatment or drug and the other treatment group
receives the standard of care or a placebo. Other examples are in A-B testing
in marketing studies or any other two group comparisons such as the mathematics
exam discussed below, where students were divided into different exam groups
and received slightly different exam tasks. In the following we will refer to
the two groups as \emph{treatment groups} and to the group indicator as
\emph{treatment indicator}, which always takes values $0$ (individual in first
group) and $1$ (individual in second group).

Treatment effect estimation is often done using simple models with the binary
treatment indicator as only covariate. In the example of a clinical trial the
treatment indicator would be $1$ if the patient receives the new treatment and
$0$ if the patient receives standard of care.  In R such a simple model can be
estimated as follows:
<<eval=FALSE>>=
base_model <- model(response ~ treatment, data)
@
with \texttt{\Sexpr{'response'}} being the response measured, \texttt{\Sexpr{'treatment'}} being
the treatment indicator and \texttt{\Sexpr{'data'}} being the data set containing these
variables.  The function \texttt{\Sexpr{'model()'}} can be replaced for example by
\texttt{\Sexpr{'lm()'}} to estimate a linear model, \texttt{\Sexpr{'glm()'}} to estimate a
generalised linear model or \texttt{\Sexpr{'survreg()'}} to estimate a parametric
survival model.  These models estimate intercept and treatment effect for all
individuals in the \texttt{\Sexpr{'data'}} and allow for predicting the response of
other individuals given they do or do not receive the treatment of interest.

For cases where the assumption that all individuals have the same intercept and
treatment effect is too strict the R package \emph{model4you} offers two
options:

\paragraph{1. Model-based trees} identify subgroups where within the subgroups
the model parameters are similar and between groups the model parameters are
different. This is achieved by finding instabilities in the model parameters
with respect to a variable (characteristic) and recursively partitioning the
data into groups. If, for example the algorithm finds that men and women have
differing treatment effects, the data are partitioned into two subgroups.
Details on model-based trees in general can be found in
\citep{zeileis_model-based_2008} and for the special use case for stratified
treatment effect estimation in \citep{seibold_model-based_2016}.  Just a single
line of code lets the user compute a model-based tree in R:
<<eval=FALSE>>=
strat_models <- pmtree(base_model)
@
Note that \texttt{\Sexpr{'pmtree()'}} uses the data given in the call of the base model.
It automatically uses variables not used in the model formula (in the example
above \texttt{\Sexpr{'response ~ treatment'}}) as potential subgroup defining variables.
This can be edited using the \texttt{\Sexpr{'zformula'}} argument.

\paragraph{2. Personalised models} use model-based random forests to estimate
similarity of individuals in terms of model parameters. For each individual a
personalised model can be estimated based on a weighted set of the original
data, where the similarity measure corresponds to the weight. Details on the
personalised models can be found in \citep{seibold_individual_2017}.  Computing
personalised models for all observations in the training data is simple:
<<eval=FALSE>>=
pm_forest <- pmforest(base_model)
pers_models <- pmodel(pm_forest)
@
Again here the potential effect-modifying variables are taken by default as all
variables not given in the model formula and can be defined using the
\texttt{\Sexpr{'zformula'}} argument in \texttt{\Sexpr{'pmforest()'}}.


In the following we will present an example application for model-based trees
and personalised models. For this we need to load the package and -- to ensure
reproducibility -- set a random seed.
\new{
Also for visualisations we need packages
\emph{ggplot2} \citep{ggplot2}, \emph{ggbeeswarm} \citep{ggbeeswarm} and
\emph{gridExtra} \citep{gridExtra}. The data used is
available in the \emph{psychotools} package \citep{psychotools}.
}
<<prep>>=
library("model4you")
set.seed(2017)

library("ggplot2")
theme_set(theme_classic())
library("ggbeeswarm")
library("gridExtra")
@

\paragraph{Mathematics exam analysis:}
In 2014 first-year business and economics students at the University of
Innsbruck were divided into two examination groups. Group 1 took the exam in
the morning and group 2 started after the first group finished. The exams for
the two groups were slightly different.  The data can be accessed and prepared
as follows:

<<math_data>>=
data("MathExam14W", package = "psychotools")

## scale points achieved to [0, 100] percent
MathExam14W$tests <- 100 * MathExam14W$tests/26
MathExam14W$pcorrect <- 100 * MathExam14W$nsolved/13

## select variables to be used
MathExam <- MathExam14W[ , c("pcorrect", "group", "tests", "study",
                             "attempt", "semester", "gender")]
@
\new{To investigate the correlation between exam group and exam performance, we
compute a simple linear model regressing the percentage points of correct
answers on the exam group.}
<<math_bmod>>=
bmod_math <- lm(pcorrect ~ group, data = MathExam)
@
The estimates and confidence intervals of this model can be computed via
<<>>=
cbind(estimate = coef(bmod_math), confint(bmod_math))
@
The model can be visualised by plotting the estimated densities
(see Figure~\ref{fig:math_bmod_vis}):
<<math_bmod_vis, fig.height=2.5, fig.width=5, out.width="0.7\\textwidth", fig.cap = "Density estimates of base model for the Mathematics Exam data.">>=
lm_plot(bmod_math)
@
Both the estimates and confidence intervals and the density curves suggest that
there is almost no difference between the two groups. But does this really hold
for all types of students?

A tree based on this model can be computed and visualised in only two lines of
code:
<<math_tree, fig.width = 13, fig.cap = "Personalised model tree for the Mathematics Exam datam.">>=
tr_math <- pmtree(bmod_math, control = ctree_control(maxdepth = 2))
plot(tr_math, terminal_panel = node_pmterminal(tr_math,
                                               plotfun = lm_plot))
@
\new{
We restrict the depth of the tree to two (\texttt{\Sexpr{'maxdepth = 2'}}) for
illustration purposes. If this setting is not used a more complex tree would be
estimated. Also the cutpoints and effect estimates are rounded in the plot.  To
view the tree in the R console, the \texttt{\Sexpr{'print()'}} function can be
used.
}
<<math_printtree>>=
print(tr_math)
@
The tree (see Figure~\ref{fig:math_tree}) divides students based on the
percentage of successful online \texttt{\Sexpr{'tests'}}. These online tests were
conducted biweekly throughout the semester.  The largest difference between the
two exam groups is in the students who did very well in the online tests (more
than \texttt{\Sexpr{round(tr_math[[5]]$node$split$breaks, 1)}} percent correct). The
tree thus gives us much more information on the fairness of the exam than the
simple linear model, which is that it does not seem to be fair for students who
did very well throughout the semester (at this point we should state that the
students self selected into the two exam groups which might also be the reason
for differences in exam performance).
\new{Note that confidence intervals shown in the tree are not adjusted for
selection of the cutpoints in the tree and
should hence be interpreted as a measure for variablity and not be used for
inference.}

Estimating personalised models is almost as simple as the stratified models:
<<math_pmodl>>=
forest_math <- pmforest(bmod_math)
pmods_math <- pmodel(forest_math)

## model parameters of first 6 students
head(pmods_math)
@

Dependence plots with the group effect (treatment effect) on the y-axis and
the student characteristics on the x-axis are a good way of visualising the
personalised models and for getting knowledge about the interactions between
student characteristics and the exam group.
\new{
Note that the plot is related to but not the same as the classical partial
dependence plot \citep{friedman2001greedy}, as each point in the figure is one
patient in the given data set.  Since the percentage of successful online tests
is measured on a grid, a beeswarm plot (a variant of jittered scatter plot)
possibly shows the relationships even better than the scatter plot (both shown
in Figure~\ref{fig:math_dp1}, Figure~\ref{fig:math_dp11} includes a loess
curve). We see a nonlinear relation between the percentage of successful online
\texttt{\Sexpr{'tests'}} and the exam group effect. Students with great
performance during the semenster are estimated to have the strongest effect.
}
<<math_dp1, fig.width=4, fig.height=3, out.width='.48\\linewidth', fig.cap="Dependence plot for percentage of tests successfully solved.", fig.subcap=c("Scatter plot with loess curve.", "Beeswarm plot.")>>=
dpdat_math <- cbind(pmods_math, MathExam)

ggplot(dpdat_math, aes(x = tests, y = group2)) +
  geom_point(alpha = 0.2, size = 1) +
  geom_smooth(fill = NA, method = "loess") +
  theme(legend.position = "none") +
  ylab("estimated individual\nexam group effect")

ggplot(dpdat_math, aes(x = tests, y = group2, color = tests)) +
  geom_quasirandom(alpha = 0.5, size = 1) +
  theme(legend.position = "none") +
  ylab("estimated individual\nexam group effect")
@

\new{
For categorical variables such as the number of previous attempts to pass the
exam (\texttt{\Sexpr{'attempt'}}) and \texttt{\Sexpr{'gender'}} we can use box plots,
\new{beeswarm} plots or a combination thereof (as in
Figure~\ref{fig:math_dp21}). Figure~\ref{fig:math_dp22} shows the number of
previous attempts on the x-axis and the color indicates the gender of each
student.  This way we can see that for students writing the exam for the first
time, gender seems to play a bigger role in terms of estimated individual exam
group effect than for others.
}
<<math_dp2, fig.width=4, fig.height=3, out.width='.48\\linewidth', fig.cap="Dependence plots for the number of previous attempts and gender.", fig.subcap=c('Gender.', 'Number of previous attempts and gender.')>>=
ggplot(dpdat_math, aes(x = gender, y = group2, color = gender)) +
  geom_boxplot() +
  geom_quasirandom(alpha = 0.5) +
  theme(legend.position = "none") +
  ylab("estimated individual\nexam group effect")

ggplot(dpdat_math, aes(x = attempt, y = group2, color = gender)) +
  geom_quasirandom(alpha = 0.5) +
  ylab("estimated individual\nexam group effect")
@

With the tools provided by the \emph{model4you} package it is very
simple to create understandable stratified and personalised models
and compelling visualisations that can be used to communicate these
models.



\section*{Implementation and architecture}
The R package \emph{model4you} is focused on ease of use and
interpretability.  Users can take a simple model that they know and
understand as basis.
\new{
Models currently available for use are linear models (function
\texttt{\Sexpr{'lm()'}} in R), generalised linear models (\texttt{\Sexpr{'glm()'}}) and
survival models (\texttt{\Sexpr{'survreg()'}} and \texttt{\Sexpr{'coxph()'}} from the R
package \emph{survival}).  Models are restricted to those with a
single binary covariate (e.g.~ two treatment options).  The user can
simply plug the model object into \texttt{\Sexpr{'pmtree()'}} or
\texttt{\Sexpr{'pmforest()'}} depending on whether they want subgroup wise or
personalised models.
}

The basis for these functionalities is provided by the
\emph{partykit} package which is a widely used R package for trees
and forests \citep{hothorn_partykit_2015, hothorn_partykit_2017}.
The \emph{model4you} package provides wrappers for the well
implemented and tested functions \texttt{\Sexpr{'partykit::ctree()'}} and
\texttt{\Sexpr{'partykit::cforest()'}} and extends the functionalities to
allow for the computation of personalised models and to improve
usability and interpretability.

The \emph{partykit} package provides the basis for functionalities in
other packages namely \emph{glmertree}, \emph{psychotree},
\emph{betareg}, \emph{disttree}, \emph{lagsarlmtree}, \emph{palmtree}
(all on CRAN), and \emph{trtf} (currently avilable on R-Forge).

%\textcolor{blue}{How the software was implemented, with details of the architecture where relevant. Use of relevant diagrams is appropriate. Please also describe any variants and associated implementation differences.}



\section*{Quality control}
All packages on CRAN undergo standard checks for compatibility with the R
package ecosystem.  The R package contains examples and tests. These were run
and checked on Linux 86\_64 and Windows.

%\textcolor{blue}{Detail the level of testing that has been carried out on the code (e.g. unit, functional, load etc.), and in which environments. If not already included in the software documentation, provide details of how a user could quickly understand if the software is working (e.g. providing examples of running the software with sample input and output data). }

\section*{(2) Availability}
\vspace{0.5cm}
\section*{Operating system}

Should work on all operating systems that run R.
%\textcolor{blue}{Please include minimum version compatibility.}

\section*{Programming language}
R (version 3.1.0 or higher)

\section*{Additional system requirements}
None.
%\textcolor{blue}{E.g. memory, disk space, processor, input devices, output devices.}

\section*{Dependencies}
R, partykit package (version 1.2 or higher)

\section*{List of contributors}
Same as the authors:
Heidi Seibold, Achim Zeileis and Torsten Hothorn


\section*{Software location:}

{\bf Archive} %\textcolor{blue}{(e.g. institutional repository, general repository) (required ??? please see instructions on journal website for depositing archive copy of software in a suitable repository)}

\begin{description}[noitemsep,topsep=0pt]
\item[Name:] CRAN
\item[Persistent identifier:] \url{https://cran.r-project.org/package=model4you} %TODO upload to CRAN
\item[Licence:] GPL-2 $\vert$ GPL-3
\item[Publisher:] Heidi Seibold
\item[Version published:] 0.9-3
\item[Date published:] 12/11/18
\end{description}



{\bf Code repository} %\textcolor{blue}{(e.g. SourceForge, GitHub etc.) (required)}

\begin{description}[noitemsep,topsep=0pt]
\item[Name:] R-forge
\item[Persistent identifier:] \url{https://r-forge.r-project.org/projects/partykit/}
\item[Licence:] GPL-2 $\vert$ GPL-3
\item[Date published:] 18/01/19
\end{description}


\section*{Language}

English


\section*{(3) Reuse potential}

The software is intentionally written to make usage as simple as possible. The
most prominent use case are clinical trials where the assumption of an average
treatment effect for all patients is too strict and the efficacy of the
treatment depends on patient characteristics (e.g.~ gender, biomarkers, etc.).
For subgroup analyses (stratified treatment effects) model-based trees
(\texttt{\Sexpr{'pmtree()'}}) can be used; For personalised treatment effects
model-based forests (\texttt{\Sexpr{'pmforest()'}}) provide a way of estimating
similarity between patients and using this similarity measure to estimate
personalised models (\texttt{\Sexpr{'pmodel()'}}).  The target audience are people who
deal with heterogeneous treatment effects, such as medical researchers,
pharmaceutical companies or analysts in marketing (A-B testing).  In general
the software is useful to researchers dealing with scenarios where two
exposures are compared and responses of subjects possibly depend on other
variables.
\new{
Currently the packages supports a limited set of model types (linear and generalised
linear models and survival models). Further models can be added.
}

We encourage users to use the party tag on Stackoverflow
(\url{http://stackoverflow.com/questions/tagged/party}) in case of questions or
problems.

%\textcolor{blue}{Please describe in as much detail as possible the ways in which the software could be reused by other researchers both within and outside of your field. This should include the use cases for the software, and also details of how the software might be modified or extended (including how contributors should contact you) if appropriate. Also you must include details of what support mechanisms are in place for this software (even if there is no support).}

\section*{Acknowledgements}
%TODO

%\textcolor{blue}{Please add any relevant acknowledgements to anyone else who supported the project in which the software was created, but did not work directly on the software itself.}


\section*{Funding statement}
Heidi Seibold and Torsten Hothorn were financially supported by the Swiss
National Science Foundation (grants 205321\_163456 and IZSEZ0\_177091).

%\textcolor{blue}{If the software resulted from funded research please give the funder and grant number.}



\section*{Competing interests}
The authors declare that they have no competing interests.


% \section*{References}
\bibliographystyle{plainnat}
\bibliography{ref_model4you}

% \textcolor{blue}{Please enter references in the Harvard style and include a DOI where available, citing them in the text with a number in square brackets, e.g. \\ }
%
% \textcolor{blue}{[1] Piwowar, H A 2011 Who Shares? Who Doesn't? Factors Associated with Openly Archiving Raw Research Data. PLoS ONE 6(7): e18657. DOI: \\ http://dx.doi.org/10.1371/journal.pone.0018657.}

\vspace{2cm}

\rule{\textwidth}{1pt}

{ \bf Copyright Notice} \\
Authors who publish with this journal agree to the following terms: \\

Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a  \href{http://creativecommons.org/licenses/by/3.0/}{Creative Commons Attribution License} that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal. \\

Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal. \\

By submitting this paper you agree to the terms of this Copyright Notice, which will apply to this submission if and when it is published by this journal.


\end{document}
