---
title: "reflimR: Estimation of reference limits from routine laboratory results"
author: "Georg Hoffmann, Sandra Klawitter, Inga Trulson & Frank Klawonn"

output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{reflimR: Estimation of reference limits from routine laboratory results}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>"
)
```

1.  [Introduction](#introduction)
2.  [Installation](#installation)
3.  [Dataset](#data)
4.  [Usage](#usage)
5.  [Quality Assessment](#quality)
6.  [Conclusion](#conclusion)
7.  [References](#references)

# Introduction <a name = "introduction"></a> {#introduction}

Reference intervals play a crucial role in the medical interpretation and statistical evaluation of laboratory results. By definition, reference intervals include the central 95% of results measured in non-diseased reference individuals [1].

In conventional approaches, reference intervals are determined using *direct methods*, which involve collecting laboratory results from a minimum of 120 apparently healthy individuals and calculating the 2.5th and 97.5th percentiles with parametric or non-parametric methods [1]. Although direct methods are currently the gold standard, their implementation can be difficult due to cost and time issues, limited resources, or ethical considerations. In addition, the definition of "obviously healthy" according to medical criteria is rather vague.

In this article, we present an alternative method that is included in the *reflimR* package [2]. It is an *indirect method* [3] for deriving reference limits from mixed populations, which contain an unknown proportion of sick individuals (usually less than 25 percent). Here, the definition of "obviously healthy" is not based on medical criteria, but on the statistical criterion of a homogeneous distribution to which a normal distribution can be fitted after suitable transformation.

Indirect methods are particularly useful when routine data from a laboratory information system are available and direct sampling of non-diseased individuals is not feasible.

# Installation <a name = "installation"></a> {#installation}

To install the *reflimR* package, open R and enter the following command in the console:

```{r, eval = FALSE}
install.packages("reflimR")
```

This command will download the package from CRAN. Once the installation is complete, load the package into your R session using the following command:

```{r}
library(reflimR)
```

The package is now ready for use in your R environment (e.g. in RStudio). To see the documentation of the package with all its help files, please enter

```{r}
help(package = reflimR)
```

# Dataset <a name = "data"></a>

*reflimR* comes with the [*livertests*](https://archive.ics.uci.edu/dataset/571/hcv+data) dataset, which has been specifically prepared for the purpose of reference interval estimation with direct and indirect methods. The dataset includes eight different biomarkers (laboratory tests) that have been measured in healthy controls and in patients with liver diseases. Here is an example excerpt from the dataset:

```{r echo = FALSE}
print(livertests[c(1, 204, 444, 589),])
```

In addition, the corresponding reference intervals are stored in the *targetvalues* dataset. The reference interval table has been derived partly from the electronic handbook [Clinical Laboratory Diagnostics](https://www.clinical-laboratory-diagnostics.com) published by L. Thomas, and partly from the manufacturer's assay sheet.

```{r, echo=FALSE}
targetvalues
```

To adress a specific target interval, e. g. for albumin in women, you may use the following command:

```{r}
targetvalues[1, 3 : 4]
```

# Usage <a name = "usage"></a> {#usage}

The *reflimR* package includes several functions, with `reflim(x)` serving as the main function. It calls the other important functions that can be arranged in three groups:

| group 1                  | group 2           | group 3        |
|--------------------------|-------------------|----------------|
| ri_hist                  | lognorm           | adjust_digits  |
| permissible_uncertainty  | iboxplot          | bowley         |
| interpretation           | truncated_qqplot  | conf_int95     |

Group 1 includes the main functions that provide the user with the final results of `reflim()` and help to interpret them: `ri_hist()` creates a graphical output (histogram with reference interval and density curves), `permissible_uncertainty()` calculates the medical tolerance limits of the results [4, 5], and `interpretation()` assesses the medical significance of deviations from given target values [2].

Group 2 performs the three underlying statistical operations *modelling*, *truncation*, and *calculation*, and group 3 contains auxiliary functions for miscellaneous tasks. Details of each function are available in the respective help files, which can be addressed with the `help()` function or a question mark followed by the function name (e.g. `help(reflim)` or `?lognorm`).

The *reflim* function can be executed by simply calling `reflim(x)`, where x is a vector of numeric data to be analyzed. For example, if the reference interval for bilirubin is to be estimated from the values included in the *livertests* dataset, the following two lines of code will suffice:

```{r fig.width=5, fig.height=5}
x <- livertests$BIL
reflim(x)
```

Please note that any filtering and cleaning (e. g. for removal of non-numeric values or duplicate results of a single individual) or partitioning of the data (for sex, age, etc) is not included in the *reflimR* package. These steps can easily be performed separately with base R functions such as `as.numeric()`, `unique()` or `subset()` to achieve the desired results.

As shown above, the main result of the *reflim* function is an illustrative graphic showing a histogram of the original data with a dotted overall density curve. The estimated reference limits are displayed as dashed vertical lines and the respective medical tolerance limits as gray bars. The blue solid density curve represents the theoretical distribution of the assumed reference population, and the density curves of potential pathological outliers are shown in red.

Below the graphic, there is a list of statistical results such as the parameters of the theoretical distribution (*stats*), the assumed distribution model (*lognormal* = TRUE or FALSE), and the calculated reference limits with tolerance intervals (*limits*). On the bottom of the list is an estimate of the 95 percent confidence intervals for the lower and upper reference limits (*confidence.int*). The remaining results are set to NA if no target values were specified.

The *reflim* function comes with additional arguments that default to meaningful values for a quick overview. They are accessible via the help file (`?reflim`) and can be changed if needed. For example, if you want the graphic to be nicely labeled and the targets from the *targetvalues* dataset to be verified, you can specify the *main*, *xlab*, and *targets* arguments. And if you want to limit the output of the function to some essentials such as just the interpretation of the graphic, you can specify what you want to see with a \$ after the closing parenthesis.

```{r fig.width=5, fig.height=5}
x <- livertests$PROT
reflim(x, main = "total protein", xlab = "g/L", targets = targetvalues[8, 3 : 4])$interpretation
```

### **Interpretation with traffic light colors**

If target values have been specified, the other two functions of group 1 are used to check whether the calculated reference limits are within the permitted tolerance limits and assigns traffic light colors to visually indicate any deviations. Green means that the calculated limit is within the tolerance interval of the target, yellow means that the calculated limit falls outside but the two tolerance intervals overlap, and red means that there is no overlap between the tolerance intervals.

`ri_hist()`, `permissible_uncertainty()` and `interpretation()` are called by the *reflim* function, but can also be used independently for the evaluation of reference limits obtained by other methods (e.g., from the *refineR* package).

### **Three steps to the result**

As already mentioned, *reflimR* performs an indirect three-step approach. The underlying functions form the group 2 in the above table. Here is an overview with some practical examples.

**Step 1:** To define an appropriate distribution model, Bowley's quartile skewness coefficient [6] is calculated for the original and the logarithmized data, using the *bowley* function from group 3. The underlying algorithm is quite robust against pathological outliers, because it is calculated from the central 50 percent of the data (the "box" of Tukey's boxplot), where the influence of pathological values is minimized [7]. If the skewness coefficient is close to 0, the distribution is symmetric and can be approximated by a normal distribution (at least in the range of interest between the 2.5th and 97.5th percentiles). If, however, the skewness coefficient is clearly positive, the distribution is right-skewed. If the shape of the curve becomes more symmetric after log-transformation of the data, the *lognorm* function indicates that the distribution can be better approximated by a lognormal model.

Here is an example of a symmetric distribution from the albumin data using the *lognorm* function:

```{r fig.width=5, fig.height=5}
lognorm(livertests$ALB, main = "albumin", xlab = "g/L")
```

The skewness coefficient of the original values is 0.019 and the shape of the black curve is not much changed after log-transformation (blue). The difference of the two skewness coefficients is 0.038, i. e. below the threshold of 0.05, which has been defined as a fixed value in the *lognorm* function [7].

In contrast, alanine aminotransferase is an example for an asymmetric distribution:

```{r fig.width=5, fig.height=5}
lognorm(livertests$ALT, main = "alanine aminotransferase", xlab = "U/L")
```

Here, the original distribution curve (black) is clearly right-skewed with a skewness coefficient of 0.207, and becomes more symmetrical after logarithmization of the data (blue). The delta of the skewness coefficients is 0.171 and thus above the threshold of 0.05.

If a lognormal distribution is recommended in step 1, the values are logarithmized by the *reflim* function so that the normal distribution assumption can be used in the next two steps.

**Step 2**: Subsequently, the data set is truncated iteratively in multiple steps to remove  pathological values while preserving the central 95% of the presumably non-pathological values as accurately as possible [8]. Instead of immediately truncating at the theoretical 2.5th and 97.5th percentiles (as implemented in the original package version 1.0.6), a more cautious, step-by-step truncation process has been introduced in the updated package version 1.1.0. This is done by successively adjusting the truncation threshold to the 99.5th, 99th, 98.5th, 0.98th, and 97.5th percentiles. After the last truncation step, the algorithm is continued with the final threshold of 97.5 percent until no more outliers can be removed.

Here is an application of the *iboxplot* function using the bilirubin dataset as an example:

```{r fig.width=5, fig.height=5}
x <- livertests$BIL
trunc.bil <- iboxplot(x, main = "bilirubin", xlab = "µmol/L")
```

The upper boxplot (white) shows a considerable amount of dots outside the whiskers, which are effectively removed with seven cycles of the *iboxplot* algorithm (blue boxplot). The eighth cycles confirms that the algorithm has converged:


```{r}
trunc.bil$progress
```

The *iboxplot* function derives the estimated percentage of "normal values" from the length of the truncated vector *trunc.bil\$trunc*, divided by the proportion of assumed non-pathological values (e.g. 0.95 as a default) :

```{r}
trunc.bil$perc.norm
```

Notably, the lower and upper truncation points are rough estimates of the reference limits (i.e. the 2.5th and 97.5th quantile of the non-pathological values):

```{r}
trunc.bil$truncation.points
```

**Step 3:** To improve the robustness of this estimate even further, a normal quantile-quantile plot (Q-Q plot) is created from the truncated data [9]. The function `truncated_qqplot()` determines the mean and standard deviation from the intercept and the slope of the regression line and extrapolates the 2.5th and 97.5th percentiles from the equation *mean ± 1.96 sd*. These percentiles represent the estimated reference interval, and thus the final result.

```{r fig.width=5, fig.height=5}
truncated_qqplot(trunc.bil$trunc)
```

If the calculation was carried out with logarithmized values, the resulting reference limits are automatically delogarithmized.

As already mentioned in step 2, the truncation points of step 2 are close to the final result of step 3, but they are not identical. The reason is that extrapolating the respective percentiles from the linear Q-Q plot obviously stabilizes the result and makes it more robust against minor overlaps with pathological values at the ends of the regression line (see above). To achieve this, only the central 90 percent of the Q-Q plot are used for the calculation of the regression line.

# Quality Assessment <a name = "quality"></a>

If you want to visualize the whole process at a glance, you can set the *plot.all* argument of the *reflim* function to TRUE to display four graphics. Here is an example of AST results for women:

```{r fig.width=8, fig.height=8}
ast.f <-  reflim(livertests$AST[livertests$Sex == "f"], plot.all = TRUE, n.min = 150,
       targets = targetvalues[3, 3 : 4],
       main = "AST (f)", xlab = "U/L")
```

In the present case, the Q-Q plot is quite linear, so the reference interval seems credible. The linearity of the Q-Q plot is a visual measure of the data quality. However, if you do the same analysis with total protein (PROT) for women, you will obtain a density curve with a small pathological fraction that partially overlaps with the "box" of the boxplot:

```{r fig.width=6, fig.height=4}
prot.f <- livertests$PROT[livertests$Sex == "f"] 
ln <- lognorm(prot.f, main = "PROT (f)", xlab = "g/L")
arrows(76, 0.055, 76, 0.075, code = 1, length = 0.1, lwd = 2)
```

This fraction cannot be completely removed by the *iboxplot* algorithm:

```{r fig.width=6, fig.height=4}
xtrunc.f <- iboxplot(prot.f, xlab = "g/L")$trunc 
arrows(77.5, 0.07, 77.5, 0.09, code = 1, length = 0.1, lwd = 2)
```

As a result, the Q-Q plot shows a characteristic deviation from linearity at the upper right end:

```{r fig.width=6, fig.height=4}
qq.f <- truncated_qqplot(xtrunc.f, n.min = 100) 
arrows(1.4, 80, 1.4, 77.5, code = 2, length = 0.1, lwd = 2)
```

Therefore, the estimated reference interval should be interpreted with caution. In such cases, we recommend to repeat the test using a comparative method (e.g. *refineR*) and to consult the literature. You may also try to run the reflim function with a higher value of the argument perc.trunc (e.g. 5 instead of 2.5) to achieve a stronger truncation of the values:

```{r fig.width=6, fig.height=6}
reflim(prot.f, perc.trunc = 5, n.min = 100,  plot.all = TRUE, print.n = FALSE, 
       main = "PROT (f)", xlab = "g/L")$limits[1 : 2]
```

The two lower graphics demonstrate that the subfraction of pathological values has now been removed (left graphic) and the Q-Q plot becomes linear (right graphic). As a consequence, the resulting reference interval becomes slightly smaller.

# Conclusion <a name = "conclusion"></a> {#conclusion}

The *reflimR* package provides a simple and robust three-step procedure for the estimation of reference intervals from routine laboratory data that may contain an unknown proportion of pathological results. Due to the fact that the initial two steps of the algorithm are based on quartiles (i. e. the central 50 percent of the original results), the proportion of potentially pathological values should not exceed 25 percent on one side of the distribution.

Compared to many other algorithms, the *reflim* function is extremely fast. This is mainly achieved by the fact that it does not use elaborate procedures to optimize a theoretical distribution model: For symmetric distributions, it assumes a normal distribution in the range between the 2.5th and 97.5th percentiles, and for right-skewed distributions, the data are logarithmized to approximate a lognormal distribution in the same range. Left-skewed distributions are assumed to be mixtures of a symmetric distribution and pathological outliers on the left side. If the distribution type is unclear, a lognormal distribution can be assumed (by setting the *lognorm* argument to TRUE) without risking large errors in the estimation [10].

Due to an intuitive traffic light color scheme, the estimated reference limits can easily be verified against target values from external sources (e. g., from package inserts or handbooks) [2]. The assessment of any deviations is based on the degree of overlap between the corresponding tolerance intervals, which makes the interpretation easy and transparent.

Our method falls into the category of the so-called "modified Hoffmann approaches", which are derived from the original method published by RG Hoffmann in 1963 [11]. The common element of these methods is that they are based on the evaluation of a regression line obtained by comparing the distribution of observed values with a standard normal distribution. In the original Hoffmann method, a probability-probability plot is generated, whereas most of the modifications including ours use a quantile-quantile plot.

Other modifications concern a) the transformation of the original data to approximate a normal distribution, b) the type of truncation to eliminate potentially pathological outliers, and c) the identification of the linear part of the regression line. In the original Hoffmann method [11] as well as in our own original modification [9], this straight part was identified by visual inspection, whereas the method included in this package is fully automated and independent of any subjective judgment.

# References <a name = "references"></a> {#references}

1. Clinical and Laboratory Standards Institute document EP28-A3c—Defining, Establishing, and Verifying Reference Intervals in the Clinical Laboratory. 2010;ISBN 1-56238-682-4. <https://clsi.org/shop/standards/ep28/>.

2. Hoffmann, G., Klawitter, S., Trulson, I., Adler, J., Holdenrieder, S., & Klawonn, F. (2024). A Novel Tool for the Rapid and Transparent Verification of Reference Intervals in Clinical Laboratories. Journal of clinical medicine, 13(15), 4397. <https://doi.org/10.3390/jcm13154397>.

3. Jones G, Haeckel R, Loh T, Sikaris K, Streichert T, Katayev A, Barth J, Ozarda Y. Indirect methods for reference interval determination: Review and recommendations. Clin. Chem. Lab. Med. 2018, 57, 20–29.<https://doi.org/10.1515/cclm-2018-0073>

4.  Haeckel R et al. Permissible limits for uncertainty of measurement in laboratory medicine. Clin Chem Lab Med 2015;53:1161–71. <https://doi.org/10.1515/cclm-2014-0874>.

5.  Haeckel R, Wosniok W, Arzideh F. Equivalence limits of reference intervals for partitioning of population data. J Lab Med 2016; 40: 199-205. <https://doi.org/10.1515/labmed-2016-0002>.

6.  Bowley, AL (1920). Elements of Statistics. London : P.S. King & Son, Ltd.

7.  Klawonn F, Hoffmann G, Orth M. Quantitative laboratory results: normal or lognormal distribution. J Lab Med 2020; 44: 143–50. <https://doi.org/10.1515/labmed-2020-0005>.

8.  Klawonn F, Hoffmann G. Using fuzzy cluster analysis to find interesting clusters. In: L.A. Garcia-Escuderoet al. (eds.): Building bridges between soft and statistical methodologies for data science. Springer, Cham (2023), 231-239. <https://doi.org/10.1007/978-3-031-15509-3_31>.

9.  Hoffmann G, Lichtinghagen R, Wosniok W. Simple estimation of reference intervals from routine laboratory data. J Lab Med 2016. <https://doi.org/10.1515/labmed-2015-0104>.

10.  Haeckel R, Wosniok W. Observed, unknown distributions of clinical chemical quantities should be considered to be log-normal: a proposal. Clin Chem Lab Med. 2010; 48: 1393–6 <https://doi.org/10.1515/CCLM.2010.273>.

11.  Hoffmann RG. Statistics in the practice of medicine. J Am Med Assoc 1963; 185: 864–73.

