# doctest

# Introduction to doctest

Documentation examples and tests are similar in some ways:

- They are self-contained pieces of code.

- They should cover the software’s most important functions and typical uses.

- They should be simple and clear: complex examples are hard for users to understand, and complex test code can introduce testing bugs.

This similarity makes it attractive to use “doctests”, which combine tests and documentation. Indeed, several languages, including Python and Rust, have doctests built in.1 R also checks for errors in examples when running `R CMD check`.

The doctest package extends this idea. It lets you write testthat tests, by adding tags to your roxygen documentation. This helps you check that your examples do what they are supposed to do.

## Example

Here’s some roxygen documentation for a function:

    #' Fibonacci function
    #'
    #' @param n Integer
    #' @return The nth Fibonacci number
    #'
    #' @doctest
    #'
    #' @expect type("integer")
    #' fib(2)
    #'
    #' n <- 6
    #' @expect equal(8)
    #' fib(n)
    #'
    #' @expect warning("not numeric")
    #' fib("a")
    #'
    #' @expect warning("NA")
    #' fib(NA)
    fib <- function (n) {
      if (! is.numeric(n)) warning("n is not numeric")
      ...
    }

Instead of an `@examples` section, we have a `@doctest` section.

This will create tests like:

    # Generated by doctest: do not edit by hand
    # Please edit file in R/<text>

    test_that("Doctest: fib", {
      # Created from @doctest for `fib`
      # Source file: <text>
      expect_type(fib(2), "integer")
      n <- 6
      expect_equal(fib(n), 8)
      expect_warning(fib("a"), "not numeric")
      expect_warning(fib(NA), "NA")
    })

The .Rd file will be created as normal, with an example section like:

    \examples{
    fib(2)

    n <- 6
    fib(n)
    fib("a")
    fib(NA)
    }

## Usage

Install doctest from r-universe:

    install.packages("doctest", repos = c("https://hughjonesd.r-universe.dev",
                                          "https://cloud.r-project.org"))

Or from CRAN:

    install.packages("doctest")

Or get the development version:

    devtools::install("hughjonesd/doctest")

To use doctest in your package, alter its DESCRIPTION file to add the `dt_roclet` roclet and `"doctest"` package to roxygen:

    Roxygen: list(roclets = c("collate", "rd", "namespace",
                  "doctest::dt_roclet"), packages = "doctest")

Then use `roxygen2::roxygenize()` or `devtools::document()` to build your package documentation.

## Adding doctests to your package

Here’s a simple workflow to start using doctest:

1. Alter your package DESCRIPTION as above.

2. In your roxygen documentation, replace `@examples` by `@doctest`.

3. In the package directory run `roxygen2::roxygenize()` or `devtools::document()` to create documentation. You should see Rd files created as normal in the `man/` directory, including `\examples` sections.

4. Add `@expect` tags to your `@doctest` sections.

5. Run `roxygenize()` again. You will now see new files created in the `tests/testthat` directory, with the name `test-doctest-<topic name>.R`.

6. Run `devtools::test()` and check that your tests pass.

At present, you can’t use doctest from the RStudio keyboard shortcut `Ctrl + Shift + D`, because this always uses the standard roxygen2 roclets. However, you can bind the RStudio addin “Devtools: document a package” to a keyboard shortcut. This will use the roclets from your package DESCRIPTION file.

You don’t need to add doctest as a dependency to your package. Just like roxygen2 itself, you can use it to create help files and tests without it being installed for users. However, you may wish to add it in `Suggests:`, to help other developers working on the package:

    usethis::use_package("doctest", type = "Suggests")

## Tags

The doctest package adds these tags to roxygen:

### `@doctest`

Use `@doctest` instead of `@examples`:

    #' @doctest
    #'
    #' # ... examples for your function

The content of `@doctest` will be used in the .Rd “examples” section, and in a testthat test.

You can have more than one `@doctest` section. Each section creates one test like `test_that("Test name", {...})`. You can name the doctest, or leave it blank for a default name. All the sections will be merged into a single .Rd example.

    #' @doctest Positive numbers
    #' x <- 1
    #' @expect equal(x)
    #' abs(x)
    #'
    #' @doctest Negative numbers
    #' x <- -1
    #' @expect equal(-x)
    #' abs(x)

### `@expect`

`@expect` writes a testthat expectation.

    #' @expect equal(4)
    #' 2 + 2

You can use any `expect_*` function from `testthat`. Omit the `expect_` at the start of the call.

The expression on the next line will be substituted as the first argument into the `expect` call:

    expect_equal(2 + 2, 4)

Use a dot `.` to substitute in different places:

    #' @expect equal(., rev(.))
    #' c("T", "E", "N", "E", "T")

This becomes:

    expect_equal(c("T", "E", "N", "E", "T"), rev(c("T", "E", "N", "E", "T")))

### `@expectRaw`

`@expectRaw` writes an expectation, without substituting the next expression:

    #' x <- 2 + 2
    #' @expectRaw equal(x, 4)

### `@snap`

`@snap` is shorthand for `@expect snapshot()`. This creates a snapshot test, which is useful for checking that complex examples haven’t changed.

### `@testRaw`

`@testRaw` adds an arbitrary line of code to your test:

    #' @testRaw skip_on_cran("Takes too long to run")
    #'
    #' @expect equal(6765)
    #' fib(20)

Tests are only written if they contain at least one `@expect` or `@expectRaw` tag, so use those tags to create expectations, not `@testRaw`.

### `@omit` and `@resume`

While `@testRaw` includes a line of code in the test but not the example, `@omit` does the opposite: it includes all following code in the example but not the test. You can use `@resume` to restart including lines without creating a new expectation.

    #' myfunc(1)
    #'
    #' @omit
    #' # No need to test plotting
    #' plot(1:10, my_func(1:10))
    #'
    #' @resume
    #' x <- NA
    #' @expect warning()
    #' myfunc(x)

If you are using `@testRaw` and `@omit` a lot, it is probably a good idea to separate out the test and the example. You can do this by renaming the `test-doctest-` file, and removing the “Generated by doctest” line within it. Then change your `@doctest` tag back to `@examples`.

### `@doctestExample`

`@doctestExample filename.R` includes the R code in filename.R as an example. It is a drop-in replacement for roxygen2’s `@example`. The R code isn’t checked for doctest tags and isn’t included in any tests.

## Caveats

- Don’t use `@doctest` and `@examples` in the same topic. That won’t work.

- Doctest currently ignores `\dontrun` and `\donttest` macros. Potentially, that could lead to dangerous code being included in tests. To avoid this, use the `@omit` tag.

- Each `@doctest` section should include a complete self-contained example, that would work inside a `test_that` expression. Don’t rely on variables from a previous `@doctest`.

- You can include expectations within e.g. `if` blocks or `for` loops. Don’t forget that each roxygen tag must be indented with a single space:

    #' # Right:
    #' if (TRUE) {
    #' @expect equals(4)
    #'   2+2
    #' }
    #' # Wrong:
    #' if (TRUE) {
    #'   @expect equals(4)
    #'   2+2
    #' }

## Writing good doctests

Tests and documentation are similar, but not identical. Tests need to cover difficult corner cases. Examples need to convey the basics to the user. I like the following advice:

> … write the best possible documentation, and [R] makes sure the code samples in your documentation actually compile and run [and do what they are supposed to do]

Programming Rust, Blandy, Orendorff and Tindall, 2021

In particular, use doctest as an addition to manually created tests, not a substitute for them. Use doctest to make sure your examples do what they expect, and for simple tests of basic functionality. If it’s hard to specify what to test for, consider using `@snap` to capture output:

    #' @snap
    summary(model)

For more complex test cases, write a test file manually.

To see an example of using the doctest package in “production”, check out `vignette("conversion")`.

## Related packages

The roxytest and roxut packages both allow you to write tests in roxygen blocks. Doctest is slightly different because it combines tests with examples. The exampletestr package uses roxygen examples to generate a test skeleton which you can fill in yourself.

1. https://docs.python.org/3/library/doctest.html, https://doc.rust-lang.org/rustdoc/write-documentation/documentation-tests.html↩︎