Author: Rafal Urniaz
Date: 2025-10-16

# Introduction to Idempotent Test Listings for testthat: Global and Section Lists

#### Rafal Urniaz

#### 2025-10-16

# Overview

This vignette introduces two functions for generating roxygen-style listings of your `testthat` tests:

- `document_file()`: parses a single test file and inserts a global listing (after `#' @testsList`) and per-section listings (after each `#' @testsSection`). The function is idempotent: re-running replaces only the `@testsItem` blocks and leaves the rest untouched.

- `document()`: walks a `tests/testthat` directory and calls `document_file()` for each file, returning per-file results and a combined table.

The listings are written as roxygen comments:

    #' @testsList
    #' @testsItem 1 alpha one
    #' @testsItem 2 alpha two

    #' @testsSection Section A
    #' @testsItem 1 alpha one
    #' @testsItem 2 alpha two

> Why? Quick navigation and auditing of tests; machine- and human-readable structure that integrates with documentation tooling; and predictable, idempotent re-generation inside CI/CD.

# Key Concepts

## Sections

- Plain section markers (default prefix `"# -"`), like `# - My Section`, are converted to roxygen markers on-the-fly:

      #' @testsSection My Section

- Any text following `@testsSection` on the same line is the section title.

## Markers and Items

- Global marker: `#' @testsList` (one per file; auto-added at top if missing).

- Section marker: `#' @testsSection [Optional title]` (one or many).

- Items: each entry is inserted immediately after the relevant marker:

      #' @testsItem <numbering> <title_expression>

## Titles: Raw but Clean

The first argument to `test_that()` is recorded as raw code: - Literals like `"alpha one"` are kept without the outer quotes → `alpha one`. - Expressions like `paste("a", b)` or `glue::glue("{x}")` are not evaluated and are listed unchanged.

## Numbering Templates

Use placeholders to control numbering in both the global and section listings: - `{g}` — global index (1..N across all tests in the file) - `{s}` — section index (1..S) - `{i}` — local index within a section (1.. per section) - `{l}` — final line number in the modified file (after insertion) - Aliases: `{local}` → `{i}`, `{line}` → `{l}`

Two presets: - `template = "simple"` → `"{g}"` - `template = "advanced"` → `"{g}.{s}.{i}.{l}"`

Override with `global_fmt` and `section_fmt` for full control.

# `document_file()`

## Arguments

- `path` (`character`): path to the test file to process.
- `section_prefix` (`character`): plain-text section header prefix to convert (default: `"# -"`).
- `template` (`"simple"|"advanced"|"custom"`): built-in number formats.
- `global_fmt`, `section_fmt` (`character`): custom templates using `{g}`, `{s}`, `{i}`, `{l}` and aliases.
- `encoding` (`character`): file encoding for read/write (default: `"UTF-8"`).
- `make_backup` (`logical`): if `TRUE`, writes a timestamped backup before overwriting.
- `write` (`logical`): if `TRUE`, overwrites the file. Set to `FALSE` for a dry-run where the function returns the would-be modified text.

## Return Value

A list of class `tests_listing_result` with: - `text`: final modified lines (character vector). - `listing`: data frame with columns: - `g`, `s`, `i`, `l` (final line number), - `title_raw` (cleaned raw title), - `section_title` (if any). - `written` (`logical`), `backup` (path or `NULL`).

## Example

    library(testthatdocs)

    res <- document_file(
      path = system.file("examples", "tests_sample_before.R", package="testthatdocs"),
      section_prefix = "# -",
      template = "advanced",   # or "simple"
      encoding = "UTF-8",
      backup = TRUE,
      write = TRUE
    )

    # Summary of tests
    res$listing

    # Modified test file
    res$text

# `document()`

Recursively processes a tree of test files (by default, `tests/testthat`).

    library(testthatdocs)

    all_res <- document(
      root = system.file("examples", package="testthatdocs"),
      template = "advanced",
      section_prefix = "# -",
      encoding = "UTF-8",
      backup = TRUE,
      write = TRUE,
      quiet = TRUE
    )

    # Combined table (with 'file' column)
    all_res$listing

## Arguments

- `root` (`character`): directory to walk. Default `"tests/testthat"`.
- `pattern` (`character`): regex to choose files (`"^[Tt]est.*\\.[rR]$"`).
- `recurse` (`logical`): whether to search subfolders.
- `exclude` (`character`): basenames to exclude (default `c("testthat.R")`).
- All the options from `document_file()` are passed through and applied to each file.

## Return Value

A list of class `tests_listing_dir_result` with: - `files`: processed file paths, - `results`: per-file `tests_listing_result` objects, - `listing`: combined table (adds a `file` column), - `backups`: vector of backup file paths (when `write=TRUE`).

# Idempotency Guarantees

- Re-runs delete and re-create only the `@testsItem` blocks immediately following `@testsList` and each `@testsSection` marker.
- Everything else in the file is preserved, including manual comments and code.
- Final numbering is computed after insertion using a two-pass approach, which makes `{l}` faithful to the ultimate positions.

# Edge Cases & Notes

- Leading whitespace before markers is tolerated (e.g., `#' @testsList`).
- Placeholder safety: a defensive cleanup removes any accidental temporary lines like `#' @testsItem {g=3}{s=2}{i=1}`.
- Quoted titles: only a fully quoted argument is unquoted in listings. Expressions are left untouched.
- Missing global marker: `#' @testsList` is auto-inserted at the top of the file.
- No sections: Sections are optional; global listing still works.
- No tests: If a file contains no `test_that()`, the function is a no-op except ensuring the global marker exists (if you want to avoid that, run in dry-run).

# Troubleshooting

- Leftover `{g}` or placeholders: ensure you’re using the latest version; the implementation rebuilds whole blocks and performs a global final cleanup.
- Encoding issues: use `encoding = "UTF-8"` (default). For legacy files, pass the correct encoding to both read & write.
- Custom numbering: verify your template string uses correct placeholders (`{g}`, `{s}`, `{i}`, `{l}`) or the aliases (`{local}`, `{line}`).

# Performance Tips

- For large trees, set `quiet = TRUE` and run from CI to avoid excessive console I/O.
- Dry-run first (`write = FALSE`) to inspect changes before writing.

# CI Integration

TBA

# Reproducible Example

In sections # `document_file()` and # `document()`