# boilerplate Package Architecture

# Overview

The `boilerplate` package is designed to manage and generate standardised text for scientific reports. It uses a unified database architecture with a hierarchical path system and template variable substitution.

# Core Architecture Components

## 1. Unified Database System

The package uses a unified database structure where all content types share a common interface:

    boilerplate_db (unified)
    ├── methods/
    │   ├── statistical/
    │   │   ├── regression/
    │   │   └── longitudinal/
    │   └── sampling/
    ├── measures/
    │   ├── psychological/
    │   └── demographic/
    ├── results/
    ├── discussion/
    ├── appendix/
    └── template/

### Key Design Principles

- Single Source of Truth: All content managed through one unified database
- Consistent Interface: Same functions work across all content types
- Hierarchical Organisation: Dot notation paths for nested content
- Format Agnostic: Supports both RDS (legacy) and JSON formats

## 2. Path System

Content is organised using dot notation paths:

    # Access nested content
    "methods.statistical.regression.linear"
    "measures.psychological.anxiety.gad7"
    "results.descriptive.demographics"

### Path Operations

- Navigation: `get_nested_folder()` traverses the hierarchy
- Modification: `modify_nested_entry()` adds/updates/removes entries
- Wildcards: `methods.statistical.*` matches all statistical methods
- Validation: `boilerplate_path_exists()` checks path validity

## 3. Template Variable System

Dynamic content substitution using `{{variable}}` syntax:

    # Template text
    "We analysed {{n}} participants using {{method}} regression."

    # Variables
    list(n = 100, method = "linear")

    # Result
    "We analysed 100 participants using linear regression."

### Variable Scoping

1. Global Variables: Available to all sections
2. Section Variables: Override globals for specific sections
3. Text Overrides: Direct text replacement

## 4. File Organisation

    R/
    ├── Core Functions
    │   ├── init-functions.R          # Database initialisation
    │   ├── import-export-functions.R # I/O operations
    │   └── utilities.R               # Core utilities
    │
    ├── User Interface
    │   ├── manage-measures.R         # Measure management
    │   ├── generate-text.R           # Text generation
    │   └── generate-measures.R       # Measure generation
    │
    ├── Data Operations
    │   ├── merge-databases.R         # Database merging
    │   ├── path-operations.R         # Path manipulation
    │   └── category-helpers.R        # Category extraction
    │
    ├── Format Support
    │   ├── json-support.R            # JSON operations
    │   ├── migration-utilities.R     # Format migration
    │   └── bibliography-support.R    # Citation handling
    │
    └── Batch Operations
        ├── boilerplate_batch_edit_functions.R
        └── boilerplate_standardise_measures.R

# Data Flow Architecture

## 1. Initialisation Flow

    boilerplate_init()
        ├── Creates directory structure
        ├── Initialises empty databases
        └── Saves as unified.json/rds

## 2. Import Flow

    External Data → boilerplate_import()
        ├── Detects format (JSON/RDS/CSV)
        ├── Validates structure
        ├── Merges with existing
        └── Updates unified database

## 3. Text Generation Flow

    boilerplate_generate_text()
        ├── Load unified database
        ├── Extract category paths
        ├── Apply template variables
        ├── Handle text overrides
        └── Return formatted text

# Key Design Patterns

## 1. Function Naming Convention

    # Public API
    boilerplate_<action>()         # Main functions
    boilerplate_<category>_<action>() # Category-specific

    # Internal functions
    <action>_<object>()            # No prefix for internals

## 2. Error Handling Strategy

- User confirmation prompts for destructive operations
- Informative error messages with suggestions
- Validation before operations
- Backup creation for critical operations

## 3. Extensibility Points

### Adding New Categories

1. Define default content in `default-databases.R`
2. Add accessor function following pattern
3. Update unified structure
4. Add tests

### Adding New Formats

1. Implement read/write functions in format-specific file
2. Add format detection in `detect_database_type()`
3. Update import/export functions
4. Ensure round-trip compatibility

## 4. Performance Considerations

- Lazy loading of large databases
- Efficient path traversal using recursive algorithms
- Minimal file I/O with in-memory operations
- Batch operations for multiple edits

# Database Schema

## Unified Database Structure

    list(
      methods = list(
        category1 = list(
          entry1 = list(
            text = "Method description with {{variables}}",
            reference = "@citation2023",
            keywords = c("keyword1", "keyword2")
          )
        )
      ),
      measures = list(
        category1 = list(
          measure1 = list(
            name = "measure_name",
            description = "Description",
            type = "continuous|categorical|ordinal|binary",
            ...
          )
        )
      ),
      template = list(
        global = list(var1 = "value1"),
        methods = list(var2 = "value2")
      )
    )

## Entry Types

### Text Entries (methods, results, discussion)

    list(
      text = "Content with {{variables}}",     # Required
      reference = "@citation",                 # Optional
      keywords = c("keyword1", "keyword2"),    # Optional
      large = "Extended version",              # Optional variant
      brief = "Short version"                  # Optional variant
    )

### Measure Entries

    list(
      name = "measure_id",              # Required
      description = "Description",      # Required
      type = "continuous",              # Required
      values = c(1, 2, 3),              # For categorical
      value_labels = c("Low", "Med", "High"),
      range = c(0, 100),                # For continuous
      unit = "points",
      reference = "@citation"
    )

# Testing Architecture

## Test Organisation

    tests/testthat/
    ├── test-init-functions.R      # Initialisation tests
    ├── test-import-export.R       # I/O operations
    ├── test-generate-text.R       # Text generation
    ├── test-path-operations.R     # Path system
    ├── test-json-support.R        # JSON functionality
    └── test-utilities.R           # Core utilities

## Testing Strategy

1. Unit Tests: Each function tested in isolation
2. Integration Tests: Full workflows tested
3. Format Tests: Round-trip compatibility
4. Edge Cases: Invalid inputs, empty databases

# Security Considerations

1. File Operations: Validated paths, no arbitrary file access
2. User Input: Sanitised for path traversal attacks
3. Confirmations: Required for destructive operations
4. Backups: Automatic for critical operations

# Future Architecture Considerations

## Planned Enhancements

1. Plugin System: Allow custom content types
2. Version Control: Built-in change tracking
3. Validation Rules: Custom validation per category
4. Performance: Caching for large databases

## Backwards Compatibility

- RDS format support maintained
- Automatic migration utilities
- Deprecation warnings for old functions
- Version detection in files