# Automatic Script Sourcing

The `autos` configuration automatically sources R scripts from specified directories, making custom functions immediately available without manual sourcing. This is perfect for project-specific utility functions and shared code libraries.

## Basic AUTOS Configuration

    default:
      autos:
        script_library: '/path/to/your/scripts'

Note: By default, auto-sourcing will overwrite any existing functions with the same name. This behavior can be controlled through the `overwrite` parameter in the underlying functions, though this is typically managed automatically by the system.

## Working Example: Single Script Library

Let’s create a practical example where Tidy McVerse has custom functions in a script library:

    library(envsetup)

    # Create temporary directory structure
    dir <- fs::file_temp()
    dir.create(dir)
    dir.create(file.path(dir, "/demo/DEV/username/project1/script_library"), recursive = TRUE)

    # Create a custom function
    file_conn <- file(file.path(dir, "/demo/DEV/username/project1/script_library/test.R"))
    writeLines(
    "test <- function(){print('Hello from auto-sourced function!')}", file_conn)
    close(file_conn)

    # Write the configuration
    config_path <- file.path(dir, "_envsetup.yml")
    file_conn <- file(config_path)
    writeLines(
      paste0(
    "default:
      autos:
        dev_script_library: '", dir,"/demo/DEV/username/project1/script_library'"
      ), file_conn)
    close(file_conn)

## Loading and Using Auto-Sourced Functions

    # Load configuration and apply it
    envsetup_config <- config::get(file = config_path)
    rprofile(envsetup_config)
    #> Assigned paths to R_GlobalEnv
    #> Sourcing file:  '/shared/R_Temp/nmasel/RtmpKbVJua/file1962b14eacebaa/demo/DEV/username/project1/script_library/test.R'
    #>
    #>  The following objects are added to .GlobalEnv:
    #>
    #>     'test'

The auto-sourced functions are now available:

    # See what functions are available
    objects()
    #> [1] "auto_stored_envsetup_config" "config_path"
    #> [3] "dir"                         "envsetup_config"
    #> [5] "file_conn"                   "test"

    # Use the function directly (no manual sourcing needed!)
    test()
    #> [1] "Hello from auto-sourced function!"

## Multiple Script Libraries

Real projects often have multiple script libraries for different purposes:

    # Create production script library
    dir.create(file.path(dir, "/demo/PROD/project1/script_library"), recursive = TRUE)

    # Add production functions
    file_conn <- file(file.path(dir, "/demo/PROD/project1/script_library/test2.R"))
    writeLines(
    "test2 <- function(){print('Hello from production function!')}", file_conn)
    close(file_conn)

    # Update configuration with multiple libraries
    file_conn <- file(config_path)
    writeLines(
      paste0(
    "default:
      autos:
        dev_script_library: '", dir,"/demo/DEV/username/project1/script_library'
        prod_script_library: '", dir,"/demo/PROD/project1/script_library'"
      ), file_conn)
    close(file_conn)

    # Reload configuration
    envsetup_config <- config::get(file = config_path)
    rprofile(envsetup_config)
    #> Assigned paths to R_GlobalEnv
    #> Sourcing file:  '/shared/R_Temp/nmasel/RtmpKbVJua/file1962b14eacebaa/demo/DEV/username/project1/script_library/test.R'
    #>
    #>  The following objects are added to .GlobalEnv:
    #>
    #>     'test'
    #>
    #> Sourcing file:  '/shared/R_Temp/nmasel/RtmpKbVJua/file1962b14eacebaa/demo/PROD/project1/script_library/test2.R'
    #>
    #>  The following objects are added to .GlobalEnv:
    #>
    #>     'test2'

## Using Multiple Libraries

    # Check search path - now includes both libraries
    # Functions from both libraries are available
    objects()
    #> [1] "auto_stored_envsetup_config" "config_path"
    #> [3] "dir"                         "envsetup_config"
    #> [5] "file_conn"                   "test"
    #> [7] "test2"

    # Use functions from both libraries
    test()   # From dev library
    #> [1] "Hello from auto-sourced function!"
    test2()  # From prod library
    #> [1] "Hello from production function!"

## Understanding Function Conflicts and the Overwrite Parameter

When auto-sourcing functions, you might encounter situations where function names conflict with existing objects in your environment. The `overwrite` parameter controls how these conflicts are handled.

### Quick Example of Function Conflicts

    # Create a function that might conflict
    summary_stats <- function(data) {
      print("Original summary function")
    }

    # Create a script with the same function name
    conflict_dir <- file.path(dir, "conflict_demo")
    dir.create(conflict_dir)

    file_conn <- file(file.path(conflict_dir, "stats.R"))
    writeLines(
    "summary_stats <- function(data) {
      print('Updated summary function from the new conflict_demo script')
    }", file_conn)
    close(file_conn)

    # Add to configuration
    file_conn <- file(config_path)
    writeLines(
      paste0(
    "default:
      autos:
        dev_script_library: '", dir,"/demo/DEV/username/project1/script_library'
        prod_script_library: '", dir,"/demo/PROD/project1/script_library'
        conflict_demo: '", conflict_dir, "'"
      ), file_conn)
    close(file_conn)

    # When we reload, the auto-sourced version will overwrite the original
    envsetup_config <- config::get(file = config_path)
    rprofile(envsetup_config)
    #> Assigned paths to R_GlobalEnv
    #> Sourcing file:  '/shared/R_Temp/nmasel/RtmpKbVJua/file1962b14eacebaa/demo/DEV/username/project1/script_library/test.R'
    #>
    #>  The following objects are added to .GlobalEnv:
    #>
    #>     'test'
    #>
    #> Sourcing file:  '/shared/R_Temp/nmasel/RtmpKbVJua/file1962b14eacebaa/demo/PROD/project1/script_library/test2.R'
    #>
    #>  The following objects are added to .GlobalEnv:
    #>
    #>     'test2'
    #>
    #> Sourcing file:  '/shared/R_Temp/nmasel/RtmpKbVJua/file1962b14eacebaa/conflict_demo/stats.R'
    #>
    #>  The following objects are added to .GlobalEnv:
    #>
    #>     'summary_stats'
    #>
    #>  The following objects were overwritten in .GlobalEnv:
    #>
    #>     'summary_stats'

    # Test which version we have now
    summary_stats()
    #> [1] "Updated summary function from the new conflict_demo script"

The output shows detailed information about what was overwritten, helping you track conflicts.

## Environment-Specific Auto-Sourcing

You might want different script libraries for different environments. For example, exclude development functions when running in production:

    # Configuration that blanks out dev scripts in production
    file_conn <- file(config_path)
    writeLines(
      paste0(
    "default:
      autos:
        dev_script_library: '", dir,"/demo/DEV/username/project1/script_library'
        prod_script_library: '", dir,"/demo/PROD/project1/script_library'

    prod:
      autos:
        dev_script_library: NULL"  # NULL disables this library
      ), file_conn)
    close(file_conn)

    # Load production configuration
    envsetup_config <- config::get(file = config_path, config = "prod")
    rprofile(envsetup_config)
    #> Assigned paths to R_GlobalEnv
    #> Sourcing file:  '/shared/R_Temp/nmasel/RtmpKbVJua/file1962b14eacebaa/demo/PROD/project1/script_library/test2.R'
    #>
    #>  The following objects are added to .GlobalEnv:
    #>
    #>     'test2'

So we can see now that only production functions are available:

    # Functions from production only
    objects()
    #> [1] "auto_stored_envsetup_config" "config_path"
    #> [3] "conflict_dir"                "dir"
    #> [5] "envsetup_config"             "file_conn"
    #> [7] "test2"

    # Use functions from production only
    test2()  # From prod library
    #> [1] "Hello from production function!"

## How Auto-Sourcing Works

When you call `rprofile()` with autos configuration:

1. Script Discovery: Finds all `.R` files in specified directories
2. Conflict Detection: Compares new functions with existing global environment objects
3. Automatic Sourcing: Sources each script into its environment
4. Conflict Resolution: Based on the `overwrite` parameter: 5 - `overwrite = TRUE` (default): Replaces existing functions and reports what was overwritten
   - `overwrite = FALSE`: Preserves existing functions and reports what was skipped . Metadata Tracking: Records which script each function came from for debugging
5. Function Availability: Functions become directly accessible

### Technical Details of Conflict Handling

The auto-sourcing system uses a sophisticated approach to handle conflicts:

- Temporary Environment: Each script is first sourced into a temporary environment
- Object Comparison: New objects are compared against the global environment
- Selective Assignment: Only specified objects are moved to the global environment
- Metadata Recording: Each function’s source script is recorded in `object_metadata`
- Detailed Reporting: Users receive clear feedback about what was added, skipped, or overwritten
- Cleanup Integration: Metadata enables precise cleanup when using `detach_autos()`

The `record_function_metadata()` function creates a comprehensive audit trail by maintaining a data frame with:

- `object_name`: The name of each sourced function

- `script`: The full path to the source script

- This is automatically updated when functions are overwritten by newer versions

## Benefits of Auto-Sourcing

1. No Manual Sourcing: Functions are automatically available
2. Organized Libraries: Separate environments for different script collections
3. Environment Isolation: Functions don’t interfere with each other
4. Dynamic Loading: Easy to add/remove script libraries
5. Team Collaboration: Shared function libraries across team members
6. Comprehensive Tracking: Metadata system tracks function sources for debugging
7. Intelligent Cleanup: Precise removal of auto-sourced functions via metadata

## Common Use Cases

### Project Utilities

    autos:
      project_utils: '/project/utilities'
      data_processing: '/project/data_functions'
      plotting_functions: '/project/viz_functions'

### Environment-Specific Functions

    default:
      autos:
        dev_helpers: '/dev/helper_functions'
        shared_utils: '/shared/utilities'

    prod:
      autos:
        shared_utils: '/shared/utilities'
        # dev_helpers excluded in production

### Team Libraries

    autos:
      team_functions: '/shared/team_library'
      personal_utils: '~/my_r_functions'
      project_specific: './project_functions'

## Managing Function Conflicts with the Overwrite Parameter

The `overwrite` parameter controls how auto-sourcing handles situations where functions with the same name already exist in your global environment. Understanding this parameter is crucial for managing function conflicts effectively.

### Default Behavior: Overwrite = TRUE

By default, auto-sourcing will overwrite existing functions:

    # Create a function in global environment
    my_function <- function() {
      print("Original function from global environment")
    }

    # Check it works
    my_function()
    #> [1] "Original function from global environment"

    # Create a script with the same function name
    dir <- fs::file_temp()
    dir.create(dir)
    script_dir <- file.path(dir, "scripts")
    dir.create(script_dir)

    file_conn <- file(file.path(script_dir, "my_function.R"))
    writeLines(
    "my_function <- function() {
      print('Updated function from auto-sourced script')
    }", file_conn)
    close(file_conn)

    # Configuration with default overwrite = TRUE
    config_path <- file.path(dir, "_envsetup.yml")
    file_conn <- file(config_path)
    writeLines(
      paste0(
    "default:
      autos:
        my_scripts: '", script_dir, "'"
      ), file_conn)
    close(file_conn)

    # Load configuration - this will overwrite the existing function
    envsetup_config <- config::get(file = config_path)
    rprofile(envsetup_config)
    #> Assigned paths to R_GlobalEnv
    #> Sourcing file:  '/shared/R_Temp/nmasel/RtmpKbVJua/file1962b13478deeb/scripts/my_function.R'
    #>
    #>  The following objects are added to .GlobalEnv:
    #>
    #>     'my_function'
    #>
    #>  The following objects were overwritten in .GlobalEnv:
    #>
    #>     'my_function'

    # The function has been overwritten
    my_function()
    #> [1] "Updated function from auto-sourced script"

### Conservative Behavior: Overwrite = FALSE

When `overwrite = FALSE`, existing functions are preserved:

    # clean up previous runs, removing all previously attached autos
    detach_autos()

    # Create a function in global environment
    my_function <- function() {
      print("Original function from global environment")
    }

    # Check it works
    my_function()
    #> [1] "Original function from global environment"

    # Create a script with the same function name
    dir <- fs::file_temp()
    dir.create(dir)
    script_dir <- file.path(dir, "scripts")
    dir.create(script_dir)

    file_conn <- file(file.path(script_dir, "my_function.R"))
    writeLines(
    "my_function <- function() {
      print('Updated function from auto-sourced script')
    }", file_conn)
    close(file_conn)

    # Configuration with default overwrite = FALSE
    config_path <- file.path(dir, "_envsetup.yml")
    file_conn <- file(config_path)
    writeLines(
      paste0(
    "default:
      autos:
        my_scripts: '", script_dir, "'"
      ), file_conn)
    close(file_conn)

    envsetup_config <- config::get(file = config_path)
    rprofile(envsetup_config, overwrite = FALSE)
    #> Assigned paths to R_GlobalEnv
    #> Sourcing file:  '/shared/R_Temp/nmasel/RtmpKbVJua/file1962b124e27861/scripts/my_function.R'
    #>
    #>  The following objects were not added to .GlobalEnv as they already exist:
    #>
    #>     'my_function'

    my_function()
    #> [1] "Original function from global environment"

### Understanding Conflict Detection

The auto-sourcing system provides detailed feedback about what happens during sourcing:

    # Create multiple functions to demonstrate conflict detection
    existing_func1 <- function() "I exist in global"
    existing_func2 <- function() "I also exist in global"

    # Create script with mix of new and conflicting functions
    file_conn <- file(file.path(script_dir, "mixed_functions.R"))
    writeLines(
    "# This will conflict with existing function
    existing_func1 <- function() {
      'Updated from script'
    }

    # This is a new function
    new_func <- function() {
      'Brand new function'
    }

    # This will also conflict
    existing_func2 <- function() {
      'Also updated from script'
    }", file_conn)
    close(file_conn)

    # Update configuration
    file_conn <- file(config_path)
    writeLines(
      paste0(
    "default:
      autos:
        my_scripts: '", script_dir, "'"
      ), file_conn)
    close(file_conn)

    # Reload - watch the detailed output
    envsetup_config <- config::get(file = config_path)
    rprofile(envsetup_config)
    #> Assigned paths to R_GlobalEnv
    #> Sourcing file:  '/shared/R_Temp/nmasel/RtmpKbVJua/file1962b124e27861/scripts/mixed_functions.R'
    #>
    #>  The following objects are added to .GlobalEnv:
    #>
    #>     'existing_func1', 'existing_func2', 'new_func'
    #>
    #>  The following objects were overwritten in .GlobalEnv:
    #>
    #>     'existing_func1', 'existing_func2'
    #>
    #> Sourcing file:  '/shared/R_Temp/nmasel/RtmpKbVJua/file1962b124e27861/scripts/my_function.R'
    #>
    #>  The following objects are added to .GlobalEnv:
    #>
    #>     'my_function'
    #>
    #>  The following objects were overwritten in .GlobalEnv:
    #>
    #>     'my_function'

## Function Metadata Tracking

The auto-sourcing system includes sophisticated metadata tracking that records detailed information about every function that gets sourced. This tracking system is invaluable for debugging, auditing, and understanding your function ecosystem.

### How Metadata Tracking Works

Every time a function is sourced through the autos system, the `record_function_metadata()` function captures:

- Object Name: The name of the function or object
- Source Script: The full path to the script file that contains the function

This information is stored in a special `object_metadata` data frame within the `envsetup_environment`.

### Accessing Function Metadata

    # After sourcing functions, you can access the metadata
    # Note: This example shows the concept - actual access depends on envsetup internals

    # Create some functions to demonstrate metadata tracking
    metadata_demo_dir <- file.path(dir, "metadata_demo")
    dir.create(metadata_demo_dir)

    # Create multiple scripts with different functions
    file_conn <- file(file.path(metadata_demo_dir, "data_functions.R"))
    writeLines(
    "load_data <- function(file) {
      paste('Loading data from:', file)
    }

    clean_data <- function(data) {
      paste('Cleaning data with', nrow(data), 'rows')
    }", file_conn)
    close(file_conn)

    file_conn <- file(file.path(metadata_demo_dir, "plot_functions.R"))
    writeLines(
    "create_plot <- function(data) {
      paste('Creating plot for', ncol(data), 'variables')
    }

    save_plot <- function(plot, filename) {
      paste('Saving plot to:', filename)
    }", file_conn)
    close(file_conn)

    # Update configuration to include metadata demo
    file_conn <- file(config_path)
    writeLines(
      paste0(
    "default:
      autos:
        metadata_demo: '", metadata_demo_dir, "'"
      ), file_conn)
    close(file_conn)

    # Source the functions
    envsetup_config <- config::get(file = config_path)
    rprofile(envsetup_config)
    #> Assigned paths to R_GlobalEnv
    #> Sourcing file:  '/shared/R_Temp/nmasel/RtmpKbVJua/file1962b124e27861/metadata_demo/data_functions.R'
    #>
    #>  The following objects are added to .GlobalEnv:
    #>
    #>     'clean_data', 'load_data'
    #>
    #> Sourcing file:  '/shared/R_Temp/nmasel/RtmpKbVJua/file1962b124e27861/metadata_demo/plot_functions.R'
    #>
    #>  The following objects are added to .GlobalEnv:
    #>
    #>     'create_plot', 'save_plot'

    # The system now tracks which script each function came from
    cat("Functions sourced with metadata tracking:")
    #> Functions sourced with metadata tracking:
    knitr::kable(envsetup_environment$object_metadata)

object_name | script
--- | ---
clean_data | /shared/R_Temp/nmasel/RtmpKbVJua/file1962b124e27861/metadata_demo/data_functions.R
create_plot | /shared/R_Temp/nmasel/RtmpKbVJua/file1962b124e27861/metadata_demo/plot_functions.R
load_data | /shared/R_Temp/nmasel/RtmpKbVJua/file1962b124e27861/metadata_demo/data_functions.R
save_plot | /shared/R_Temp/nmasel/RtmpKbVJua/file1962b124e27861/metadata_demo/plot_functions.R

### Benefits of Metadata Tracking

#### 1. Debugging Function Issues

When a function isn’t working as expected, metadata helps you quickly identifyw hich script file contains the function

#### 2. Audit Trail

Metadata provides a complete audit trail of your function ecosystem.

### Metadata and the detach_autos() Function

The metadata tracking system integrates closely with cleanup operations:

1. Identify all auto-sourced functions
2. Remove them from the global environment
3. Clean up the metadata records

## Best Practices

Even though your functions are not a part of a package, you should follow best practices to ensure your functions work as expected.

1. Use Clear Names: Library names should indicate their purpose
2. Monitor Conflicts: Regularly check for and resolve function name conflicts
3. Document Functions: Include roxygen2 comments in your functions
4. Test Functions: Ensure auto-sourced functions work correctly
5. Package Prefix: Use package prefix when writing your functions, for example, `dplyr::filter`