# ford-demo

# Introduction

In this vignette, we demonstrate FORD algorithm in A New Measure Of Dependence: Integrated R2, a forward stepwise variable selection algorithm based on the integrated \(R^2\) dependence measure. FORD is designed for variable ranking in both linear and nonlinear multivariate regression settings.

FORD closely follows the structure of FOCI A Simple Measure Of Conditional Dependence, but replaces the core dependence measure with irdc.

# Algorithm

Let \(Y\) be the response variable and \(\mathbf{X} = (X_1, \dots, X_p)\) the predictor variables. Given \(n\) i.i.d. samples of \((Y, \mathbf{X})\), FORD proceeds as follows:

1. Select \(j_1 = \arg\max_j \nu_n(Y, X_j)\)
   If \(\nu_n(Y, X_{j_1}) \leq 0\), return \(\hat{V} = \emptyset\)

2. Iteratively add the feature that gives the maximum increase in irdc: $$ j_{k+1} = \arg\max_{j \notin {j_1, \ldots, j_k}} \nu_n(Y, (X_{j_1}, \ldots, X_{j_k}, X_j)) $$

3. Stop when the irdc does not increase anymore: $$ \nu_n(Y, (X_{j_1}, \ldots, X_{j_k}, X_{j_{k+1}})) \leq \nu_n(Y, (X_{j_1}, \ldots, X_{j_k})) $$

If no such \(k\) exists, select all variables.

# Example 1 — Complex nonlinear function of first 4 features

Here, \(Y\) depends only on the first 4 features of \(X\) in a nonlinear way.

    set.seed(42)
    n <- 2000
    p <- 100
    X <- matrix(rnorm(n * p), ncol = p)
    colnames(X) <- paste0("X", seq_len(p))
    Y <- X[, 1] * X[, 2] + sin(X[, 1] * X[, 3]) + X[, 4]^2

## FOCI Result

    result_foci_1 <- foci(Y, X, numCores = 1)
    result_foci_1
    #> $selectedVar
    #>    index  names
    #>    <num> <char>
    #> 1:     4     X4
    #> 2:     1     X1
    #> 3:     2     X2
    #> 4:     3     X3
    #>
    #> $stepT
    #> [1] 0.3356423 0.4027284 0.6226254 0.7619649
    #>
    #> attr(,"class")
    #> [1] "foci"

## FORD Result

    result_ford_1 <- ford(Y, X, numCores = 1)
    result_ford_1
    #> $selectedVar
    #>    index  names
    #>    <num> <char>
    #> 1:     4     X4
    #> 2:     1     X1
    #> 3:     2     X2
    #> 4:     3     X3
    #>
    #> $step_nu
    #> [1] 0.3198165 0.4026348 0.6324854 0.7668089
    #>
    #> attr(,"class")
    #> [1] "ford"

# Example 2 — Selecting a fixed number of variables

We can force both FOCI and FORD to select a specific number of variables instead of using an automatic stopping rule.

## FOCI with 5 selected features

    result_foci_2 <- foci(Y, X, num_features = 5, stop = FALSE, numCores = 1)
    result_foci_2
    #> $selectedVar
    #>    index  names
    #>    <num> <char>
    #> 1:     4     X4
    #> 2:     1     X1
    #> 3:     2     X2
    #> 4:     3     X3
    #> 5:    66    X66
    #>
    #> $stepT
    #> [1] 0.3356423 0.4027284 0.6226254 0.7619649 0.6900384
    #>
    #> attr(,"class")
    #> [1] "foci"

## FORD with 5 selected features

    result_ford_2 <- ford(Y, X, num_features = 5, stop = FALSE, numCores = 1)
    result_ford_2
    #> $selectedVar
    #>    index  names
    #>    <num> <char>
    #> 1:     4     X4
    #> 2:     1     X1
    #> 3:     2     X2
    #> 4:     3     X3
    #> 5:    31    X31
    #>
    #> $step_nu
    #> [1] 0.3198165 0.4026348 0.6324854 0.7668089 0.6988827
    #>
    #> attr(,"class")
    #> [1] "ford"

# Conclusion

FORD provides an interpretable, irdc-based alternative to FOCI for variable selection in regression tasks. It offers a principled forward selection framework that can detect complex nonlinear relationships and be adapted for fixed-size feature subsets.

For further theoretical details, see our paper:
Azadkia and Roudaki (2025), A New Measure Of Dependence: Integrated R2