# Getting Started with evoFE

## What is evoFE?

evoFE (Evolutionary Feature Engineering) uses a genetic algorithm to automatically discover useful feature transformations for tabular data. Instead of manually crafting interaction terms, ratios, or binning strategies, you let evolution explore the space of possible transformations and keep the ones that improve predictive performance.

The result is an evo_recipe — a reusable transformation pipeline that can be applied to new data at prediction time.

### How it works

1. Initialisation — A population of individuals is created. Each individual is a “recipe” containing a set of feature transformations (genes).
2. Evaluation — Every individual is scored via cross-validated or split model performance (LightGBM or XGBoost).
3. Selection — The top 50 % survive to breed.
4. Breeding — Survivors are combined (crossover) and randomly altered (mutation) to produce the next generation.
5. Repeat — The cycle continues until the fitness plateaus or the generation budget is exhausted.

## Installation

    # Install from local source
    devtools::install_local("path/to/evoFE")

    # Or install directly from a git repo
    # devtools::install_github("tanopereira/evoFE")

## Quick Start — Binary Classification

Let’s classify whether a car has an automatic or manual transmission using the `mtcars` dataset.

    library(evoFE)

    data(mtcars)
    df <- mtcars
    df$am <- as.integer(df$am)   # target: 0 = automatic, 1 = manual

    set.seed(42)
    res <- evolve_features(
      data      = df,
      target_col = "am",
      task       = "classification",
      evaluator  = "xgboost",
      generations = 5,
      pop_size    = 8,
      cv_folds    = 3,
      early_stopping_rounds = 3,
      verbose     = TRUE
    )
    #> Starting Evolutionary Feature Engineering...
    #>   Task: classification
    #>   Evaluator: xgboost
    #>   Generations: 5, Population Size: 8, CV Folds: 3
    #>   Original Numeric columns: mpg, cyl, disp, hp, drat, wt, qsec, vs, gear, carb
    #>   Original Categorical columns:
    #>
    #> [Gen 0] Initialized Population:
    #>   Individual 1: [Original features only]
    #>   Individual 2: [mst_score(mpg, gear, cyl, hp, vs, wt), mst_score(gear, drat, wt, hp, cyl)]
    #>   Individual 3: [genie(gear, carb, disp, hp), subtract(hp, cyl)]
    #>   Individual 4: [lumbermark(carb, mpg, cyl, wt), log_binning9(carb)]
    #>   Individual 5: [add(wt, cyl, drat, hp, cyl), log_binning_cat4(cyl)]
    #>   Individual 6: [log_binning_cat2(qsec), quantile_binning4(carb)]
    #>   Individual 7: [multiply(cyl, hp, disp), log_binning3(drat)]
    #>   Individual 8: [quantile_binning_cat4(wt), truncated_svd1(mpg, carb, qsec, hp, drat, cyl), truncated_svd2(mpg, carb, qsec, hp, drat, cyl), truncated_svd3(mpg, carb, qsec, hp, drat, cyl)]
    #>
    #> --- Generation 1 / 5 ---
    #>   Tested Individual 1 (New Best!) -> Fitness: 0.6862
    #>   Tested Individual 2 -> Fitness: 0.6836
    #>   Tested Individual 3 -> Fitness: 0.6862
    #>   Tested Individual 4 -> Fitness: 0.6857
    #>   Tested Individual 5 -> Fitness: 0.6862
    #>   Tested Individual 6 -> Fitness: 0.6862
    #>   Tested Individual 7 -> Fitness: 0.6862
    #>   Tested Individual 8 -> Fitness: 0.6862
    #>   Gen 1 Best Fitness: 0.6862
    #>   Gen 1 Best Recipe: [Original features only]
    #>
    #> --- Generation 2 / 5 (Current Best Fitness: 0.6862) ---
    #>   Tested Individual 2 -> Fitness: 0.6862
    #>   Tested Individual 3 -> Fitness: 0.6862
    #>   Tested Individual 4 -> Fitness: 0.6862
    #>   Tested Individual 5 -> Fitness: 0.6862
    #>   Tested Individual 6 -> Fitness: 0.6787
    #>   Tested Individual 7 -> Fitness: 0.6862
    #>   Tested Individual 8 -> Fitness: 0.6862
    #>   Gen 2 Best Fitness: 0.6862
    #>   Gen 2 Best Recipe: [Original features only]
    #>
    #> --- Generation 3 / 5 (Current Best Fitness: 0.6862) ---
    #>   Tested Individual 2 -> Fitness: 0.6862
    #>   Tested Individual 3 -> Fitness: 0.6862
    #>   Tested Individual 4 (New Best!) -> Fitness: 0.6896
    #>   Tested Individual 5 -> Fitness: 0.6862
    #>   Tested Individual 6 (New Best!) -> Fitness: 0.7051
    #>   Tested Individual 7 -> Fitness: 0.6862 (cached)
    #>   Tested Individual 8 -> Fitness: 0.6862
    #>   Tested Individual 9 -> Fitness: 0.6875
    #>   Tested Individual 10 -> Fitness: 0.6862
    #>   Tested Individual 11 -> Fitness: 0.6862
    #>   Tested Individual 12 -> Fitness: 0.6862
    #>   Gen 3 Best Fitness: 0.7051
    #>   Gen 3 Best Recipe: [umap1(disp, wt), umap2(disp, wt)]
    #>
    #> --- Generation 4 / 5 (Current Best Fitness: 0.7051) ---
    #>   Tested Individual 2 -> Fitness: 0.6908
    #>   Tested Individual 3 -> Fitness: 0.6896
    #>   Tested Individual 4 -> Fitness: 0.6862
    #>   Tested Individual 5 -> Fitness: 0.6862
    #>   Tested Individual 6 -> Fitness: 0.6831
    #>   Tested Individual 7 (New Best!) -> Fitness: 0.7073
    #>   Tested Individual 8 -> Fitness: 0.6816
    #>   Gen 4 Best Fitness: 0.7073
    #>   Gen 4 Best Recipe: [multiply(disp, gear, wt, wt, wt), pca1(carb, vs, hp), pca2(carb, vs, hp), pca3(carb, vs, hp), umap1(disp, wt), umap2(disp, wt)]
    #>
    #> --- Generation 5 / 5 (Current Best Fitness: 0.7073) ---
    #>   Tested Individual 2 -> Fitness: 0.6910
    #>   Tested Individual 3 -> Fitness: 0.6607
    #>   Tested Individual 4 -> Fitness: 0.6819
    #>   Tested Individual 5 (New Best!) -> Fitness: 0.7179
    #>   Tested Individual 6 -> Fitness: 0.7147
    #>   Tested Individual 7 -> Fitness: 0.6814
    #>   Tested Individual 8 -> Fitness: 0.7073
    #>   Gen 5 Best Fitness: 0.7179
    #>   Gen 5 Best Recipe: [multiply(disp, gear, wt, wt, wt), pca1(carb, vs, hp), pca2(carb, vs, hp), umap2(disp, wt)]
    #>
    #> Evolution Complete. Best Fitness: 0.7179
    #> Best recipe: [multiply(disp, gear, wt, wt, wt), pca1(carb, vs, hp), pca2(carb, vs, hp), umap2(disp, wt)]
    #> Generated columns: ((disp*gear*wt*wt*wt)), PCA1(car_vs_hp), PCA2(car_vs_hp), UMAP2(dis_wt)
    #> Training final model on full dataset...

The returned `evo_recipe` object contains the best individual (feature recipe), the fitted model, and the evolution history.

    # View the winning recipe
    cat("Best recipe:", individual_to_recipe_string(res$best_individual), "\n")
    #> Best recipe: [multiply(disp, gear, wt, wt, wt), pca1(carb, vs, hp), pca2(carb, vs, hp), umap2(disp, wt)]
    cat("Fitness:    ", res$best_individual$fitness, "\n")
    #> Fitness:     0.7178567

### Applying the recipe to new data

`predict()` applies the evolved transformations to new data and returns the engineered feature matrix:

    engineered <- predict(res, df[1:5, ])
    head(engineered)
    #>      mpg   cyl  disp    hp  drat    wt  qsec    vs  gear  carb
    #>    <num> <num> <num> <num> <num> <num> <num> <num> <num> <num>
    #> 1:  21.0     6   160   110  3.90 2.620 16.46     0     4     4
    #> 2:  21.0     6   160   110  3.90 2.875 17.02     0     4     4
    #> 3:  22.8     4   108    93  3.85 2.320 18.61     1     4     1
    #> 4:  21.4     6   258   110  3.08 3.215 19.44     1     3     1
    #> 5:  18.7     8   360   175  3.15 3.440 17.02     0     3     2
    #>    ((disp*gear*wt*wt*wt)) PCA1(car_vs_hp) PCA2(car_vs_hp) UMAP2(dis_wt)
    #>                     <num>           <num>           <num>         <num>
    #> 1:              11510.226      -0.5758895      0.17083974      4.737757
    #> 2:              15208.750      -0.5758895      0.17083974      4.526803
    #> 3:               5394.457       1.7331290     -0.02830549      7.232712
    #> 4:              25720.766       1.5826875     -0.04046764     -6.888471
    #> 5:              43964.191      -0.4499549      0.95866969     -5.179038

`predict_model()` goes one step further — it applies the transformations and runs the trained model to produce predictions:

    preds <- predict_model(res, df[1:5, ])
    preds
    #> [1] 0.96213979 0.96213979 0.91886902 0.03086278 0.05843740

## Regression

Predict petal length from the iris dataset:

    data(iris)

    set.seed(123)
    res_reg <- evolve_features(
      data       = iris[, c("Sepal.Length", "Sepal.Width", "Petal.Width", "Petal.Length")],
      target_col = "Petal.Length",
      task       = "regression",
      evaluator  = "xgboost",
      generations = 5,
      pop_size    = 8,
      cv_folds    = 3,
      early_stopping_rounds = 3,
      verbose     = TRUE
    )
    #> Starting Evolutionary Feature Engineering...
    #>   Task: regression
    #>   Evaluator: xgboost
    #>   Generations: 5, Population Size: 8, CV Folds: 3
    #>   Original Numeric columns: Sepal.Length, Sepal.Width, Petal.Width
    #>   Original Categorical columns:
    #>
    #> [Gen 0] Initialized Population:
    #>   Individual 1: [Original features only]
    #>   Individual 2: [reciprocal(Sepal.Width), genie(Sepal.Width, Sepal.Length)]
    #>   Individual 3: [log_binning6(Sepal.Length), random_projection(Sepal.Length, Sepal.Width)]
    #>   Individual 4: [normalized_difference(Sepal.Length, Sepal.Length), reciprocal(Sepal.Width)]
    #>   Individual 5: [divide(Sepal.Width, Sepal.Length), add(Sepal.Length, Petal.Width, Petal.Width)]
    #>   Individual 6: [quantile_binning10(Petal.Width), truncated_svd1(Sepal.Length, Petal.Width), truncated_svd2(Sepal.Length, Petal.Width), truncated_svd3(Sepal.Length, Petal.Width)]
    #>   Individual 7: [quantile_binning_cat6(Petal.Width), quantile_binning7(Petal.Width)]
    #>   Individual 8: [random_projection(Sepal.Width, Sepal.Length), sqrt(Sepal.Length)]
    #>
    #> --- Generation 1 / 5 ---
    #>   Tested Individual 1 (New Best!) -> Fitness: -0.2939
    #>   Tested Individual 2 (New Best!) -> Fitness: -0.2925
    #>   Tested Individual 3 -> Fitness: -0.3072
    #>   Tested Individual 4 -> Fitness: -0.2939
    #>   Tested Individual 5 (New Best!) -> Fitness: -0.2701
    #>   Tested Individual 6 -> Fitness: -0.2983
    #>   Tested Individual 7 -> Fitness: -0.2939
    #>   Tested Individual 8 -> Fitness: -0.3072
    #>   Gen 1 Best Fitness: -0.2701
    #>   Gen 1 Best Recipe: [divide(Sepal.Width, Sepal.Length), add(Sepal.Length, Petal.Width, Petal.Width)]
    #>
    #> --- Generation 2 / 5 (Current Best Fitness: -0.2701) ---
    #>   Tested Individual 2 -> Fitness: -0.2734
    #>   Tested Individual 3 -> Fitness: -0.2701
    #>   Tested Individual 4 -> Fitness: -0.2736
    #>   Tested Individual 5 -> Fitness: -0.2925
    #>   Tested Individual 6 -> Fitness: -0.2701
    #>   Tested Individual 7 -> Fitness: -0.2923
    #>   Tested Individual 8 -> Fitness: -0.3541
    #>   Gen 2 Best Fitness: -0.2701
    #>   Gen 2 Best Recipe: [divide(Sepal.Width, Sepal.Length), add(Sepal.Length, Petal.Width, Petal.Width)]
    #>
    #> --- Generation 3 / 5 (Current Best Fitness: -0.2701) ---
    #>   Tested Individual 2 -> Fitness: -0.2701
    #>   Tested Individual 3 -> Fitness: -0.2701
    #>   Tested Individual 4 -> Fitness: -0.2701 (cached)
    #>   Tested Individual 5 -> Fitness: -0.2734
    #>   Tested Individual 6 -> Fitness: -0.2701
    #>   Tested Individual 7 -> Fitness: -0.2788
    #>   Tested Individual 8 -> Fitness: -0.2939
    #>   Tested Individual 9 -> Fitness: -0.2734
    #>   Tested Individual 10 -> Fitness: -0.2701
    #>   Tested Individual 11 -> Fitness: -0.2701
    #>   Tested Individual 12 -> Fitness: -0.2770
    #>   Gen 3 Best Fitness: -0.2701
    #>   Gen 3 Best Recipe: [divide(Sepal.Width, Sepal.Length), add(Sepal.Length, Petal.Width, Petal.Width)]
    #>
    #> --- Generation 4 / 5 (Current Best Fitness: -0.2701) ---
    #>   Tested Individual 2 -> Fitness: -0.2736
    #>   Tested Individual 3 -> Fitness: -0.2822
    #>   Tested Individual 4 -> Fitness: -0.2822
    #>   Tested Individual 5 (New Best!) -> Fitness: -0.2680
    #>   Tested Individual 6 -> Fitness: -0.2847
    #>   Tested Individual 7 -> Fitness: -0.2701
    #>   Tested Individual 8 -> Fitness: -0.2996
    #>   Tested Individual 9 -> Fitness: -0.2685
    #>   Tested Individual 10 -> Fitness: -0.2701
    #>   Tested Individual 11 -> Fitness: -0.2701
    #>   Tested Individual 12 -> Fitness: -0.2772
    #>   Tested Individual 14 -> Fitness: -0.2701
    #>   Tested Individual 16 -> Fitness: -0.2701
    #>   Tested Individual 17 -> Fitness: -0.2731
    #>   Tested Individual 18 -> Fitness: -0.2701
    #>   Gen 4 Best Fitness: -0.2680
    #>   Gen 4 Best Recipe: [divide(Sepal.Width, Sepal.Length), add(Sepal.Length, Petal.Width, Petal.Width), add(((Sepal.Length+Petal.Width+Petal.Width)), ((Sepal.Length+Petal.Width+Petal.Width)), ((Sepal.Length+Petal.Width+Petal.Width)), ((Sepal.Length+Petal.Width+Petal.Width)), ((Sepal.Length+Petal.Width+Petal.Width))), reciprocal(Sepal.Width), genie(Sepal.Width, Sepal.Length, Petal.Width), log_binning8(rec(Sepal.Width)), divide(((Sepal.Length+Petal.Width+Petal.Width)), Petal.Width)]
    #>
    #> --- Generation 5 / 5 (Current Best Fitness: -0.2680) ---
    #>   Tested Individual 2 -> Fitness: -0.2939
    #>   Tested Individual 3 -> Fitness: -0.3183
    #>   Tested Individual 4 -> Fitness: -0.2878
    #>   Tested Individual 5 -> Fitness: -0.2701
    #>   Tested Individual 6 -> Fitness: -0.3021
    #>   Tested Individual 7 -> Fitness: -0.2760
    #>   Tested Individual 8 -> Fitness: -0.2701
    #>   Gen 5 Best Fitness: -0.2680
    #>   Gen 5 Best Recipe: [divide(Sepal.Width, Sepal.Length), add(Sepal.Length, Petal.Width, Petal.Width), add(((Sepal.Length+Petal.Width+Petal.Width)), ((Sepal.Length+Petal.Width+Petal.Width)), ((Sepal.Length+Petal.Width+Petal.Width)), ((Sepal.Length+Petal.Width+Petal.Width)), ((Sepal.Length+Petal.Width+Petal.Width))), reciprocal(Sepal.Width), genie(Sepal.Width, Sepal.Length, Petal.Width), log_binning8(rec(Sepal.Width)), divide(((Sepal.Length+Petal.Width+Petal.Width)), Petal.Width)]
    #>
    #> Evolution Complete. Best Fitness: -0.2680
    #> Best recipe: [divide(Sepal.Width, Sepal.Length), add(Sepal.Length, Petal.Width, Petal.Width), add(((Sepal.Length+Petal.Width+Petal.Width)), ((Sepal.Length+Petal.Width+Petal.Width)), ((Sepal.Length+Petal.Width+Petal.Width)), ((Sepal.Length+Petal.Width+Petal.Width)), ((Sepal.Length+Petal.Width+Petal.Width))), reciprocal(Sepal.Width), genie(Sepal.Width, Sepal.Length, Petal.Width), log_binning8(rec(Sepal.Width)), divide(((Sepal.Length+Petal.Width+Petal.Width)), Petal.Width)]
    #> Generated columns: ((Sepal.Width/Sepal.Length)), ((Sepal.Length+Petal.Width+Petal.Width)), ((((Sepal.Length+Petal.Width+Petal.Width))+((Sepal.Length+Petal.Width+Petal.Width))+((Sepal.Length+Petal.Width+Petal.Width))+((Sepal.Length+Petal.Width+Petal.Width))+((Sepal.Length+Petal.Width+Petal.Width)))), rec(Sepal.Width), Genie2(Sep_Sep_Pet), logbin8(rec(Sepal.Width)), ((((Sepal.Length+Petal.Width+Petal.Width))/Petal.Width))
    #> Training final model on full dataset...

    cat("Best recipe:", individual_to_recipe_string(res_reg$best_individual), "\n")
    #> Best recipe: [divide(Sepal.Width, Sepal.Length), add(Sepal.Length, Petal.Width, Petal.Width), add(((Sepal.Length+Petal.Width+Petal.Width)), ((Sepal.Length+Petal.Width+Petal.Width)), ((Sepal.Length+Petal.Width+Petal.Width)), ((Sepal.Length+Petal.Width+Petal.Width)), ((Sepal.Length+Petal.Width+Petal.Width))), reciprocal(Sepal.Width), genie(Sepal.Width, Sepal.Length, Petal.Width), divide(((Sepal.Length+Petal.Width+Petal.Width)), Petal.Width)]
    cat("Fitness (neg RMSE):", res_reg$best_individual$fitness, "\n")
    #> Fitness (neg RMSE): -0.2679901

    preds_reg <- predict_model(res_reg, iris[1:10, ])
    # Compare predictions to actuals
    data.frame(
      actual    = iris$Petal.Length[1:10],
      predicted = round(preds_reg, 2)
    )
    #>    actual predicted
    #> 1     1.4      1.48
    #> 2     1.4      1.48
    #> 3     1.3      1.49
    #> 4     1.5      1.42
    #> 5     1.4      1.45
    #> 6     1.7      1.51
    #> 7     1.4      1.43
    #> 8     1.5      1.49
    #> 9     1.4      1.40
    #> 10    1.5      1.49

## Multiclass Classification

Classify iris species (3 classes). Note `task = "multiclass"`:

    iris_mc <- iris
    iris_mc$Species <- as.character(iris_mc$Species)

    set.seed(99)
    res_mc <- evolve_features(
      data       = iris,
      target_col = "Species",
      task       = "multiclass",
      evaluator  = "xgboost",
      generations = 5,
      pop_size    = 8,
      cv_folds    = 3,
      early_stopping_rounds = 3,
      verbose     = TRUE
    )
    #> Starting Evolutionary Feature Engineering...
    #>   Task: multiclass
    #>   Evaluator: xgboost
    #>   Generations: 5, Population Size: 8, CV Folds: 3
    #>   Original Numeric columns: Sepal.Length, Sepal.Width, Petal.Length, Petal.Width
    #>   Original Categorical columns:
    #>
    #> [Gen 0] Initialized Population:
    #>   Individual 1: [Original features only]
    #>   Individual 2: [mst_score(Sepal.Length, Sepal.Width), quantile_binning_cat5(Sepal.Length)]
    #>   Individual 3: [sqrt(Petal.Width), multiply(Petal.Width, Petal.Width)]
    #>   Individual 4: [add(Sepal.Width, Petal.Width), quantile_binning_cat7(Petal.Width)]
    #>   Individual 5: [add(Sepal.Length, Petal.Length, Sepal.Width), divide(Sepal.Width, Petal.Width)]
    #>   Individual 6: [subtract(Petal.Width, Petal.Length), random_projection(Sepal.Width, Sepal.Length)]
    #>   Individual 7: [mst_score(Sepal.Width, Petal.Length), log(Sepal.Width)]
    #>   Individual 8: [deadwood(Petal.Length, Sepal.Width), multiply(Petal.Length, Petal.Width, Petal.Width, Sepal.Length)]
    #>
    #> --- Generation 1 / 5 ---
    #>   Tested Individual 1 (New Best!) -> Fitness: 0.8425
    #>   Tested Individual 2 -> Fitness: 0.8391
    #>   Tested Individual 3 -> Fitness: 0.8425
    #>   Tested Individual 4 -> Fitness: 0.8382
    #>   Tested Individual 5 (New Best!) -> Fitness: 0.8499
    #>   Tested Individual 6 -> Fitness: 0.8460
    #>   Tested Individual 7 -> Fitness: 0.8456
    #>   Tested Individual 8 -> Fitness: 0.8467
    #>   Gen 1 Best Fitness: 0.8499
    #>   Gen 1 Best Recipe: [add(Sepal.Length, Petal.Length, Sepal.Width), divide(Sepal.Width, Petal.Width)]
    #>
    #> --- Generation 2 / 5 (Current Best Fitness: 0.8499) ---
    #>   Tested Individual 2 -> Fitness: 0.8467
    #>   Tested Individual 3 -> Fitness: 0.8456
    #>   Tested Individual 4 -> Fitness: 0.8431
    #>   Tested Individual 5 -> Fitness: 0.8499
    #>   Tested Individual 6 -> Fitness: 0.8448
    #>   Tested Individual 7 (New Best!) -> Fitness: 0.8536
    #>   Tested Individual 8 -> Fitness: 0.8456
    #>   Gen 2 Best Fitness: 0.8536
    #>   Gen 2 Best Recipe: [add(Sepal.Length, Petal.Length, Sepal.Width), divide(Sepal.Width, Petal.Width), deadwood(Petal.Length, Sepal.Width), multiply(Petal.Length, Petal.Width, Petal.Width, Sepal.Length)]
    #>
    #> --- Generation 3 / 5 (Current Best Fitness: 0.8536) ---
    #>   Tested Individual 2 -> Fitness: 0.8536
    #>   Tested Individual 3 -> Fitness: 0.8499
    #>   Tested Individual 4 -> Fitness: 0.8499
    #>   Tested Individual 5 -> Fitness: 0.8443
    #>   Tested Individual 6 (New Best!) -> Fitness: 0.8589
    #>   Tested Individual 7 -> Fitness: 0.8536
    #>   Tested Individual 8 -> Fitness: 0.8536
    #>   Gen 3 Best Fitness: 0.8589
    #>   Gen 3 Best Recipe: [add(Sepal.Length, Petal.Length, Sepal.Width), divide(Sepal.Width, Petal.Width), deadwood(Petal.Length, Sepal.Width), multiply(Petal.Length, Petal.Width, Petal.Width, Sepal.Length), add(Petal.Length, Petal.Length, Petal.Width, Petal.Length)]
    #>
    #> --- Generation 4 / 5 (Current Best Fitness: 0.8589) ---
    #>   Tested Individual 2 -> Fitness: 0.8536
    #>   Tested Individual 3 -> Fitness: 0.8499
    #>   Tested Individual 4 -> Fitness: 0.8536
    #>   Tested Individual 5 -> Fitness: 0.8440
    #>   Tested Individual 6 -> Fitness: 0.8565
    #>   Tested Individual 7 -> Fitness: 0.8536
    #>   Tested Individual 8 -> Fitness: 0.8589
    #>   Gen 4 Best Fitness: 0.8589
    #>   Gen 4 Best Recipe: [add(Sepal.Length, Petal.Length, Sepal.Width), divide(Sepal.Width, Petal.Width), deadwood(Petal.Length, Sepal.Width), multiply(Petal.Length, Petal.Width, Petal.Width, Sepal.Length), add(Petal.Length, Petal.Length, Petal.Width, Petal.Length)]
    #>
    #> --- Generation 5 / 5 (Current Best Fitness: 0.8589) ---
    #>   Tested Individual 2 -> Fitness: 0.8589
    #>   Tested Individual 3 -> Fitness: 0.8536
    #>   Tested Individual 4 -> Fitness: 0.8477
    #>   Tested Individual 5 -> Fitness: 0.8565
    #>   Tested Individual 6 -> Fitness: 0.8589
    #>   Tested Individual 7 -> Fitness: 0.8589
    #>   Tested Individual 8 -> Fitness: 0.8550
    #>   Tested Individual 9 (New Best!) -> Fitness: 0.8591
    #>   Tested Individual 10 -> Fitness: 0.8575
    #>   Tested Individual 11 -> Fitness: 0.8589
    #>   Tested Individual 12 -> Fitness: 0.8589
    #>   Gen 5 Best Fitness: 0.8591
    #>   Gen 5 Best Recipe: [add(Sepal.Length, Petal.Length, Sepal.Width), divide(Sepal.Width, Petal.Width), deadwood(Petal.Length, Sepal.Width), multiply(Petal.Length, Petal.Width, Petal.Width, Sepal.Length), add(Petal.Length, Petal.Length, Petal.Width, Petal.Length), mst_score(((Sepal.Length+Petal.Length+Sepal.Width)), Sepal.Length)]
    #>
    #> Evolution Complete. Best Fitness: 0.8591
    #> Best recipe: [add(Sepal.Length, Petal.Length, Sepal.Width), divide(Sepal.Width, Petal.Width), deadwood(Petal.Length, Sepal.Width), multiply(Petal.Length, Petal.Width, Petal.Width, Sepal.Length), add(Petal.Length, Petal.Length, Petal.Width, Petal.Length), mst_score(((Sepal.Length+Petal.Length+Sepal.Width)), Sepal.Length)]
    #> Generated columns: ((Sepal.Length+Petal.Length+Sepal.Width)), ((Sepal.Width/Petal.Width)), Deadwood(Pet_Sep), ((Petal.Length*Petal.Width*Petal.Width*Sepal.Length)), ((Petal.Length+Petal.Length+Petal.Width+Petal.Length)), MSTScore(((S_Sep)
    #> Training final model on full dataset...

    cat("Best recipe:", individual_to_recipe_string(res_mc$best_individual), "\n")
    #> Best recipe: [add(Sepal.Length, Petal.Length, Sepal.Width), divide(Sepal.Width, Petal.Width), deadwood(Petal.Length, Sepal.Width), multiply(Petal.Length, Petal.Width, Petal.Width, Sepal.Length), add(Petal.Length, Petal.Length, Petal.Width, Petal.Length), mst_score(((Sepal.Length+Petal.Length+Sepal.Width)), Sepal.Length)]

For multiclass, `predict_model()` returns a probability matrix — one column per class:

    probs <- predict_model(res_mc, iris_mc[c(1, 51, 101), ])
    round(probs, 3)
    #>      setosa versicolor virginica
    #> [1,]  0.986      0.008     0.006
    #> [2,]  0.009      0.973     0.018
    #> [3,]  0.006      0.007     0.987

## Transformer Reference

evoFE ships with 30 built-in transformers that the genetic algorithm can select from during evolution. The table below groups them by category.

### Arithmetic (numeric → numeric)

Transformer | Arity | Description
--- | --- | ---
`log` | unary | Natural logarithm (safe: `log(abs(x) + 1)`)
`sqrt` | unary | Square root (safe: `sqrt(abs(x))`)
`reciprocal` | unary | `1 / (x + ε)`
`add` | multi | Element-wise sum of 2+ columns
`subtract` | binary | `x₁ − x₂`
`multiply` | multi | Element-wise product of 2+ columns
`divide` | binary | `x₁ / (x₂ + ε)`
`normalized_difference` | binary | `(x₁ − x₂) / (x₁ + x₂ + ε)`
`log_ratio` | binary | `log((x₁ + ε) / (x₂ + ε))`

### Group-by aggregations (mixed → numeric)

These combine a categorical grouping column with a numeric value column.

Transformer | Description
--- | ---
`groupby_mean` | Mean of value within each group
`groupby_sd` | Standard deviation within each group
`groupby_max` / `groupby_min` | Max / min within each group
`groupby_ratio` | `value / group_mean`
`groupby_zscore` | `(value − group_mean) / group_sd`

### Encoding & binning

Transformer | Input → Output | Description
--- | --- | ---
`target_encode` | cat → num | Supervised mean-target encoding with leave-one-out
`frequency_encode` | cat → num | Proportion of each category in the data
`one_hot_encode` | cat → num | Binary one-hot encoding indicator for a specific category (or “other” for rare categories)
`quantile_binning` | num → num | Assign quantile rank (1–5)
`log_binning` | num → num | Assign log-scale bin index
`quantile_binning_cat` | num → cat | Same as `quantile_binning`, output as factor
`log_binning_cat` | num → cat | Same as `log_binning`, output as factor

### Dimensionality reduction (numeric → numeric)

Transformer | Description
--- | ---
`pca` | First principal component of 2+ columns
`truncated_svd` | First component from truncated SVD
`random_projection` | Random linear combination of 2+ columns
`umap` | Low-dimensional UMAP projection

### Manifold & Graph Learning (numeric → categorical/numeric)

Transformer | Output | Description
--- | --- | ---
`genie` | categorical | Genie robust hierarchical clustering
`lumbermark` | categorical | Lumbermark hierarchical clustering
`mst_score` | numeric | Minimum Spanning Tree-based anomaly score
`deadwood` | categorical | Deadwood anomaly detection (outlier indicators)

## Hierarchical Features (Gene Chaining)

One of evoFE’s powerful capabilities is hierarchical feature construction. After a gene has been evaluated and proven useful, subsequent generations can build on top of its output.

For example:

    Gen 1: log_ratio(Sepal.Length, Petal.Width)        → tested ✓
    Gen 2: divide(Petal.Width, logratio(…))            → chains from tested gene ✓

Important safety rule: a gene can only chain from outputs that have been evaluated in a previous generation. A brand-new untested gene is never used as input for another gene in the same individual. This prevents fragile dependency chains built on unproven transformations.

## Understanding the Output

`evolve_features()` returns an `evo_recipe` S3 object with:

Field | Description
--- | ---
`best_individual` | The winning recipe (list of genes, column sets, fitness)
`best_model` | The final LightGBM/XGBoost model trained on all data
`history` | Full final-generation population (for inspection)
`task` | The task type used
`evaluator` | The evaluator used
`classes` | Class labels (multiclass only)

### Inspecting the recipe

    ind <- res$best_individual

    # Human-readable recipe string
    cat(individual_to_recipe_string(ind), "\n")
    #> [multiply(disp, gear, wt, wt, wt), pca1(carb, vs, hp), pca2(carb, vs, hp), umap2(disp, wt)]

    # Number of evolved genes
    cat("Evolved genes:", length(ind$genes), "\n")
    #> Evolved genes: 4

    # Original columns retained
    cat("Numeric cols: ", paste(ind$numeric_cols, collapse = ", "), "\n")
    #> Numeric cols:  mpg, cyl, disp, hp, drat, wt, qsec, vs, gear, carb
    cat("Categorical cols:", paste(ind$categorical_cols, collapse = ", "), "\n")
    #> Categorical cols:

    # Individual gene details
    for (g in ind$genes) {
      cat(sprintf("  %s(%s) → %s\n",
        g$transformer_name,
        paste(g$input_cols, collapse = ", "),
        g$output_col))
    }
    #>   multiply(disp, gear, wt, wt, wt) → ((disp*gear*wt*wt*wt))
    #>   pca(carb, vs, hp) → PCA1(car_vs_hp)
    #>   pca(carb, vs, hp) → PCA2(car_vs_hp)
    #>   umap(disp, wt) → UMAP2(dis_wt)

## Evaluation Strategies

`evoFE` supports two evaluation strategies for scoring individuals:

1. Cross-Validation (`cv`): The default strategy. Evaluates the fitness of individuals using \(K\)-fold cross-validation (`cv_folds` parameter).
2. Train/Validation/Holdout Split (`split`): Useful for faster evaluation on larger datasets. You configure it with `evaluation_strategy = "split"` and `split_ratio` (e.g., `c(0.6, 0.2, 0.2)`).
   - The first two portions of `split_ratio` are used as the Train and Validation sets to score the candidate recipes during the evolutionary search.
   - The third portion (if provided) is the Holdout set. To prevent data leakage/snooping and optimize computation time, the holdout set is only evaluated once at the very end of evolution on the final selected best individual.

## Tuning Parameters

### Key parameters for `evolve_features()`

Parameter | Default | Description
--- | --- | ---
`generations` | 10 | Maximum number of evolutionary generations
`pop_size` | 10 | Number of individuals per generation
`evaluation_strategy` | `"cv"` | Evaluation method: `"cv"` (cross-validation) or `"split"` (train/val/holdout split)
`cv_folds` | 3 | Cross-validation folds for fitness evaluation (only used if strategy is `"cv"`)
`split_ratio` | `c(0.6, 0.2, 0.2)` | Proportions for Train/Val/Holdout split (only used if strategy is `"split"`)
`split_ids` | `NULL` | Optional user-defined vector of split assignments (`"train"`, `"val"`, `"holdout"`)
`early_stopping_rounds` | 3 | Stop if no improvement for n generations
`evaluator` | `"lightgbm"` | Model backend: `"lightgbm"` or `"xgboost"`
`dynamic_population` | `TRUE` | Expand population during stagnation
`crossover_type` | `"both"` | `"random"`, `"union"`, or `"both"`
`threads` | 2 | Parallelism for model training

### Practical advice

- Start small: `generations = 5, pop_size = 8` is enough to validate the pipeline. Scale up once you confirm the setup works.
- Increase `pop_size` for wider exploration. Useful when you have many columns (> 20) and diverse transformer options.
- Increase `generations` for deeper search. Works best when combined with `dynamic_population = TRUE` so stagnation triggers population expansion.
- Use `set.seed()` before calling `evolve_features()` for reproducible experiments and benchmarking.
- `crossover_type = "union"` tends to produce larger recipes (more features). `"random"` keeps recipes leaner.

## Reproducibility

Calling `set.seed()` before `evolve_features()` guarantees identical results across runs:

    set.seed(42)
    r1 <- evolve_features(iris[,1:5], "Petal.Length", task = "regression",
                           generations = 3, pop_size = 5, evaluator = "xgboost",
                           verbose = FALSE)
    set.seed(42)
    r2 <- evolve_features(iris[,1:5], "Petal.Length", task = "regression",
                           generations = 3, pop_size = 5, evaluator = "xgboost",
                           verbose = FALSE)

    identical(r1$best_individual$fitness, r2$best_individual$fitness)
    #> [1] TRUE
    identical(
      individual_to_recipe_string(r1$best_individual),
      individual_to_recipe_string(r2$best_individual)
    )
    #> [1] TRUE

## End-to-End Example: Train/Test Split

A realistic workflow with hold-out evaluation:

    data(iris)
    set.seed(1)
    idx <- sample(nrow(iris), 0.7 * nrow(iris))
    train <- iris[idx, ]
    test  <- iris[-idx, ]

    # Evolve on training data only
    set.seed(7)
    recipe <- evolve_features(
      data      = train[, 1:4],               # exclude Species
      target_col = "Petal.Length",
      task       = "regression",
      evaluator  = "xgboost",
      generations = 5,
      pop_size    = 8,
      verbose     = FALSE
    )

    # Predict on held-out test data
    test_preds <- predict_model(recipe, test[, 1:4])

    # Evaluate
    rmse <- sqrt(mean((test$Petal.Length - test_preds)^2))
    cat(sprintf("Test RMSE: %.4f\n", rmse))
    #> Test RMSE: 0.2764
    cat(sprintf("Recipe:    %s\n", individual_to_recipe_string(recipe$best_individual)))
    #> Recipe:    [lumbermark(Sepal.Length, Petal.Width), add(Petal.Width, Sepal.Length), log_ratio(((Petal.Width+Sepal.Length)), Petal.Width)]

## Session Info

    sessionInfo()
    #> R version 4.6.0 (2026-04-24)
    #> Platform: aarch64-apple-darwin25.4.0
    #> Running under: macOS Tahoe 26.5
    #>
    #> Matrix products: default
    #> BLAS:   /opt/homebrew/Cellar/openblas/0.3.33/lib/libopenblasp-r0.3.33.dylib
    #> LAPACK: /opt/homebrew/Cellar/r/4.6.0/lib/R/lib/libRlapack.dylib;  LAPACK version 3.12.1
    #>
    #> locale:
    #> [1] C/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
    #>
    #> time zone: Europe/Stockholm
    #> tzcode source: internal
    #>
    #> attached base packages:
    #> [1] stats     graphics  grDevices utils     datasets  methods   base
    #>
    #> other attached packages:
    #> [1] evoFE_0.1.0
    #>
    #> loaded via a namespace (and not attached):
    #>  [1] lumbermark_0.9.0    cli_3.6.6           knitr_1.51
    #>  [4] rlang_1.2.0         xfun_0.57           otel_0.2.0
    #>  [7] quitefastmst_0.9.1  jsonlite_2.0.0      data.table_1.18.4
    #> [10] htmltools_0.5.9     sass_0.4.10         uwot_0.2.4
    #> [13] rmarkdown_2.31      grid_4.6.0          evaluate_1.0.5
    #> [16] jquerylib_0.1.4     deadwood_0.9.0-3    fastmap_1.2.0
    #> [19] yaml_2.3.12         lifecycle_1.0.5     RhpcBLASctl_0.23-42
    #> [22] compiler_4.6.0      codetools_0.2-20    Rcpp_1.1.1-1.1
    #> [25] lattice_0.22-9      digest_0.6.39       xgboost_3.2.1.1
    #> [28] R6_2.6.1            bslib_0.10.0        Matrix_1.7-5
    #> [31] tools_4.6.0         genieclust_1.3.0    RcppAnnoy_0.0.23
    #> [34] cachem_1.1.0