Author: Victor Navarro

# using_time_models

#### Victor Navarro

## Time models in calmr

Version 0.5 of `calmr` introduced its first time-based model, ANCCR (Jeong et al., 2022), and with it, I wrote several additional tools for future time-based models.

#### Changes to trial-based models

The biggest change in `calmr` version 0.5 is the use of the “>” character and its effect on trial-based models. With the advent of time-based models, some generalizations had to be made to enable those models to update across adjacent trial periods. You can learn more about this in the directional_models vignette.

#### Specifying a design for time-based models

The designs for time-based models are nearly identical to those for trial-based models. However, clever use of the “>” character will enrich them. Let’s specify a serial feature discrimination experiment:

    library(calmr)
    fpfn <- data.frame(
      group = c("FP", "FN"),
      phase1 = c("!100F>T>(US)/100T", "!100F>T/100T>(US)")
    )
    parse_design(fpfn)
    #> CalmrDesign built from data.frame:
    #>   group            phase1
    #> 1    FP !100F>T>(US)/100T
    #> 2    FN !100F>T/100T>(US)
    #> ----------------
    #> Trials detected:
    #>   group  phase trial_names trial_repeats is_test stimuli
    #> 1    FP phase1    F>T>(US)           100   FALSE  F;T;US
    #> 2    FP phase1           T           100   FALSE       T
    #> 3    FN phase1         F>T           100   FALSE     F;T
    #> 4    FN phase1      T>(US)           100   FALSE    T;US

We can manually specify the timing for the above experiment by calling the `get_timings()` function. Manipulating the list returned by that function will result in a manipulation of the timing between the experimental events.

    ts <- get_timings(fpfn, model = "ANCCR")
    ts
    #> $use_exponential
    #> [1] TRUE
    #>
    #> $sample_timings
    #> [1] TRUE
    #>
    #> $trial_ts
    #>      trial post_trial_delay mean_ITI max_ITI
    #> 1 F>T>(US)                1       30      90
    #> 2        T                1       30      90
    #> 3      F>T                1       30      90
    #> 4   T>(US)                1       30      90
    #>
    #> $transition_ts
    #>      trial transition transition_delay
    #> 1 F>T>(US)        F>T                1
    #> 2 F>T>(US)     T>(US)                1
    #> 3      F>T        F>T                1
    #> 4   T>(US)     T>(US)                1

And now let’s get the parameters for the ANCCR model.

    pars <- get_parameters(fpfn, model = "ANCCR")
    # increase learning rates
    pars$alpha_reward <- 0.8
    pars$alpha <- 0.08
    # increase sampling interval to speed up the model
    pars$sampling_interval <- 5
    pars
    #> $reward_magnitude
    #>  F  T US
    #>  1  1  1
    #>
    #> $betas
    #>  F  T US
    #>  1  1  1
    #>
    #> $cost
    #> [1] 0
    #>
    #> $temperature
    #> [1] 1
    #>
    #> $threshold
    #> [1] 0.6
    #>
    #> $k
    #> [1] 1
    #>
    #> $w
    #> [1] 0.5
    #>
    #> $minimum_rate
    #> [1] 0.001
    #>
    #> $sampling_interval
    #> [1] 5
    #>
    #> $use_exact_mean
    #> [1] 0
    #>
    #> $t_ratio
    #> [1] 1.2
    #>
    #> $t_constant
    #> [1] NA
    #>
    #> $alpha
    #> [1] 0.08
    #>
    #> $alpha_reward
    #> [1] 0.8
    #>
    #> $use_timed_alpha
    #> [1] 0
    #>
    #> $alpha_exponent
    #> [1] 1
    #>
    #> $alpha_init
    #> [1] 1
    #>
    #> $alpha_min
    #> [1] 0
    #>
    #> $add_beta
    #> [1] 0
    #>
    #> $jitter
    #> [1] 1

Let’s make the model’s experience and look at the first 20 entries.

    experiment <- make_experiment(fpfn,
      parameters = pars,
      timings = ts,
      model = "ANCCR"
    )
    head(experiences(experiment)[[1]], 20)
    #>    model group  phase tp       tn is_test block_size trial stimulus      time reward_mag
    #> 1  ANCCR    FP phase1  2        T   FALSE          2     1        T  28.49805          1
    #> 2  ANCCR    FP phase1  1 F>T>(US)   FALSE          2     2        F  55.50233          1
    #> 3  ANCCR    FP phase1  1 F>T>(US)   FALSE          2     2        T  56.50233          1
    #> 4  ANCCR    FP phase1  1 F>T>(US)   FALSE          2     2       US  57.50233          1
    #> 5  ANCCR    FP phase1  2        T   FALSE          2     3        T  70.39851          1
    #> 6  ANCCR    FP phase1  1 F>T>(US)   FALSE          2     4        F  77.58430          1
    #> 7  ANCCR    FP phase1  1 F>T>(US)   FALSE          2     4        T  78.58430          1
    #> 8  ANCCR    FP phase1  1 F>T>(US)   FALSE          2     4       US  79.58430          1
    #> 9  ANCCR    FP phase1  1 F>T>(US)   FALSE          2     5        F  85.42575          1
    #> 10 ANCCR    FP phase1  1 F>T>(US)   FALSE          2     5        T  86.42575          1
    #> 11 ANCCR    FP phase1  1 F>T>(US)   FALSE          2     5       US  87.42575          1
    #> 12 ANCCR    FP phase1  2        T   FALSE          2     6        T 104.34924          1
    #> 13 ANCCR    FP phase1  2        T   FALSE          2     7        T 161.84664          1
    #> 14 ANCCR    FP phase1  1 F>T>(US)   FALSE          2     8        F 164.21165          1
    #> 15 ANCCR    FP phase1  1 F>T>(US)   FALSE          2     8        T 165.21165          1
    #> 16 ANCCR    FP phase1  1 F>T>(US)   FALSE          2     8       US 166.21165          1
    #> 17 ANCCR    FP phase1  2        T   FALSE          2     9        T 186.19612          1
    #> 18 ANCCR    FP phase1  1 F>T>(US)   FALSE          2    10        F 249.42371          1
    #> 19 ANCCR    FP phase1  1 F>T>(US)   FALSE          2    10        T 250.42371          1
    #> 20 ANCCR    FP phase1  1 F>T>(US)   FALSE          2    10       US 251.42371          1

As you can see above, there are several rows per trial, each specifying a different stimulus. Time-based models like ANCCR run over a time log because they make ample use of the temporal difference between events.

Let’s run the model and see some plots.

    experiment <- run_experiment(experiment)

    # Action values
    patch_plots(plot(experiment, type = "action_values"))

    # ANCCR
    patch_plots(plot(experiment, type = "anccrs"))

    # Dopamine transients
    patch_plots(plot(experiment, type = "dopamines"))

And that’s it! Easy, right?

#### References

Jeong, H., Taylor, A., Floeder, J. R., Lohmann, M., Mihalas, S., Wu, B., Zhou, M., Burke, D. A., & Namboodiri, V. M. K. (2022). Mesolimbic dopamine release conveys causal associations. Science (New York, N.Y.), 378, eabq6740. https://doi.org/10.1126/science.abq6740