# Tutorial: Eliciting Bayesian Priors with ElicitationWizard

This tutorial walks through a complete prior elicitation session using the built-in default example: a hierarchical Poisson-Gamma model for adverse event counts in a multi-center clinical trial.

## Setup

Launch the app:

    library(ElicitationWizard)
    run_elicitation()

If you have not set `BASE_URL` and `API_KEY` as environment variables, expand the API Connection panel and enter them now.

## Step 1 — Define the Clinical Context

Expand the Clinical Context panel. The default text describes a metastatic hormone-resistant prostate cancer (HRPC) multi-center RCT. Edit this to match your own study if needed.

    Disease: Metastatic hormone-resistant prostate cancer (HRPC)
    Treatment: Control arm (placebo/standard care)
    Population: Adult oncology patients
    Study: Multi-center RCT

The clinical context is injected verbatim into every expert’s elicitation prompt. Be as specific as possible — endpoint definition, patient population, and comparator all influence the prior a domain expert would choose.

## Step 2 — Specify the Model and Parameters

Expand the Result Expectations panel. This tells the experts exactly which parameters to provide priors for, and in what format.

The default specifies the Poisson-Gamma hierarchical model:

    Model:
    Each patient i in site j has AE count:
    y_ij ~ Poisson(lambda_j)
    Site-specific rates: lambda_j ~ Gamma(alpha, beta)
    REQUIRED: alpha ~ Exponential(rate_alpha), beta ~ Exponential(rate_beta)

The `REQUIRED:` line instructs the LLM to provide priors specifically for `rate_alpha` and `rate_beta`, the hyperprior rates of the Exponential distributions on `alpha` and `beta`.

## Step 3 — Configure Experts

In the Expert Configuration section, choose how many LLM experts to query (1–4). Each expert is assigned an independent model and temperature:

- Model — select from the available elicitation models
- Temperature — controls response diversity; higher values (e.g. `1.0`) produce more varied priors; lower values (e.g. `0.1`) produce more conservative, deterministic responses

> Tip: When using the same model for multiple experts, set different temperatures to encourage diverse prior proposals rather than identical responses.

## Step 4 — Elicit Priors

Click Elicit Priors. All experts are queried in parallel — the button is disabled and each expert’s card shows a loading spinner until its response arrives.

When all responses have arrived, the button re-enables and the status clears.

## Step 5 — Review Expert Responses

Navigate to the Expert Responses tab. Each expert’s full response is rendered as a card with markdown formatting and LaTeX equations (MathJax).

At the top of the tab, an Extracted Prior Distributions box shows the distributions parsed from all responses — useful for a quick sanity check.

### Generating Stan Code from an Expert Response

Click the `</>` (code) icon in any expert card header to send that expert’s full response plus the model specification to the Stan code generation model. The generated Stan model appears in a dark card below the panels.

## Step 6 — Inspect the Prior Summary

Navigate to the Prior Summary tab. This shows:

- A table of all parsed distributions with a Valid column (✓ valid / ⚠ invalid)
- A density plot with one subplot per distribution family, overlaying all experts’ curves

Invalid distributions (e.g. out-of-range parameters) are flagged in the table and excluded from the density plot.

## Step 7 — Pool Expert Opinions

Navigate to the Opinion Pooling tab. This applies linear opinion pooling:

\[f(\theta) = \sum_{i} w_i f_i(\theta)\]

with equal weights across all valid experts. Distributions are grouped by family and parameter index, so e.g. `Exponential(rate_alpha)` and `Exponential(rate_beta)` are pooled separately.

The table shows a Valid column for each expert row and a summary row for the pooled distribution. If any experts are excluded (invalid distributions), the pooled row shows “K/N valid”.

Click Generate Stan Code to produce a Stan model incorporating the pooled hyperpriors.

## Step 8 (Optional) — Delphi Iterative Elicitation

The Delphi Method tab implements a structured consensus process. Instead of a single round, experts iteratively revise their priors after seeing each other’s responses.

### How it works

Round | Expert sees
--- | ---
Round 1 | Clinical context + model spec only
Round 2 | Clinical context + model spec + all other experts’ Round 1 parsed distributions
Round 3 | Clinical context + model spec + all other experts’ Round 2 parsed distributions

Up to 3 rounds are supported.

### Running a Delphi session

1. Click Run Delphi Round to start Round 1. Expert configuration is locked from this point.

2. Click Run Delphi Round again to run Round 2. Each expert now receives the other experts’ Round 1 distributions as peer feedback.

3. After any round completes, the Converged Distributions section appears at the bottom, showing the current consensus: a table, density plot, and Generate Stan Code button.

4. Click Reset to clear all rounds and start a new Delphi session. Expert configuration is unlocked.

> Note: The Delphi tab is independent from the main Elicit Priors button. Expert model and count selections apply to both, but Delphi rounds must be triggered separately.

## Exporting Results

### Stan Model

Any of the three Generate Stan Code buttons (Expert Responses, Opinion Pooling, Delphi) will produce a complete Stan model. Copy the code from the dark card — it includes the model structure from Result Expectations and incorporates the elicited hyperpriors.

### Downstream Validation

The generated hyperpriors can be passed directly to Stan as `rate_alpha` and `rate_beta` data to run cross-validation and sample efficiency experiments against real data.

## Troubleshooting

Symptom | Likely cause | Fix
--- | --- | ---
“Please provide a Base URL and API Key” notification | Missing credentials | Set `BASE_URL` / `API_KEY` env vars or fill in the API Connection panel
Expert card shows an error (red) | API call failed | Check Base URL format (must end in `/v1`), API key validity, and model availability
All distributions marked ⚠ Invalid | LLM did not follow the required format | Try a lower temperature, or use a model with stronger instruction-following (e.g. MedGemma, Llama-3.3)
Delphi round 2 shows “partial peer feedback” warning | One or more Round 1 experts returned no valid distributions | Review the invalid expert’s response in the round accordion; proceed or Reset and re-run
No distributions appear in Prior Summary | Parsing found nothing | Check that expert responses contain distribution strings in the form `Family(param1, param2)`