> ## Documentation Index
> Fetch the complete documentation index at: https://docs.nimbusbci.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Evidence
> Exploratory Personalizer evidence from Papers 1 and 1b: head vs probe vs FT, LDA/QDA/Softmax, severity sweep, REVE, transfer map.
Figures and tables below come from two exploratory manuscripts:
1. **Universal Personalizer** — *Universal BCI Personalization: One API for Frozen EEG Trunks and Foundation Models* (Paper 1)
2. **Resource Ladder** — *When Should a Deployed BCI Decoder Spend Adaptation Resources? A Resource Ladder for Frozen Encoders* (Paper 1b)
All numbers are **exploratory**, protocol-local, and locked to the evaluation suite CSVs. Do not treat them as suite-wide constants.
**Headline (Paper 1):** ship one Personalizer; treat fine-tune as a priced escape hatch. Use the head where capacity exists; stay cal-only when clean wins.
Labels **L0 / L1 / LP / L2 / L3** — see the [plain-language legend](/personalizer/overview#ladder-legend-plain-language). **Shipped path = L0/L1**; LP is a bench control; L2 is protocol/bench (affine); L3 is the FT hatch.
## How the experiments work (data flow)
Before the numbers: this is the shared Paper 1 / 1b experimental pipeline.
*Papers 1 & 1b — end-to-end data flow. Calibration stays clean; synthetic stress hits adaptation and evaluation EEG only.*
| Step | What happens |
| - | - |
| **Datasets** | Motor imagery: BNCI2014-004 (9 subjects), Zhou2016 (4), Kumar2024 (18; post-first sessions online), Shin2017-A (9; short adaptation session) |
| **Preprocess** | Bandpass **8–30 Hz**, epoch window **\[0.5, 2.5] s** after cue (classical trunks). Foundation REVE cells may use **1–40 Hz** under the same window |
| **Sessions** | **Calibration** = first → **adaptation** = middle → **evaluation** = last |
| **Encoder** | Train on clean calibration only, freeze, map trials to embeddings |
| **Stress** | Optional synthetic **channel-mix** on concatenated adaptation + evaluation raw EEG (not on embeddings): mix toward a random channel permutation + signed bias at levels `{0, 0.25, 0.5, 0.75}`. Level 0 = no synthetic stress (natural session layout may still differ). Calibration stays clean |
| **Adapt** | L0 calibration-only head (Paper 1) / frozen trunk softmax (Paper 1b); L1 online head updates (\~40 labels, Shin \~20); linear probe = control; L2 affine map (bench); L3 warm-start fine-tune |
| **Score** | Evaluation accuracy + adaptation fit time under utility = accuracy − fit-time and label costs |
Not claimed as covered by this stress: physical electrode displacement, SNR drop, or every clinical shift type — only controlled synthetic channel mixing.
## Core: L0 / Bayesian head / linear probe / full FT
Main multi-trunk accuracy–cost block (Paper 1): under the same transfer cells (severe channel-mix stress, n=40, CUDA), calibration-only Personalizer (L0), stream Personalizer (L1), a matched linear probe on the same frozen embeddings, and warm-start full fine-tune (L3).
*Paper 1 — top: L0 / L1 / linear probe / L3 accuracy; blue `rec` labels = (acc\_L1 − acc\_L0) / (acc\_L3 − acc\_L0), or “≥FT” when L1 matches or beats L3. Bottom: adapt fit wall (L0 stream wall = 0) with L3/L1 ratios.*
| Dataset | Trunk | Acc L0 | Acc L1 | Acc LP | Acc L3 | Recovery | Wall L3/L1 |
| - | - | - | - | - | - | - | - |
| BNCI | EEGNet | 0.514 | 0.550 | 0.514 | 0.548 | ≥FT | \~31× |
| BNCI | Shallow | 0.495 | 0.572 | 0.580 | 0.553 | ≥FT | \~2× |
| BNCI | Deep | 0.497 | 0.573 | 0.536 | 0.594 | 78% | \~24× |
| BNCI | Conformer | 0.500 | 0.578 | 0.537 | 0.618 | 66% | \~33× |
| BNCI | ATCNet | 0.501 | 0.508 | 0.510 | 0.602 | 8% | \~40× |
| Zhou | EEGNet | 0.333 | 0.540 | 0.422 | 0.498 | ≥FT | \~21× |
| Zhou | Shallow | 0.390 | 0.425 | 0.445 | 0.543 | 23% | \~2× |
| Zhou | Deep | 0.335 | 0.470 | 0.462 | 0.482 | 92% | \~14× |
| Zhou | Conformer | 0.338 | 0.398 | 0.377 | 0.488 | 40% | \~29× |
| Zhou | ATCNet | 0.333 | 0.425 | 0.453 | 0.527 | 47% | \~39× |
Recovery = (acc\_L1 − acc\_L0) / (acc\_L3 − acc\_L0). On most trunks the Bayesian head recovers \~65–90% of the FT gain over cal-only and beats the matched probe (EEGNet / Deep / Conformer). Shallow and ATCNet are probe-competitive (LP can edge L1 by a few points) while both stay far below L3 wall — Personalizer remains the product default (same API + UQ / `BrainState`), not a claim that L1 always wins raw accuracy. BNCI Shallow is still the Pareto win vs FT (\~2× wall); ATCNet on BNCI is the hatch case (8% recovery).
**Mid severity strengthens “head first”:** across BNCI+Zhou subject cells, stream L1 matches or beats warm-start L3 on accuracy in about half of cells at severity 0.25 / 0.5 (\~49% / \~48%) versus about a third at 0.75 (\~35%). Wall L3/L1 ratios stay expensive at every severity — severe is where FT most often still buys residual pp.
## Which head? (LDA vs QDA vs Softmax)
The multi-trunk map locks **LDA** for comparability. On the EEGNet path, Paper 1 also compares QDA and Softmax under the same bake-off geometry (severe and clean, n=40).
*Paper 1 — top: accuracy (Δacc vs LDA). Bottom: adapt fit wall with mean utility labels.*
| Dataset | Sev | Head | Acc | Wall (s) | ΔU vs LDA |
| - | - | - | - | - | - |
| BNCI | 0.75 | LDA | 0.556 | 0.36 | — |
| BNCI | 0.75 | QDA | 0.644 | 0.36 | +0.085 |
| BNCI | 0.75 | Softmax | 0.589 | 0.59 | +0.031 |
| BNCI | 0.00 | LDA / QDA / Softmax | ≈0.71 | 0.35–0.71 | ≈0 |
| Zhou | 0.75 | LDA | 0.462 | 0.43 | — |
| Zhou | 0.75 | QDA | 0.452 | 0.43 | −0.010 |
| Zhou | 0.75 | Softmax | 0.457 | 0.71 | −0.005 |
* **Product default:** LDA (`head="lda"`) — no optional dependency; locked ladder / FT narrative.
* **BNCI severe:** QDA lifts utility (\~+0.085–0.088) at essentially the same wall; Softmax gains accuracy but pays \~1.7× LDA wall.
* **Zhou severe (n=4):** inconclusive — QDA / Softmax at or below LDA. Clean BNCI/Zhou: heads nearly tied on U.
* Softmax stays optional (`nimbus-bci[softmax]`) for UQ-oriented deployments. The 18-cell transfer map is not re-run for every head.
API knobs: [Encoder contract](/personalizer/encoder-contract).
## More evidence
Stream Personalizer (L1) vs calibration-only (L0) across synthetic channel-mix stress levels on an 18-cell trunk×dataset map:
| Severity | Cells with ΔU(L1−L0) > 0 |
| - | - |
| 0 (clean) | **4 / 18** |
| 0.25 | **13 / 18** |
| 0.50 | **16 / 18** |
| 0.75 (severe) | **12 / 18** |
*Paper 1 — mid shift already recovers; severe is a detailed slice, not the only supporting point. Kumar stays near zero (boundary).*
At severity 0.75, which ladder level wins by mean utility (L0 / L1 / **L2 protocol affine+head** — L2 is bench-only):
| Dataset | n | EEGNet | Shallow | Deep | Conformer | ATCNet |
| - | - | - | - | - | - | - |
| BNCI | 9 | L2 | L2 | L1 | L2 | L0 |
| Zhou | 4 | L2 | L1 | L1 | L2 | L2 |
| Kumar | 18 | L1 | L2 | — | L0 | — |
| Shin | 9 | L0 | L0 | L1 | L2 | L2 |
Zhou: head helps on every trunk. BNCI: often useful, not automatic (ATCNet stays L0). Kumar/Shin: boundaries — Personalizer is the default *integration* path, not a guarantee that L1 beats L0 everywhere.
Controlled bake-off on one classical reference trunk (BNCI primary, Paper 1b). Under severe shift with n=40 labeled stream trials, the Bayesian head can **beat** full FT on BNCI in this protocol; Zhou is the honest softer boundary.
*Paper 1b — BNCI2014-004, severity 0.75, n=40 (mean over 9 subjects).*
| Arm | Acc | ECE | Fit time (s) | Recovery vs full FT |
| - | - | - | - | - |
| Frozen EEGNet | 0.497 | 0.347 | 0 | — |
| **Bayesian head** | **0.556** | **0.142** | **0.35** | **1.59** |
| Linear probe | 0.514 | 0.117 | 0.35 | 0.46 |
| LoRA | 0.463 | 0.224 | 1.08 | −0.92 |
| Last-layer FT | 0.496 | 0.329 | 1.31 | −0.03 |
| Full FT | 0.534 | 0.191 | 1.78 | 1.00 |
Recovery = Δacc / Δacc\_fullFT (values > 1 mean the head beats FT). Treat head-beats-FT as **BNCI-primary** in this protocol, not a suite-wide constant. On Zhou2016 (supporting, 4 subjects): frozen 0.317 → head 0.462 → full FT 0.540 (\~0.43× of FT gain at \~5.5× lower fit time).
Paper 1 EEGNet arm zoo (same denser grammar; bake-off walls \~5× ≠ ladder CUDA L3/L1 \~20–30×):
| Dataset | Acc frozen | Acc head | Acc FT | Recovery | Bake-off wall FT/head |
| - | - | - | - | - | - |
| BNCI | 0.497 | 0.556 | 0.534 | **1.59×** | **5.0×** |
| Zhou | 0.317 | 0.462 | 0.540 | **0.43×** | **5.5×** |
The same Personalizer contract runs on Hub REVE. Under severe shift, adapt wall stays sub-second while last-layer / full FT cost about **20×–40×** more on Zhou/Kumar.
*Paper 1 — REVE severe n=40; accuracy × price.*
| Dataset | Acc frozen | Acc head | Acc LoRA | Acc last | Acc full | Wall LoRA/head | Wall full/head |
| - | - | - | - | - | - | - | - |
| Zhou | 0.365 | 0.412 | 0.452 | 0.395 | 0.393 | **\~8×** | **\~36×** |
| Kumar | 0.490 | 0.548 | 0.539 | 0.511 | 0.528 | **\~16×** | **\~51×** |
* **Kumar:** head Pareto-dominates short FT (+5.8 pp vs last +5.1 pp at far lower wall).
* **Zhou:** last-layer can buy extra pp (+13 vs head +4.7) at \~21× wall — priced purchase, not the default.
* **Not claimed:** frozen REVE + Personalizer matches cal-trained EEGNet absolute accuracy. BNCI REVE cost is a 3-subject smoke — not a headline.
Soft operational default vs strict ordinal schedule (18 cells; exploratory). “Severe → L2” means the **protocol** affine+head winner — in 0.6 affine is available as an opt-in `transform=` / `fit_affine()`, not a deployment default.
| Slice | Clean → L0 | Severe → L2 | Strict ordinal |
| - | - | - | - |
| Zhou | 5/5 | 3/5 | 3/5 |
| BNCI | 3/5 | 3/5 | 1/5 |
| Kumar | 3/3 | 1/3 | 1/3 |
| Shin | 1/5 | 2/5 | 0/5 |
| **Pooled** | **12/18** | **9/18** | **5/18** |
*Paper 1 — cal-only-when-clean transfers often (12/18); strict ordinal (clean→L0 and severe→L2) is conditional (5/18) — boundary, not the product headline.*
## Product claim (honest boundaries)
| Regime | When | What happens |
| - | - | - |
| Factory works | Clean / weak natural transfer | Stay cal-only (L0) — 12/18 cells |
| Factory breaks | Mid / severe controlled stress | Stream head (L1) often lifts utility; usually beats matched linear probe |
| Full retrain unnecessary | Same budgets as stress recovery | Classical: L1 recovers most of L3 at \~14–40× wall; EEGNet BNCI head can beat FT (\~5× bake-off wall); REVE \~20–40× vs short FT |
| Head choice | Same Personalizer API | Default LDA; QDA optional under severe shift (BNCI); Softmax optional for UQ |
Not claimed: L1 always beats L0 or LP; QDA always beats LDA; universal ordinal severity→level law; coverage of session / electrode / SNR shift (evidence is synthetic channel mixing); foundation SOTA; suite-wide 30× ratios from other protocols; shipped “Active Inference for EEG”; affine maps as a **deployment default** (they remain opt-in / research via `transform=`).
## Next Read
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.