> ## Documentation Index
> Fetch the complete documentation index at: https://docs.nimbusbci.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Active Learning
> Use nimbus-bci CalibrationSession and active learning helpers to reduce calibration time with pool-based trial ranking, streaming query gates, and label-free stopping.
Use active learning to collect labels for the trials that are most likely to improve your BCI model. The Python SDK exposes `CalibrationSession` for full calibration loops, plus helpers for pool ranking, streaming label requests, and label-free stopping.
Active learning answers **which trials to label**. That is different from `Personalizer.recommend_adapt` (whether to spend labels on stream `partial_fit`) and from `BrainState` trial rejection. See [Personalizer overview](/personalizer/overview) for the adapt helper.
Active learning operates on **preprocessed feature rows** (or already-encoded embeddings), not raw EEG. Use arrays shaped `(n_trials, n_features)` for pools and `(n_features,)` or `(1, n_features)` for single streaming trials. With Personalizer, run AL on the head (`adapter.head`) or feed encoded pools into `partial_fit`.
## With Personalizer
Compose active learning with the product surface: choose labels, then update the Personalizer head.
```python theme={null}
from copy import deepcopy
from nimbus_bci import Personalizer, wrap, calibration_sufficient
from nimbus_bci.active_learning import CalibrationSession
enc = wrap(model.encode, model_id="partner-encoder", embedding_dim=64)
adapter = Personalizer(encoder=enc, head="lda", classes=["left", "right"])
adapter.fit(X_seed, y_seed)
# Prefer the head for CalibrationSession (sklearn-compatible)
session = CalibrationSession(
adapter.head,
Z_pool, # already-encoded unlabeled pool
pool_strategy="bald",
batch_size=4,
stopping_threshold=0.02,
)
for _ in range(max_rounds):
ranked = session.suggest_next_trial()
global_indices = session.remaining_indices[ranked.indices]
y_new = collect_labels_for(global_indices)
X_new = X_pool[global_indices] # raw trials for Personalizer encode path
session.update(ranked.indices, y_new)
adapter.partial_fit(X_new, y_new)
if session.calibration_sufficient().is_sufficient:
break
# Or label-free stopping directly on the Personalizer head
previous = deepcopy(adapter.head)
status = calibration_sufficient(
adapter.head, Z_pool, previous=previous, criterion="posterior_stability"
)
```
## Core Workflow
Start with a small seed calibration set, rank an unlabeled feature pool, collect labels for the most informative trials, update with `partial_fit()`, and stop when the posterior stops changing. For most applications, use `CalibrationSession` so pool bookkeeping and model snapshots stay consistent.
```mermaid theme={null}
flowchart LR
seedModel["Seed model"] --> unlabeledPool["Unlabeled feature pool"]
unlabeledPool --> rankTrials["CalibrationSession.suggest_next_trial"]
rankTrials --> labelTrials["Collect labels"]
labelTrials --> updateModel["partial_fit"]
updateModel --> stopCheck["calibration_sufficient"]
stopCheck -->|"not stable"| rankTrials
stopCheck -->|"stable"| calibratedModel["Calibrated model"]
```
```python theme={null}
from nimbus_bci import NimbusLDA
from nimbus_bci.active_learning import CalibrationSession
clf = NimbusLDA().fit(X_seed, y_seed)
session = CalibrationSession(
clf,
X_pool,
pool_strategy="bald",
batch_size=4,
stopping_threshold=0.02,
num_posterior_samples=256,
)
for _ in range(max_rounds):
ranked = session.suggest_next_trial()
global_indices = session.remaining_indices[ranked.indices]
y_new = collect_labels_for(global_indices)
session.update(ranked.indices, y_new)
status = session.calibration_sufficient()
if status.is_sufficient:
break
```
## CalibrationSession
Use `CalibrationSession` when you want the SDK to manage the active pool, selected index history, `partial_fit()` updates, and the previous model snapshot needed by `posterior_stability`.
```python theme={null}
from nimbus_bci.active_learning import CalibrationSession
session = CalibrationSession(
clf,
X_pool,
pool_strategy="bald",
streaming_strategy="entropy",
batch_size=4,
stopping_threshold=0.02,
streaming_threshold=1.0,
num_posterior_samples=256,
rng_seed=0,
)
ranked = session.suggest_next_trial()
global_indices = session.remaining_indices[ranked.indices]
y_new = collect_labels_for(global_indices)
session.update(ranked.indices, y_new)
status = session.calibration_sufficient()
decision = session.should_query(x_new)
```
`suggest_next_trial()` returns indices local to the current active pool. Use `session.remaining_indices[ranked.indices]` when labels are stored against the original pool. `update()` captures the pre-update model snapshot, calls `partial_fit()`, removes selected rows, and increments `round_index` and `n_labeled`.
For `posterior_stability`, call `session.calibration_sufficient()` only after at least one `session.update(...)`. Before then, there is no previous model snapshot to compare.
### Session State
Useful properties and history fields:
* `remaining_indices`: original pool indices still available for querying.
* `remaining_pool`: feature rows still available for querying.
* `n_remaining`: number of candidates left.
* `is_exhausted`: whether no candidates remain.
* `n_labeled`: number of labels applied through `update()`.
* `query_history`: `QueryResult` objects returned by session ranking calls.
* `stopping_history`: `CalibrationStatus` objects returned by stopping checks.
* `selected_global_indices`: original pool indices selected each round.
### When to Use Stateless Helpers
Use the lower-level helpers directly when you are building a custom loop, working with raw `NimbusModel` snapshots, or do not want the SDK to mutate a fitted classifier. `CalibrationSession.update(...)` requires a fitted Nimbus classifier with `partial_fit()`.
## Pool-Based Trial Ranking
Use `suggest_next_trial()` when you have an unlabeled pool of candidate feature rows and want the top `n` trials to label next.
```python theme={null}
from nimbus_bci.active_learning import suggest_next_trial
result = suggest_next_trial(
clf,
X_pool,
strategy="bald", # or "entropy", "margin", "least_confidence"
n=4,
num_posterior_samples=256,
rng_seed=0,
)
print(result.indices) # top pool indices, most informative first
print(result.scores) # one informativeness score per row in X_pool
```
`suggest_next_trial()` accepts either a fitted Nimbus classifier (`NimbusLDA`, `NimbusQDA`, `NimbusSoftmax`, `NimbusSTS`) or a raw `NimbusModel` snapshot. It returns a `QueryResult` dataclass with:
* `indices`: top-`n` indices into `X_pool`.
* `scores`: raw informativeness score for every row in `X_pool`.
* `strategy`: the strategy used.
* `n_posterior_samples`: posterior samples used for the score (`1` for cheap strategies).
`strategy="bald"` is supported for `NimbusLDA`, `NimbusQDA`, and `NimbusSoftmax`. It is not supported for `NimbusSTS` in this release because STS posterior sampling needs temporal-coupling support.
## Streaming Query Gate
Use `should_query()` when a single trial arrives during a live session and you need to decide whether asking for a label is worth the calibration cost.
```python theme={null}
from nimbus_bci.active_learning import should_query
decision = should_query(
clf,
x_new,
strategy="entropy",
threshold=1.0,
)
if decision.should_query:
request_label(x_new)
```
The result is a `StreamingQueryDecision` dataclass with:
* `should_query`: whether the score crossed the threshold.
* `score`: raw informativeness score.
* `threshold`: threshold used for the decision.
* `strategy`: strategy used.
`should_query()` intentionally supports only cheap strategies: `entropy`, `margin`, and `least_confidence`. Single-point BALD is too noisy; batch streaming trials and call `suggest_next_trial(strategy="bald")` if you need sample-based ranking.
## Stopping Calibration
Use `calibration_sufficient()` to stop collecting labels once additional cues are unlikely to change predictions over the pool.
```python theme={null}
from nimbus_bci.active_learning import calibration_sufficient
status = calibration_sufficient(
clf,
X_pool,
criterion="posterior_stability",
previous=previous_model_snapshot,
threshold=0.02,
)
if status.is_sufficient:
print("Calibration can stop")
```
`calibration_sufficient()` returns a `CalibrationStatus` dataclass with:
* `is_sufficient`: `True` when the criterion signal is below the threshold.
* `signal`: mean total variation for `posterior_stability`, or mean BALD for `expected_info_gain`.
* `threshold`: threshold used for the comparison.
* `criterion`: criterion used.
* `details`: extra diagnostic values such as max/min TV or BALD.
### Stopping Criteria
`posterior_stability` compares two consecutive model snapshots over the same `X_pool`. It measures the mean total-variation distance between `predict_proba` outputs and works for every Nimbus head, including `NimbusSTS`.
```python theme={null}
status = calibration_sufficient(
clf,
X_pool,
criterion="posterior_stability",
previous=previous,
threshold=0.02,
)
```
`expected_info_gain` measures mean BALD over the current pool. It does not use `previous`, and it is available for `NimbusLDA`, `NimbusQDA`, and `NimbusSoftmax`.
```python theme={null}
status = calibration_sufficient(
clf,
X_pool,
criterion="expected_info_gain",
threshold=0.05,
num_posterior_samples=256,
)
```
For conservative calibration, use `posterior_stability` as the primary stopping criterion and treat `expected_info_gain` as a supporting signal. Early models can be confidently wrong and may underestimate expected information gain.
## Strategy Guide
| Strategy | Query direction | Units | Works with STS? | Best use |
| ------------------ | -------------------------- | ---------------------------- | --------------- | -------------------------------------------- |
| `entropy` | Higher is more informative | Bits, `[0, log2(n_classes)]` | Yes | Fast default for uncertain predictions |
| `margin` | Lower is more informative | Probability gap, `[0, 1]` | Yes | Top-1 vs top-2 ambiguity |
| `least_confidence` | Higher is more informative | `1 - max(p)` | Yes | Simple max-confidence thresholding |
| `bald` | Higher is more informative | Bits, `[0, log2(n_classes)]` | No | Epistemic uncertainty from posterior samples |
## Practical Defaults
* Use `CalibrationSession` for end-to-end calibration loops.
* Start with `strategy="bald"` for pool-based calibration when using `NimbusLDA`, `NimbusQDA`, or `NimbusSoftmax`.
* Use `num_posterior_samples=256` for BALD ranking stability. Lower values can be faster but noisier.
* Use `strategy="entropy"` for streaming `should_query()` gates.
* Use `criterion="posterior_stability"` for label-free stopping, with a threshold near `0.02` as an initial tuning point.
* Keep `X_pool` fixed across a calibration round so scores and stability checks are comparable.
## Next Read
Function signatures and dataclass fields for active learning.
Combine real-time prediction with query gates and feedback.
Use Nimbus classifiers inside sklearn workflows.
Choose the right Bayesian head before calibration.