Active Learning

Use active learning to collect labels for the trials that are most likely to improve your BCI model. The Python SDK exposes CalibrationSession for full calibration loops, plus helpers for pool ranking, streaming label requests, and label-free stopping.

Active learning answers which trials to label. That is different from Personalizer.recommend_adapt (whether to spend labels on stream partial_fit) and from BrainState trial rejection. See Personalizer overview for the adapt helper.

Active learning operates on preprocessed feature rows (or already-encoded embeddings), not raw EEG. Use arrays shaped (n_trials, n_features) for pools and (n_features,) or (1, n_features) for single streaming trials. With Personalizer, run AL on the head (adapter.head) or feed encoded pools into partial_fit.

With Personalizer

Compose active learning with the product surface: choose labels, then update the Personalizer head.

from copy import deepcopy
from nimbus_bci import Personalizer, wrap, calibration_sufficient
from nimbus_bci.active_learning import CalibrationSession

enc = wrap(model.encode, model_id="partner-encoder", embedding_dim=64)
adapter = Personalizer(encoder=enc, head="lda", classes=["left", "right"])
adapter.fit(X_seed, y_seed)

# Prefer the head for CalibrationSession (sklearn-compatible)
session = CalibrationSession(
    adapter.head,
    Z_pool,  # already-encoded unlabeled pool
    pool_strategy="bald",
    batch_size=4,
    stopping_threshold=0.02,
)

for _ in range(max_rounds):
    ranked = session.suggest_next_trial()
    global_indices = session.remaining_indices[ranked.indices]
    y_new = collect_labels_for(global_indices)
    X_new = X_pool[global_indices]  # raw trials for Personalizer encode path
    session.update(ranked.indices, y_new)
    adapter.partial_fit(X_new, y_new)
    if session.calibration_sufficient().is_sufficient:
        break

# Or label-free stopping directly on the Personalizer head
previous = deepcopy(adapter.head)
status = calibration_sufficient(
    adapter.head, Z_pool, previous=previous, criterion="posterior_stability"
)

Core Workflow

Start with a small seed calibration set, rank an unlabeled feature pool, collect labels for the most informative trials, update with partial_fit(), and stop when the posterior stops changing. For most applications, use CalibrationSession so pool bookkeeping and model snapshots stay consistent.

from nimbus_bci import NimbusLDA
from nimbus_bci.active_learning import CalibrationSession

clf = NimbusLDA().fit(X_seed, y_seed)
session = CalibrationSession(
    clf,
    X_pool,
    pool_strategy="bald",
    batch_size=4,
    stopping_threshold=0.02,
    num_posterior_samples=256,
)

for _ in range(max_rounds):
    ranked = session.suggest_next_trial()
    global_indices = session.remaining_indices[ranked.indices]
    y_new = collect_labels_for(global_indices)
    session.update(ranked.indices, y_new)

    status = session.calibration_sufficient()
    if status.is_sufficient:
        break

CalibrationSession

Use CalibrationSession when you want the SDK to manage the active pool, selected index history, partial_fit() updates, and the previous model snapshot needed by posterior_stability.

from nimbus_bci.active_learning import CalibrationSession

session = CalibrationSession(
    clf,
    X_pool,
    pool_strategy="bald",
    streaming_strategy="entropy",
    batch_size=4,
    stopping_threshold=0.02,
    streaming_threshold=1.0,
    num_posterior_samples=256,
    rng_seed=0,
)

ranked = session.suggest_next_trial()
global_indices = session.remaining_indices[ranked.indices]
y_new = collect_labels_for(global_indices)
session.update(ranked.indices, y_new)

status = session.calibration_sufficient()
decision = session.should_query(x_new)

suggest_next_trial() returns indices local to the current active pool. Use session.remaining_indices[ranked.indices] when labels are stored against the original pool. update() captures the pre-update model snapshot, calls partial_fit(), removes selected rows, and increments round_index and n_labeled.

For posterior_stability, call session.calibration_sufficient() only after at least one session.update(...). Before then, there is no previous model snapshot to compare.

Session State

Useful properties and history fields:

remaining_indices: original pool indices still available for querying.
remaining_pool: feature rows still available for querying.
n_remaining: number of candidates left.
is_exhausted: whether no candidates remain.
n_labeled: number of labels applied through update().
query_history: QueryResult objects returned by session ranking calls.
stopping_history: CalibrationStatus objects returned by stopping checks.
selected_global_indices: original pool indices selected each round.

When to Use Stateless Helpers

Use the lower-level helpers directly when you are building a custom loop, working with raw NimbusModel snapshots, or do not want the SDK to mutate a fitted classifier. CalibrationSession.update(...) requires a fitted Nimbus classifier with partial_fit().

Pool-Based Trial Ranking

Use suggest_next_trial() when you have an unlabeled pool of candidate feature rows and want the top n trials to label next.

from nimbus_bci.active_learning import suggest_next_trial

result = suggest_next_trial(
    clf,
    X_pool,
    strategy="bald",          # or "entropy", "margin", "least_confidence"
    n=4,
    num_posterior_samples=256,
    rng_seed=0,
)

print(result.indices)  # top pool indices, most informative first
print(result.scores)   # one informativeness score per row in X_pool

suggest_next_trial() accepts either a fitted Nimbus classifier (NimbusLDA, NimbusQDA, NimbusSoftmax, NimbusSTS) or a raw NimbusModel snapshot. It returns a QueryResult dataclass with:

indices: top-n indices into X_pool.
scores: raw informativeness score for every row in X_pool.
strategy: the strategy used.
n_posterior_samples: posterior samples used for the score (1 for cheap strategies).

strategy="bald" is supported for NimbusLDA, NimbusQDA, and NimbusSoftmax. It is not supported for NimbusSTS in this release because STS posterior sampling needs temporal-coupling support.

Streaming Query Gate

Use should_query() when a single trial arrives during a live session and you need to decide whether asking for a label is worth the calibration cost.

from nimbus_bci.active_learning import should_query

decision = should_query(
    clf,
    x_new,
    strategy="entropy",
    threshold=1.0,
)

if decision.should_query:
    request_label(x_new)

The result is a StreamingQueryDecision dataclass with:

should_query: whether the score crossed the threshold.
score: raw informativeness score.
threshold: threshold used for the decision.
strategy: strategy used.

should_query() intentionally supports only cheap strategies: entropy, margin, and least_confidence. Single-point BALD is too noisy; batch streaming trials and call suggest_next_trial(strategy="bald") if you need sample-based ranking.

Stopping Calibration

Use calibration_sufficient() to stop collecting labels once additional cues are unlikely to change predictions over the pool.

from nimbus_bci.active_learning import calibration_sufficient

status = calibration_sufficient(
    clf,
    X_pool,
    criterion="posterior_stability",
    previous=previous_model_snapshot,
    threshold=0.02,
)

if status.is_sufficient:
    print("Calibration can stop")

calibration_sufficient() returns a CalibrationStatus dataclass with:

is_sufficient: True when the criterion signal is below the threshold.
signal: mean total variation for posterior_stability, or mean BALD for expected_info_gain.
threshold: threshold used for the comparison.
criterion: criterion used.
details: extra diagnostic values such as max/min TV or BALD.

Stopping Criteria

posterior_stability compares two consecutive model snapshots over the same X_pool. It measures the mean total-variation distance between predict_proba outputs and works for every Nimbus head, including NimbusSTS.

status = calibration_sufficient(
    clf,
    X_pool,
    criterion="posterior_stability",
    previous=previous,
    threshold=0.02,
)

expected_info_gain measures mean BALD over the current pool. It does not use previous, and it is available for NimbusLDA, NimbusQDA, and NimbusSoftmax.

status = calibration_sufficient(
    clf,
    X_pool,
    criterion="expected_info_gain",
    threshold=0.05,
    num_posterior_samples=256,
)

For conservative calibration, use posterior_stability as the primary stopping criterion and treat expected_info_gain as a supporting signal. Early models can be confidently wrong and may underestimate expected information gain.

Strategy Guide

Strategy	Query direction	Units	Works with STS?	Best use
`entropy`	Higher is more informative	Bits, `[0, log2(n_classes)]`	Yes	Fast default for uncertain predictions
`margin`	Lower is more informative	Probability gap, `[0, 1]`	Yes	Top-1 vs top-2 ambiguity
`least_confidence`	Higher is more informative	`1 - max(p)`	Yes	Simple max-confidence thresholding
`bald`	Higher is more informative	Bits, `[0, log2(n_classes)]`	No	Epistemic uncertainty from posterior samples

Practical Defaults

Use CalibrationSession for end-to-end calibration loops.
Start with strategy="bald" for pool-based calibration when using NimbusLDA, NimbusQDA, or NimbusSoftmax.
Use num_posterior_samples=256 for BALD ranking stability. Lower values can be faster but noisier.
Use strategy="entropy" for streaming should_query() gates.
Use criterion="posterior_stability" for label-free stopping, with a threshold near 0.02 as an initial tuning point.
Keep X_pool fixed across a calibration round so scores and stability checks are comparable.

Next Read

Python API Reference

Function signatures and dataclass fields for active learning.

Streaming Inference

Combine real-time prediction with query gates and feedback.

sklearn Integration

Use Nimbus classifiers inside sklearn workflows.

Model Selection

Choose the right Bayesian head before calibration.

Getting started

Personalizer

Python SDK

Inference & pipelines

Heads & models

Concepts

Julia SDK

Cloud API

Examples

Active Learning

With Personalizer

Core Workflow

CalibrationSession

Session State

When to Use Stateless Helpers

Pool-Based Trial Ranking

Streaming Query Gate

Stopping Calibration

Stopping Criteria

Strategy Guide

Practical Defaults

Next Read

Python API Reference

Streaming Inference

sklearn Integration

Model Selection

​With Personalizer

​Core Workflow

​CalibrationSession

​Session State

​When to Use Stateless Helpers

​Pool-Based Trial Ranking

​Streaming Query Gate

​Stopping Calibration

​Stopping Criteria

​Strategy Guide

​Practical Defaults

​Next Read

Python API Reference

Streaming Inference

sklearn Integration

Model Selection

With Personalizer

Core Workflow

CalibrationSession

Session State

When to Use Stateless Helpers

Pool-Based Trial Ranking

Streaming Query Gate

Stopping Calibration

Stopping Criteria

Strategy Guide

Practical Defaults

Next Read