Bayesian LDA (RxLDA) - Bayesian Linear Discriminant Analysis

API Name: RxLDAModel
Mathematical Model: Pooled Gaussian Classifier (PGC) Bayesian LDA (also known as RxLDA in the codebase) is a Bayesian classification model with uncertainty quantification, implemented using RxInfer.jl’s reactive message passing framework. It uses a shared precision matrix across all classes, making it fast and efficient for BCI classification.

Bayesian LDA (RxLDA) is currently implemented in NimbusSDK.jl and ready for use in production BCI applications. LDA is widely recognized in the BCI community, and “Bayesian” signals our uncertainty quantification capabilities.

Overview

Bayesian LDA extends traditional Linear Discriminant Analysis with full Bayesian inference, providing:

✅ Posterior probability distributions (not just point estimates)
✅ Uncertainty quantification for each prediction
✅ Probabilistic confidence scores
✅ Fast inference (<20ms per trial)
✅ Training and calibration support
✅ Batch and streaming inference modes

When to Use Bayesian LDA

Bayesian LDA is ideal for:

Motor Imagery classification (2-4 classes)
Well-separated class distributions
Fast inference requirements (<20ms)
Interpretable results for medical applications
When classes have similar covariance structures

Consider Bayesian GMM instead if:

Classes have significantly different covariance structures
Complex, overlapping distributions
Need more flexibility in modeling per-class variances

Model Architecture

Mathematical Foundation (Pooled Gaussian Classifier)

Bayesian LDA implements a Pooled Gaussian Classifier (PGC), which models class-conditional distributions with a shared precision matrix:

p(x | y=k) = N(μ_k, W^-1)

Where:

μ_k = mean vector for class k (learned from data)
W = shared precision matrix (same for all classes)
Assumes classes have similar covariance structure

Key Assumption: All classes share the same covariance structure, which makes training and inference faster.

Model Structure

struct RxLDAModel <: BCIModel
    means::Vector{Vector{Float64}}            # Class means [μ₁, μ₂, ..., μₖ]
    precisions::Vector{Matrix{Float64}}       # Precision matrices (all equal to W)
    global_precision::Matrix{Float64}         # Shared precision W
    metadata::ModelMetadata                   # Model info
end

RxInfer Implementation

The Bayesian LDA model uses RxInfer.jl for variational Bayesian inference: Learning Phase:

@model function RxLDA_learning_model(y, labels, n_features, n_classes)
    # Prior on shared precision
    dof = n_features + 5
    W ~ Wishart(dof, I)
    
    # Priors on class means
    for k in 1:n_classes
        m[k] ~ MvNormal(0, 10*I)
    end
    
    # Likelihood
    for i in eachindex(y)
        k = labels[i]
        y[i] ~ MvNormal(m[k], inv(W))
    end
end

Prediction Phase:

@model function RxLDA_predictive(y, means, precisions)
    # Class assignment
    z ~ Categorical(uniform_priors)
    
    # Mixture model
    y ~ NormalMixture(means, precisions, z)
end

Usage

1. Load Pre-trained Model

using NimbusSDK

# Authenticate
NimbusSDK.install_core("nbci_live_your_key")

# Load from Nimbus model zoo
model = load_model(RxLDAModel, "motor_imagery_4class_v1")

println("Model loaded:")
println("  Features: $(get_n_features(model))")
println("  Classes: $(get_n_classes(model))")
println("  Paradigm: $(get_paradigm(model))")

2. Train Custom Model

using NimbusSDK

# Prepare training data with labels
train_features = csp_features  # (16 × 250 × 100)
train_labels = [1, 2, 3, 4, 1, 2, ...]  # 100 labels

train_data = BCIData(
    train_features,
    BCIMetadata(
        sampling_rate = 250.0,
        paradigm = :motor_imagery,
        feature_type = :csp,
        n_features = 16,
        n_classes = 4,
        chunk_size = nothing
    ),
    train_labels  # Required for training!
)

# Train RxLDA model
model = train_model(
    RxLDAModel,
    train_data;
    iterations = 50,        # Inference iterations
    showprogress = true,    # Show progress bar
    name = "my_motor_imagery",
    description = "4-class MI classifier with CSP"
)

# Save for later use
save_model(model, "my_model.jld2")

Training Parameters:

iterations: Number of variational inference iterations (default: 50)
- More iterations = better convergence but slower training
- 50-100 is typically sufficient
showprogress: Display progress bar during training
name: Model identifier
description: Model description for documentation

3. Subject-Specific Calibration

Fine-tune a pre-trained model with subject-specific data (much faster than training from scratch):

# Load base model
base_model = load_model(RxLDAModel, "motor_imagery_baseline_v1")

# Collect 10-20 calibration trials from new subject
calib_features = collect_calibration_trials()  # Your function
calib_labels = [1, 2, 3, 4, 1, 2, ...]

calib_data = BCIData(calib_features, metadata, calib_labels)

# Calibrate (personalize) the model
personalized_model = calibrate_model(
    base_model,
    calib_data;
    iterations = 20  # Fewer iterations needed
)

save_model(personalized_model, "subject_001_calibrated.jld2")

Calibration Benefits:

Requires only 10-20 trials per class (vs 50-100 for training from scratch)
Faster: 20 iterations vs 50-100
Better generalization: Uses pre-trained model as prior
Typical accuracy improvement: 5-15% over generic model

4. Batch Inference

Process multiple trials efficiently:

# Prepare test data
test_data = BCIData(test_features, metadata, test_labels)

# Run batch inference
results = predict_batch(model, test_data; iterations=10)

# Analyze results
println("Predictions: ", results.predictions)
println("Mean confidence: ", mean(results.confidences))

# Calculate accuracy
accuracy = sum(results.predictions .== test_labels) / length(test_labels)
println("Accuracy: $(round(accuracy * 100, digits=1))%")

# Calculate ITR
itr = calculate_ITR(accuracy, 4, 4.0)  # 4 classes, 4-second trials
println("ITR: $(round(itr, digits=1)) bits/minute")

5. Streaming Inference

Real-time chunk-by-chunk processing:

# Initialize streaming session
session = init_streaming(model, metadata_with_chunk_size)

# Process chunks as they arrive
for chunk in eeg_feature_stream
    result = process_chunk(session, chunk; iterations=10)
    println("Chunk: pred=$(result.prediction), conf=$(round(result.confidence, digits=3))")
end

# Finalize trial with aggregation
final_result = finalize_trial(session; method=:weighted_vote)
println("Final: pred=$(final_result.prediction), conf=$(round(final_result.confidence, digits=3))")

Training Requirements

Data Requirements

Minimum: 40 trials per class (160 total for 4-class)
Recommended: 80+ trials per class (320+ total for 4-class)
For calibration: 10-20 trials per class sufficient

Bayesian LDA requires at least 2 observations to estimate class statistics and shared precision matrix. Single-trial training will raise an error.

Feature Requirements

Bayesian LDA expects preprocessed features, not raw EEG: ✅ Required preprocessing:

Bandpass filtering (8-30 Hz for motor imagery)
Artifact removal (ICA recommended)
Spatial filtering (CSP for motor imagery)
Feature extraction (log-variance for CSP features)

❌ NOT accepted:

Raw EEG channels
Unfiltered data
Non-extracted features

See Preprocessing Requirements for details.

Performance Characteristics

Computational Performance

Operation	Latency	Notes
Training	10-30 seconds	50 iterations, 100 trials per class
Calibration	5-15 seconds	20 iterations, 20 trials per class
Batch Inference	10-20ms per trial	10 iterations
Streaming Chunk	10-20ms	10 iterations per chunk

All measurements on standard CPU (no GPU required).

Classification Accuracy

Paradigm	Classes	Typical Accuracy	ITR
Motor Imagery	2 (L/R hand)	75-90%	15-25 bits/min
Motor Imagery	4 (L/R/Feet/Tongue)	70-85%	20-35 bits/min
P300	2 (Target/Non-target)	80-95%	25-40 bits/min

Accuracy is highly subject-dependent. Subject-specific calibration typically improves accuracy by 5-15%.

Model Inspection

View Model Parameters

# Model structure
println("Class means:")
for (k, mean) in enumerate(model.means)
    println("  Class $k: ", mean)
end

println("\nShared precision matrix (first 3x3):")
println(model.global_precision[1:3, 1:3])

# Model metadata
println("\nMetadata:")
println("  Name: ", model.metadata.name)
println("  Paradigm: ", model.metadata.paradigm)
println("  Features: ", model.metadata.n_features)
println("  Classes: ", model.metadata.n_classes)

Compare Models

# Train multiple models and compare
models = []
for n_iter in [20, 50, 100]
    model = train_model(RxLDAModel, train_data; iterations=n_iter)
    results = predict_batch(model, test_data)
    accuracy = sum(results.predictions .== test_labels) / length(test_labels)
    
    println("Iterations: $n_iter, Accuracy: $(round(accuracy*100, digits=1))%")
    push!(models, (n_iter, model, accuracy))
end

Advantages & Limitations

Advantages

✅ Fast Training: Shared covariance estimation is efficient
✅ Fast Inference: Analytical posterior computation (<20ms)
✅ Interpretable: Clear probabilistic formulation
✅ Memory Efficient: Single shared precision matrix
✅ Robust: Handles uncertainty naturally via Bayesian inference
✅ Production-Ready: Battle-tested in real BCI applications

Limitations

❌ Shared Covariance Assumption: May not fit well if classes have very different spreads
❌ Linear Decision Boundary: Cannot capture non-linear class boundaries
❌ Gaussian Assumption: Assumes normal class distributions
❌ Not Ideal for Overlapping Classes: Use RxGMM for complex distributions

Comparison: Bayesian LDA vs Bayesian GMM

Aspect	Bayesian LDA (RxLDA)	Bayesian GMM (RxGMM)
Precision Matrix	Shared across all classes	Class-specific
Mathematical Model	Pooled Gaussian Classifier (PGC)	Heteroscedastic Gaussian Classifier (HGC)
Training Speed	Faster	Slower
Inference Speed	Faster (~10-15ms)	Slightly slower (~15-20ms)
Flexibility	Less flexible	More flexible
Best For	Well-separated classes	Overlapping/complex distributions
Memory	Lower	Higher
Parameters	Fewer (n_classes means + 1 precision)	More (n_classes means + precisions)

Rule of thumb: Start with Bayesian LDA. Switch to Bayesian GMM if accuracy is unsatisfactory.

Next Steps

Bayesian GMM (RxGMM)

More flexible Gaussian Mixture Model with class-specific covariances

Training Tutorial

Complete training walkthrough

Julia SDK

Full SDK reference

Code Examples

Working examples

References

Implementation:

RxInfer.jl: https://rxinfer.com/
Source code: /src/models/rxlda/ in NimbusSDK.jl

Theory:

Fisher, R. A. (1936). “The use of multiple measurements in taxonomic problems”
Bishop, C. M. (2006). “Pattern Recognition and Machine Learning” (Chapter 4)
Pooled Gaussian Classifier (PGC) with shared covariance structure

BCI Applications:

Blankertz et al. (2008). “Optimizing spatial filters for robust EEG single-trial analysis”
Lotte et al. (2018). “A review of classification algorithms for EEG-based BCI”

Getting started

API documentation

Core concepts

Model specification

Inference configuration

BCI Models

Examples & tutorials

Development

​Bayesian LDA (RxLDA) - Bayesian Linear Discriminant Analysis

​Overview

​When to Use Bayesian LDA

​Model Architecture

​Mathematical Foundation (Pooled Gaussian Classifier)

​Model Structure

​RxInfer Implementation

​Usage

​1. Load Pre-trained Model

​2. Train Custom Model

​3. Subject-Specific Calibration

​4. Batch Inference

​5. Streaming Inference

​Training Requirements

​Data Requirements

​Feature Requirements

​Performance Characteristics

​Computational Performance

​Classification Accuracy

​Model Inspection

​View Model Parameters

​Compare Models

​Advantages & Limitations

​Advantages

​Limitations

​Comparison: Bayesian LDA vs Bayesian GMM

​Next Steps

Bayesian GMM (RxGMM)

Training Tutorial

Julia SDK

Code Examples

​References

Bayesian LDA (RxLDA) - Bayesian Linear Discriminant Analysis

Overview

When to Use Bayesian LDA

Model Architecture

Mathematical Foundation (Pooled Gaussian Classifier)

Model Structure

RxInfer Implementation

Usage

1. Load Pre-trained Model

2. Train Custom Model

3. Subject-Specific Calibration

4. Batch Inference

5. Streaming Inference

Training Requirements

Data Requirements

Feature Requirements

Performance Characteristics

Computational Performance

Classification Accuracy

Model Inspection

View Model Parameters

Compare Models

Advantages & Limitations

Advantages

Limitations

Comparison: Bayesian LDA vs Bayesian GMM

Next Steps

References