Active Continual Learning with Metaplastic Binary Bayesian Neural Networks

We are excited to share that our paper, Active Continual Learning with Metaplastic Binary Bayesian Neural Networks , has been accepted as a poster at ICML 2026 in Seoul, South Korea!

This work investigates a major limitation of continual learning systems deployed on embedded and low-power hardware: over time, many models progressively lose the ability to adapt. Parameters become increasingly rigid, uncertainty collapses, and learning slows down until the system effectively stops incorporating new information.

This issue becomes particularly important for edge AI systems operating in changing environments. Robots, embedded sensors, medical devices, and smart cameras cannot rely on large-scale retraining pipelines or cloud infrastructure. They must continue learning directly on-device while remaining computationally efficient.

Binary Bayesian Neural Networks at the Edge

Binary neural networks are attractive for energy-constrained hardware because they represent weights and activations using only two states, typically $+1$ and $-1$. This drastically reduces memory usage and enables fast bitwise computations compatible with microcontrollers and specialized hardware.

However, efficiency alone is insufficient for real-world deployment. Systems interacting with uncertain environments also require calibrated uncertainty estimates. Bayesian binary neural networks address this by maintaining probability distributions over binary weights instead of deterministic values.

Yet these models exhibit a major issue:

As learning progresses, sign flips become increasingly rare. The posterior distributions gradually collapse toward deterministic states. Eventually, probabilities saturate near $0$ or $1$, uncertainty disappears, and weights become effectively frozen. The network remains efficient, but loses plasticity and adaptability.

BiMU: Preventing Synaptic Freezing

To address this issue, we introduce BiMU , a metaplastic learning rule designed to preserve adaptability in Bayesian binary networks.

BiMU combines Bayesian updates with a controlled forgetting mechanism that prevents the indefinite accumulation of old evidence. Instead of allowing certainty to grow without bound, the method maintains a bounded memory state where outdated information can progressively decay.

Stable knowledge should remain protected, while uncertain or obsolete knowledge should stay easy to modify. This preserves meaningful uncertainty over long learning horizons and prevents the degeneracy typically observed in binary Bayesian models.

Active Continual Learning

Maintaining uncertainty is not only important for stability. It also enables efficient active learning.

Because the model retains calibrated uncertainty estimates, it can dynamically determine when human supervision is actually informative. Instead of requesting labels for every incoming sample, the system selectively queries only uncertain or high-value examples.

In our experiments, this reduced online supervision requirements by up to a factor of $32\times$ while maintaining strong continual learning performance.

This creates an appealing direction for long-lived embedded AI systems capable of:

• learning continuously after deployment,
• adapting to changing environments,
• operating under tight energy constraints,
• and minimizing costly human annotations.

Acknowledgements

This work was conducted with:

• Théo Ballet
• Djohan Bonnet
• Damien Querlioz

at Centre de Nanosciences et de Nanotechnologies, CNRS, Université Paris-Saclay.

This research is part of the Metaspin project, with connections to PEPR IA and France 2030.

Links

• Paper on arXiv: https://arxiv.org/abs/2605.30198

We are looking forward to presenting this work in Seoul at ICML 2026.