Bayesian Continual Learning in Nature Communications

I’m super happy to announce that Djohan Bonnet, Damien Querlioz and I, as well as co-authors recently published a paper about Bayesian continual learning & forgetting in Neural Networks in Nature Communications! It’s been a long journey to get this paper published and we are very proud of our research.

In this article, we investigate how Bayesian neural networks learn when faced with infinite streams of data, and how non-relevant information is discarded through time thanks to our weight-informed forgetting mechanism based on synaptic importance, circumventing both catastrophic forgetting - when neural networks forget about previous representations - and catastrophic remembering - when neural networks cannot learn anymore -.

Furthermore, we are drawing parallels between plausible neuroscience theories about the brain and machine learning, by finding equivalent update rules for synapses through our mathematical derivation.

We hope to pave the way towards Bayesian-based machine learning on-chip, where memory is limited and large models don’t fit. We are taking steps towards embedded devices generating uncertainties from the random variables representing the synapses, to achieve reliable lifelong learning.

You can find a conference talk taken at the UQ4ML Cometa Workshop at CEA on Youtube: COMETA Workshop - Kellian COTTART

Check out the open-access paper on the Nature website!

Bayesian Continual Learning and Forgetting in Neural Networks - Nature Communications