Statistical mechanics of continual learning: variational principle and mean-field potential

1 code implementation • 6 Dec 2022 • Chan Li, Zhenye Huang, Wenxuan Zou, Haiping Huang

A variational Bayesian learning setting is thus proposed, where the neural networks are trained in a field-space, rather than gradient-ill-defined discrete-weight space, and furthermore, weight uncertainty is naturally incorporated, and modulates synaptic resources among tasks.

Continual Learning Multi-Task Learning