Gaussian Stochastic Weight Averaging for Bayesian Low-Rank Adaptation of Large Language Models

no code implementations • 6 May 2024 • Emre Onal, Klemens Flöge, Emma Caldwell, Arsen Sheverdin, Vincent Fortuin

Fine-tuned Large Language Models (LLMs) often suffer from overconfidence and poor calibration, particularly when fine-tuned on small datasets.

Bayesian Inference