1 code implementation • 29 Feb 2024 • Weijieying Ren, Xinlong Li, Lei Wang, Tianxiang Zhao, Wei Qin
Through extensive experiments, we uncover the mode connectivity phenomenon in the LLMs continual learning scenario and find that it can strike a balance between plasticity and stability.