no code implementations • 7 Mar 2024 • Yanqi Dai, Dong Jing, Nanyi Fei, Zhiwu Lu
To mitigate this issue, we propose a novel Comprehensive Task Balancing (CoTBal) algorithm for multi-task visual instruction tuning of LMMs.
1 code implementation • 28 Jul 2023 • Yanqi Dai, Nanyi Fei, Zhiwu Lu
In this paper, following the loss balancing framework, we propose two novel improvable gap balancing (IGB) algorithms for MTL: one takes a simple heuristic, and the other (for the first time) deploys deep reinforcement learning for MTL.