no code implementations • 5 Mar 2024 • Gang Liu, Hongyang Li, Zerui He, Shenjun Zhong
In this paper, we introduce a method that incorporates gradient-guided parameter perturbations to the visual encoder of the multimodality model during both pre-training and fine-tuning phases, to improve model generalization for downstream medical VQA tasks.