AMPLIFY:Attention-based Mixup for Performance Improvement and Label Smoothing in Transformer

1 code implementation • 22 Sep 2023 • Leixin Yang, Yu Xiang

This method uses the Attention mechanism of Transformer itself to reduce the influence of noises and aberrant values in the original samples on the prediction results, without increasing additional trainable parameters, and the computational cost is very low, thereby avoiding the problem of high resource consumption in common Mixup methods such as Sentence Mixup .

Data Augmentation Sentence +2