Unveiling Vulnerability of Self-Attention

1 code implementation • 26 Feb 2024 • Khai Jiet Liong, Hongqiu Wu, Hai Zhao

(2) We introduce \textit{S-Attend}, a novel smoothing technique that effectively makes SA robust via structural perturbations.