1 code implementation • 20 Jan 2024 • Georgios Ioannides, Aman Chadha, Aaron Elkins
We propose the Multi-Head Gaussian Adaptive Attention Mechanism (GAAM), a novel probabilistic attention framework, and the Gaussian Adaptive Transformer (GAT), designed to enhance information aggregation across multiple modalities, including Speech, Text and Vision.