Search Results for author:
Found 2 papers, 2 papers with code
Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Tasks
State-space models (SSMs), such as Mamba (Gu & Dao, 2023), have been proposed as alternatives to Transformer networks in language modeling, by incorporating gating, convolutions, and input-dependent token selection to mitigate the quadratic cost of multi-head attention.
Teaching Arithmetic to Small Transformers
Even in the complete absence of pretraining, this approach significantly and simultaneously improves accuracy, sample complexity, and convergence speed.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
