In-context Learning with Transformer Is Really Equivalent to a Contrastive Learning Pattern

no code implementations • 20 Oct 2023 • Ruifeng Ren, Yong liu

To the best of our knowledge, our work is the first to provide the understanding of ICL from the perspective of contrastive learning and has the potential to facilitate future model design by referring to related works on contrastive learning.

Contrastive Learning In-Context Learning