no code implementations • ACL 2022 • Mostafa Abdou, Vinit Ravishankar, Artur Kulmizev, Anders Søgaard
Recent studies have shown that language models pretrained and/or fine-tuned on randomly permuted sentences exhibit competitive performance on GLUE, putting into question the importance of word order information.
no code implementations • EMNLP 2020 • Anne Lauscher, Vinit Ravishankar, Ivan Vuli{\'c}, Goran Glava{\v{s}}
Massively multilingual transformers (MMTs) pretrained via language modeling (e. g., mBERT, XLM-R) have become a default paradigm for zero-shot language transfer in NLP, offering unmatched transfer performance.
no code implementations • NoDaLiDa 2021 • Vinit Ravishankar, Andrey Kutuzov, Lilja Øvrelid, Erik Velldal
Multilingual pretrained language models are rapidly gaining popularity in NLP systems for non-English languages.
no code implementations • COLING 2022 • Raúl Vázquez, Hande Celikkanat, Vinit Ravishankar, Mathias Creutz, Jörg Tiedemann
We analyze the learning dynamics of neural language and translation models using Loss Change Allocation (LCA), an indicator that enables a fine-grained analysis of parameter updates when optimizing for the loss function.
no code implementations • 1 Mar 2024 • Qinghua Zhao, Vinit Ravishankar, Nicolas Garneau, Anders Søgaard
Word order is an important concept in natural language, and in this work, we study how word order affects the induction of world knowledge from raw text using language models.
no code implementations • 26 Feb 2024 • Isabelle Mohr, Markus Krimmel, Saba Sturua, Mohammad Kalim Akram, Andreas Koukounas, Michael Günther, Georgios Mastrapas, Vinit Ravishankar, Joan Fontanals Martínez, Feng Wang, Qi Liu, Ziniu Yu, Jie Fu, Saahil Ognawala, Susana Guzman, Bo wang, Maximilian Werk, Nan Wang, Han Xiao
We introduce a novel suite of state-of-the-art bilingual text embedding models that are designed to support English and another target language.
no code implementations • 21 Mar 2022 • Vinit Ravishankar, Mostafa Abdou, Artur Kulmizev, Anders Søgaard
Recent studies have shown that language models pretrained and/or fine-tuned on randomly permuted sentences exhibit competitive performance on GLUE, putting into question the importance of word order information.
no code implementations • EMNLP 2021 • Vinit Ravishankar, Anders Søgaard
In order to preserve word-order information in a non-autoregressive setting, transformer architectures tend to include positional knowledge, by (for instance) adding positional encodings to token embeddings.
no code implementations • EACL 2021 • Vinit Ravishankar, Artur Kulmizev, Mostafa Abdou, Anders Søgaard, Joakim Nivre
Since the popularization of the Transformer as a general-purpose feature encoder for NLP, many studies have attempted to decode linguistic structure from its novel multi-head attention mechanism.
2 code implementations • ACL 2020 • Mostafa Abdou, Vinit Ravishankar, Maria Barrett, Yonatan Belinkov, Desmond Elliott, Anders Søgaard
Large-scale pretrained language models are the major driving force behind recent improvements in performance on the Winograd Schema Challenge, a widely employed test of common sense reasoning ability.
no code implementations • 1 May 2020 • Anne Lauscher, Vinit Ravishankar, Ivan Vulić, Goran Glavaš
Massively multilingual transformers pretrained with language modeling objectives (e. g., mBERT, XLM-R) have become a de facto default transfer paradigm for zero-shot cross-lingual transfer in NLP, offering unmatched transfer performance.
no code implementations • ACL 2020 • Artur Kulmizev, Vinit Ravishankar, Mostafa Abdou, Joakim Nivre
Recent work on the interpretability of deep neural language models has concluded that many properties of natural language syntax are encoded in their representational spaces.
1 code implementation • 19 Feb 2020 • Jeremy Barnes, Vinit Ravishankar, Lilja Øvrelid, Erik Velldal
Documents are composed of smaller pieces - paragraphs, sentences, and tokens - that have complex relationships between one another.
no code implementations • WS 2019 • Vinit Ravishankar, Memduh G{\"o}k{\i}rmak, Lilja {\O}vrelid, Erik Velldal
Encoders that generate representations based on context have, in recent years, benefited from adaptations that allow for pre-training on large text corpora.
no code implementations • WS 2019 • Vinit Ravishankar, Lilja Øvrelid, Erik Velldal
This paper extends the task of probing sentence representations for linguistic insight in a multilingual domain.
no code implementations • WS 2018 • Franck Burlot, Yves Scherrer, Vinit Ravishankar, Ond{\v{r}}ej Bojar, Stig-Arne Gr{\"o}nroos, Maarit Koponen, Tommi Nieminen, Fran{\c{c}}ois Yvon
Progress in the quality of machine translation output calls for new automatic evaluation procedures and metrics.
no code implementations • EMNLP 2018 • Mostafa Abdou, Artur Kulmizev, Vinit Ravishankar, Lasha Abzianidze, Johan Bos
We investigate the effects of multi-task learning using the recently introduced task of semantic tagging.
no code implementations • SEMEVAL 2018 • Artur Kulmizev, Mostafa Abdou, Vinit Ravishankar, Malvina Nissim
We participated to the SemEval-2018 shared task on capturing discriminative attributes (Task 10) with a simple system that ranked 8th amongst the 26 teams that took part in the evaluation.
no code implementations • JEPTALNRECITAL 2018 • Francis M. Tyers, Vinit Ravishankar
This paper describes the development of the first syntactically-annotated corpus of Breton.