no code implementations • ACL 2022 • Mostafa Abdou, Vinit Ravishankar, Artur Kulmizev, Anders Søgaard
Recent studies have shown that language models pretrained and/or fine-tuned on randomly permuted sentences exhibit competitive performance on GLUE, putting into question the importance of word order information.
1 code implementation • LREC (LAW) 2022 • Terne Sasha Thorn Jakobsen, Maria Barrett, Anders Søgaard, David Lassen
NLP models are dependent on the data they are trained on, including how this data is annotated.
no code implementations • ACL (WAT) 2021 • Rahul Aralikatte, Héctor Ricardo Murrieta Bello, Miryam de Lhoneux, Daniel Hershcovich, Marcel Bollmann, Anders Søgaard
This work shows that competitive translation results can be obtained in a constrained setting by incorporating the latest advances in memory and compute optimization.
no code implementations • NAACL (AmericasNLP) 2021 • Marcel Bollmann, Rahul Aralikatte, Héctor Murrieta Bello, Daniel Hershcovich, Miryam de Lhoneux, Anders Søgaard
We evaluated a range of neural machine translation techniques developed specifically for low-resource scenarios.
no code implementations • WNUT (ACL) 2021 • Heather Lent, Anders Søgaard
Large-scale language models such as ELMo and BERT have pushed the horizon of what is possible in semantic role labeling (SRL), solving the out-of-vocabulary problem and enabling end-to-end systems, but they have also introduced significant biases.
no code implementations • EMNLP (MRL) 2021 • Riccardo Bassani, Anders Søgaard, Tejaswini Deoskar
This work explores the idea of learning multilingual language models based on clustering of monolingual segments.
1 code implementation • ACL (BPPF) 2021 • Victor Petrén Bach Hansen, Anders Søgaard
NLP models struggle with generalization due to sampling and annotator bias.
1 code implementation • CoNLL (EMNLP) 2021 • Mareike Hartmann, Miryam de Lhoneux, Daniel Hershcovich, Yova Kementchedjhieva, Lukas Nielsen, Chen Qiu, Anders Søgaard
Negation is one of the most fundamental concepts in human cognition and language, and several natural language inference (NLI) probes have been designed to investigate pretrained language models’ ability to detect and reason with negation.
no code implementations • CRAC (ACL) 2021 • Maria Barrett, Hieu Lam, Martin Wu, Ophélie Lacroix, Barbara Plank, Anders Søgaard
Automatic coreference resolution is understudied in Danish even though most of the Danish Dependency Treebank (Buch-Kromann, 2003) is annotated with coreference relations.
no code implementations • EACL (Louhi) 2021 • Mareike Hartmann, Anders Søgaard
Negation scope resolution is key to high-quality information extraction from clinical texts, but so far, efforts to make encoders used for information extraction negation-aware have been limited to English.
no code implementations • EMNLP 2021 • Anders Søgaard
I highlight a simple failure mode of state-of-the-art machine reading systems: when contexts do not align with commonly shared beliefs.
no code implementations • EMNLP 2021 • Sheng Zhang, Xin Zhang, Weiming Zhang, Anders Søgaard
Using data from English cloze tests, in which subjects also self-reported their gender, age, education, and race, we examine performance differences of pretrained language models across demographic groups, defined by these (protected) attributes.
no code implementations • EMNLP 2021 • Jonathan Christiansen, Mathias Gammelgaard, Anders Søgaard
Sentiment analysis systems have been shown to exhibit sensitivity to protected attributes.
no code implementations • 23 Apr 2024 • Constanza Fierro, Jiaang Li, Anders Søgaard
The purpose of instruction tuning is enabling zero-shot performance, but instruction tuning has also been shown to improve chain-of-thought reasoning and value alignment (Si et al., 2023).
1 code implementation • 3 Apr 2024 • Constanza Fierro, Nicolas Garneau, Emanuele Bugliarello, Yova Kementchedjhieva, Anders Søgaard
Facts are subject to contingencies and can be true or false in different circumstances.
no code implementations • 22 Mar 2024 • Kun Sun, Rong Wang, Haitao Liu, Anders Søgaard
Evaluations have revealed that factors such as scaling, training types, architectures and other factors profoundly impact the performance of LLMs.
no code implementations • 1 Mar 2024 • Qinghua Zhao, Vinit Ravishankar, Nicolas Garneau, Anders Søgaard
Word order is an important concept in natural language, and in this work, we study how word order affects the induction of world knowledge from raw text using language models.
1 code implementation • 29 Feb 2024 • Stephanie Brandl, Oliver Eberle, Tiago Ribeiro, Anders Søgaard, Nora Hollenstein
Rationales in the form of manually annotated input spans usually serve as ground truth when evaluating explainability methods in NLP.
1 code implementation • 30 Oct 2023 • Heather Lent, Kushal Tatariya, Raj Dabre, Yiyi Chen, Marcell Fekete, Esther Ploeger, Li Zhou, Ruth-Ann Armstrong, Abee Eijansantos, Catriona Malau, Hans Erik Heje, Ernests Lavrinovics, Diptesh Kanojia, Paul Belony, Marcel Bollmann, Loïc Grobol, Miryam de Lhoneux, Daniel Hershcovich, Michel DeGraff, Anders Søgaard, Johannes Bjerva
Creoles represent an under-explored and marginalized group of languages, with few available resources for NLP research. While the genealogical ties between Creoles and a number of highly-resourced languages imply a significant potential for transfer learning, this potential is hampered due to this lack of annotated data.
1 code implementation • 20 Oct 2023 • Antonia Karamolegkou, Jiaang Li, Li Zhou, Anders Søgaard
Language models may memorize more than just facts, including entire chunks of texts seen during training.
no code implementations • 29 Aug 2023 • Mathias Lykke Gammelgaard, Jonathan Gabel Christiansen, Anders Søgaard
Large language models show human-like performance in knowledge extraction, reasoning and dialogue, but it remains controversial whether this performance is best explained by memorization and pattern matching, or whether it reflects human-like inferential semantics and world knowledge.
no code implementations • 17 Aug 2023 • Phillip Rust, Anders Søgaard
Language models such as mBERT, XLM-R, and BLOOM aim to achieve multilingual generalization or compression to facilitate transfer to a large number of (potentially unseen) languages.
no code implementations • 8 Jun 2023 • Antonia Karamolegkou, Mostafa Abdou, Anders Søgaard
Over the years, many researchers have seemingly made the same observation: Brain and language model activations exhibit some structural similarities, enabling linear partial mappings between features extracted from neural recordings and computational language models.
1 code implementation • 2 Jun 2023 • Jiaang Li, Antonia Karamolegkou, Yova Kementchedjhieva, Mostafa Abdou, Sune Lehmann, Anders Søgaard
Human language processing is also opaque, but neural response measurements can provide (noisy) recordings of activation during listening or reading, from which we can extract similar representations of words and phrases.
1 code implementation • 1 Jun 2023 • Terne Sasha Thorn Jakobsen, Laura Cabello, Anders Søgaard
Explainability methods are used to benchmark the extent to which model predictions align with human rationales i. e., are 'right for the right reasons'.
no code implementations • 31 May 2023 • Ruixiang Cui, Seolhwa Lee, Daniel Hershcovich, Anders Søgaard
Humans can effortlessly understand the coordinate structure of sentences such as "Niels Bohr and Kurt Cobain were born in Copenhagen and Seattle, respectively".
1 code implementation • 25 May 2023 • Seolhwa Lee, Anders Søgaard
Meeting summarization has an enormous business potential, but in addition to being a hard problem, roll-out is challenged by privacy concerns.
1 code implementation • 12 May 2023 • Ilias Chalkidis, Nicolas Garneau, Catalina Goanta, Daniel Martin Katz, Anders Søgaard
To this end, we release a multinational English legal corpus (LeXFiles) and a legal knowledge probing benchmark (LegalLAMA) to facilitate training and detailed analysis of legal-oriented PLMs.
no code implementations • 20 Apr 2023 • Laura Cabello, Anna Katrine Jørgensen, Anders Søgaard
To this end, we first provide a thought experiment, showing how association bias and empirical fairness can be completely orthogonal.
1 code implementation • 31 Mar 2023 • Tiago Ribeiro, Stephanie Brandl, Anders Søgaard, Nora Hollenstein
We present WebQAmGaze, a multilingual low-cost eye-tracking-while-reading dataset, designed as the first webcam-based eye-tracking corpus of reading to support the development of explainable computational language processing models.
no code implementations • 20 Feb 2023 • Anders Søgaard, Daniel Hershcovich, Miryam de Lhoneux
Van Miltenburg et al. (2021) suggest NLP research should adopt preregistration to prevent fishing expeditions and to promote publication of negative results.
no code implementations • 13 Feb 2023 • Jiaang Li, Yova Kementchedjhieva, Anders Søgaard
Large-scale pretrained language models (LMs) are said to ``lack the ability to connect [their] utterances to the world'' (Bender and Koller, 2020).
1 code implementation • 19 Dec 2022 • Lester James Miranda, Ákos Kádár, Adriane Boyd, Sofie Van Landeghem, Anders Søgaard, Matthew Honnibal
In this technical report we lay out a bit of history and introduce the embedding methods in spaCy in detail.
1 code implementation • COLING 2022 • Laura Cabello Piqueras, Anders Søgaard
Pretrained multilingual language models can help bridge the digital language divide, enabling high-quality NLP models for lower resourced languages.
1 code implementation • Findings (ACL) 2022 • Sebastian Ruder, Ivan Vulić, Anders Søgaard
Most work targeting multilinguality, for example, considers only accuracy; most work on fairness or interpretability considers only English; and so on.
no code implementations • insights (ACL) 2022 • Heather Lent, Emanuele Bugliarello, Anders Søgaard
We aim to learn language models for Creole languages for which large volumes of data are not readily available, and therefore explore the potential transfer from ancestor languages (the 'Ancestry Transfer Hypothesis').
no code implementations • LREC 2022 • Heather Lent, Kelechi Ogueji, Miryam de Lhoneux, Orevaoghene Ahia, Anders Søgaard
We demonstrate, through conversations with Creole experts and surveys of Creole-speaking communities, how the things needed from language technology can change dramatically from one language to another, even when the languages are considered to be very similar to each other, as with Creoles.
1 code implementation • Findings (NAACL) 2022 • Zechen Li, Anders Søgaard
Synthetic datasets have successfully been used to probe visual question-answering datasets for their reasoning abilities.
Ranked #1 on Visual Question Answering (VQA) on QLEVR
1 code implementation • 3 May 2022 • Stephanie Brandl, Daniel Hershcovich, Anders Søgaard
We argue that we need to evaluate model interpretability methods 'in the wild', i. e., in situations where professionals make critical decisions, and models can potentially assist them.
1 code implementation • ACL 2022 • Stephanie Brandl, Oliver Eberle, Jonas Pilot, Anders Søgaard
We investigate whether self-attention in large-scale pre-trained language models is as predictive of human eye fixation patterns during task-reading as classical cognitive models of human attention.
1 code implementation • NAACL (DADC) 2022 • Ruixiang Cui, Daniel Hershcovich, Anders Søgaard
Logical approaches to representing language have developed and evaluated computational models of quantifier words since the 19th century, but today's NLU models still struggle to capture their semantics.
1 code implementation • NAACL 2022 • Stephanie Brandl, Ruixiang Cui, Anders Søgaard
Gender-neutral pronouns have recently been introduced in many languages to a) include non-binary people and b) as a generic singular.
1 code implementation • Findings (ACL) 2022 • Constanza Fierro, Anders Søgaard
However, for that, we need to know how reliable this knowledge is, and recent work has shown that monolingual English language models lack consistency when predicting factual knowledge, that is, they fill-in-the-blank differently for paraphrases describing the same fact.
no code implementations • 21 Mar 2022 • Vinit Ravishankar, Mostafa Abdou, Artur Kulmizev, Anders Søgaard
Recent studies have shown that language models pretrained and/or fine-tuned on randomly permuted sentences exhibit competitive performance on GLUE, putting into question the importance of word order information.
no code implementations • ACL 2022 • Daniel Hershcovich, Stella Frank, Heather Lent, Miryam de Lhoneux, Mostafa Abdou, Stephanie Brandl, Emanuele Bugliarello, Laura Cabello Piqueras, Ilias Chalkidis, Ruixiang Cui, Constanza Fierro, Katerina Margatina, Phillip Rust, Anders Søgaard
Various efforts in the Natural Language Processing (NLP) community have been made to accommodate linguistic diversity and serve speakers of many different languages.
1 code implementation • ACL 2022 • Miryam de Lhoneux, Sheng Zhang, Anders Søgaard
Large multilingual pretrained language models such as mBERT and XLM-RoBERTa have been found to be surprisingly effective for cross-lingual transfer of syntactic parsing models (Wu and Dredze 2019), but only between related languages.
1 code implementation • Findings (ACL) 2022 • Ilias Chalkidis, Anders Søgaard
In document classification for, e. g., legal and biomedical text, we often deal with hundreds of classes, including very infrequent ones, as well as temporal concept drift caused by the influence of real world events, e. g., policy changes, conflicts, or pandemics.
1 code implementation • ACL 2022 • Ilias Chalkidis, Tommaso Pasini, Sheng Zhang, Letizia Tomada, Sebastian Felix Schwemer, Anders Søgaard
We present a benchmark suite of four datasets for evaluating the fairness of pre-trained language models and the techniques used to fine-tune them for downstream tasks.
1 code implementation • NAACL (PrivateNLP) 2022 • Victor Petrén Bach Hansen, Atula Tejaswi Neerkaje, Ramit Sawhney, Lucie Flek, Anders Søgaard
The performance cost of differential privacy has, for some applications, been shown to be higher for minority groups; fairness, conversely, has been shown to disproportionally compromise the privacy of members of such groups.
no code implementations • 24 Feb 2022 • Frederik Noe, Rasmus Herskind, Anders Søgaard
We establish a negative, logarithmic correlation between privacy and fairness in the case of linear classification and robust deep learning.
no code implementations • 29 Nov 2021 • Lasse Borgholt, Jakob Drachmann Havtorn, Mostafa Abdou, Joakim Edin, Lars Maaløe, Anders Søgaard, Christian Igel
We compare learned speech features from wav2vec 2. 0, state-of-the-art ASR transcripts, and the ground truth text as input for a novel speech-based named entity recognition task, a cardiac arrest detection task on real-world emergency calls and two existing SLU benchmarks.
Ranked #7 on Spoken Language Understanding on Fluent Speech Commands (using extra training data)
Automatic Speech Recognition Automatic Speech Recognition (ASR) +8
no code implementations • 8 Nov 2021 • Karthikeyan K, Anders Søgaard
Several instance-based explainability methods for finding influential training examples for test-time decisions have been proposed recently, including Influence Functions, TraceIn, Representer Point Selection, Grad-Dot, and Grad-Cos.
no code implementations • EMNLP 2021 • Yova Kementchedjhieva, Anders Søgaard
This approach shows mixed results: in a high-quality data setting, a longer average forecast horizon can be achieved at the cost of a small drop in F1; in a low-quality data setting, however, dynamic training propagates the noise and is highly detrimental to performance.
no code implementations • EMNLP (newsum) 2021 • Anna Jørgensen, Anders Søgaard
Summarization systems are ultimately evaluated by human annotators and raters.
no code implementations • EMNLP (BlackboxNLP) 2021 • Bastien Liétard, Mostafa Abdou, Anders Søgaard
The global geometry of language models is important for a range of applications, but language model probes tend to evaluate rather local relations, for which ground truths are easily obtained.
1 code implementation • CoNLL (EMNLP) 2021 • Heather Lent, Emanuele Bugliarello, Miryam de Lhoneux, Chen Qiu, Anders Søgaard
Creole languages such as Nigerian Pidgin English and Haitian Creole are under-resourced and largely ignored in the NLP literature.
no code implementations • CoNLL (EMNLP) 2021 • Mostafa Abdou, Artur Kulmizev, Daniel Hershcovich, Stella Frank, Ellie Pavlick, Anders Søgaard
Pretrained language models have been shown to encode relational information, such as the relations between entities or concepts in knowledge-bases -- (Paris, Capital, France).
no code implementations • EMNLP 2021 • Vinit Ravishankar, Anders Søgaard
In order to preserve word-order information in a non-autoregressive setting, transformer architectures tend to include positional knowledge, by (for instance) adding positional encodings to token embeddings.
no code implementations • Findings (ACL) 2021 • Ana Valeria Gonzalez, Anna Rogers, Anders Søgaard
A myriad of explainability methods have been proposed in recent years, but there is little consensus on how to evaluate them.
no code implementations • ACL (WAT) 2021 • Rahul Aralikatte, Miryam de Lhoneux, Anoop Kunchukuttan, Anders Søgaard
This work introduces Itihasa, a large-scale translation dataset containing 93, 000 pairs of Sanskrit shlokas and their English translations.
Ranked #1 on Machine Translation on Itihasa
no code implementations • 2 Jun 2021 • Yova Kementchedjhieva, Mark Anderson, Anders Søgaard
We hypothesize that the temporary challenge humans face in integrating the two contradicting signals, one from the lexical semantics of the verb, one from the sentence-level semantics, would be reflected in higher error rates for models on tasks dependent on causal links.
1 code implementation • 2 Jun 2021 • Edoardo Maria Ponti, Rahul Aralikatte, Disha Shrivastava, Siva Reddy, Anders Søgaard
In fact, under a decision-theoretic framework, MAML can be interpreted as minimising the expected risk across training languages (with a uniform prior), which is known as Bayes criterion.
no code implementations • 1 Jun 2021 • Mark Anderson, Anders Søgaard, Carlos Gómez Rodríguez
S{\o}gaard (2020) obtained results suggesting the fraction of trees occurring in the test data isomorphic to trees in the training set accounts for a non-trivial variation in parser performance.
no code implementations • 17 Feb 2021 • Lasse Borgholt, Jakob Drachmann Havtorn, Željko Agić, Anders Søgaard, Lars Maaløe, Christian Igel
We test this hypothesis by measuring temporal context sensitivity and evaluate how the models perform when we constrain the amount of contextual information in the audio input.
no code implementations • 29 Jan 2021 • Mostafa Abdou, Ana Valeria Gonzalez, Mariya Toneva, Daniel Hershcovich, Anders Søgaard
We evaluate across two fMRI datasets whether language models align better with brain recordings, if their attention is biased by annotations from syntactic or semantic formalisms.
no code implementations • EACL 2021 • Vinit Ravishankar, Artur Kulmizev, Mostafa Abdou, Anders Søgaard, Joakim Nivre
Since the popularization of the Transformer as a general-purpose feature encoder for NLP, many studies have attempted to decode linguistic structure from its novel multi-head attention mechanism.
no code implementations • EMNLP 2020 • Simon Flachs, Ophélie Lacroix, Helen Yannakoudakis, Marek Rei, Anders Søgaard
Evaluation of grammatical error correction (GEC) systems has primarily focused on essays written by non-native learners of English, which however is only part of the full spectrum of GEC applications.
1 code implementation • 12 Oct 2020 • Rahul Aralikatte, Mostafa Abdou, Heather Lent, Daniel Hershcovich, Anders Søgaard
Coreference resolution and semantic role labeling are NLP tasks that capture different aspects of semantics, indicating respectively, which expressions refer to the same entity, and what semantic roles expressions serve in the sentence.
2 code implementations • EMNLP 2020 • Ana Valeria Gonzalez, Maria Barrett, Rasmus Hvingelby, Kellie Webster, Anders Søgaard
The one-sided focus on English in previous studies of gender bias in NLP misses out on opportunities in other languages: English challenge datasets such as GAP and WinoGender highlight model preferences that are "hallucinatory", e. g., disambiguating gender-ambiguous occurrences of 'doctor' as male doctors.
1 code implementation • 23 Sep 2020 • Sheng Zhang, Xin Zhang, Weiming Zhang, Anders Søgaard
Multi-task transfer learning based on pre-trained language encoders achieves state-of-the-art performance across a range of tasks.
2 code implementations • ACL 2020 • Mostafa Abdou, Vinit Ravishankar, Maria Barrett, Yonatan Belinkov, Desmond Elliott, Anders Søgaard
Large-scale pretrained language models are the major driving force behind recent improvements in performance on the Winograd Schema Challenge, a widely employed test of common sense reasoning ability.
1 code implementation • EACL 2021 • Anders Søgaard, Sebastian Ebert, Jasmijn Bastings, Katja Filippova
We argue that random splits, like standard splits, lead to overly optimistic performance estimates.
no code implementations • 28 Apr 2020 • Katharina Kann, Ophélie Lacroix, Anders Søgaard
Part-of-speech (POS) taggers for low-resource languages which are exclusively based on various forms of weak supervision - e. g., cross-lingual transfer, type-level supervision, or a combination thereof - have been reported to perform almost as well as supervised ones.
1 code implementation • EMNLP 2020 • Ivan Vulić, Sebastian Ruder, Anders Søgaard
Existing algorithms for aligning cross-lingual word vector spaces assume that vector spaces are approximately isomorphic.
2 code implementations • 5 Feb 2020 • David Vilares, Michalina Strzyz, Anders Søgaard, Carlos Gómez-Rodríguez
We first cast constituent and dependency parsing as sequence tagging.
no code implementations • NeurIPS 2019 • Mareike Hartmann, Yova Kementchedjhieva, Anders Søgaard
Cross-lingual word vector space alignment is the task of mapping the vocabularies of two languages into a shared semantic space, which can be used for dictionary induction, unsupervised machine translation, and transfer learning.
1 code implementation • 21 Nov 2019 • Victor Petrén Bach Hansen, Anders Søgaard
In conversation, we often ask one-word questions such as `Why?'
no code implementations • 30 Sep 2019 • Ana Valeria Gonzalez, Isabelle Augenstein, Anders Søgaard
Most research on dialogue has focused either on dialogue generation for openended chit chat or on state tracking for goal-directed dialogue.
1 code implementation • 16 Sep 2019 • Joachim Bingel, Victor Petrén Bach Hansen, Ana Valeria Gonzalez, Paweł Budzianowski, Isabelle Augenstein, Anders Søgaard
Task oriented dialogue systems rely heavily on specialized dialogue state tracking (DST) modules for dynamically predicting user intent throughout the conversation.
2 code implementations • IJCNLP 2019 • Yova Kementchedjhieva, Mareike Hartmann, Anders Søgaard
We study the composition and quality of the test sets for five diverse languages from this dataset, with concerning findings: (1) a quarter of the data consists of proper nouns, which can be hardly indicative of BDI performance, and (2) there are pervasive gaps in the gold-standard targets.
no code implementations • IJCNLP 2019 • Clara Vania, Yova Kementchedjhieva, Anders Søgaard, Adam Lopez
Parsers are available for only a handful of the world's languages, since they require lots of training data.
1 code implementation • IJCNLP 2019 • Rahul Aralikatte, Heather Lent, Ana Valeria Gonzalez, Daniel Hershcovich, Chen Qiu, Anders Sandholm, Michael Ringaard, Anders Søgaard
Unresolved coreference is a bottleneck for relation extraction, and high-quality coreference resolvers may produce an output that makes it a lot easier to extract knowledge triples.
no code implementations • IJCNLP 2019 • Mostafa Abdou, Artur Kulmizev, Felix Hill, Daniel M. Low, Anders Søgaard
Representational Similarity Analysis (RSA) is a technique developed by neuroscientists for comparing activity patterns of different measurement modalities (e. g., fMRI, electrophysiology, behavior).
1 code implementation • EACL 2021 • Rahul Aralikatte, Matthew Lamm, Daniel Hardt, Anders Søgaard
Most, if not all forms of ellipsis (e. g., so does Mary) are similar to reading comprehension questions (what does Mary do), in that in order to resolve them, we need to identify an appropriate text span in the preceding discourse.
1 code implementation • WS 2019 • Mostafa Abdou, Cezar Sas, Rahul Aralikatte, Isabelle Augenstein, Anders Søgaard
Although the vast majority of knowledge bases KBs are heavily biased towards English, Wikipedias do cover very different topics in different languages.
1 code implementation • LREC 2020 • Rahul Aralikatte, Anders Søgaard
Humans do not make inferences over texts, but over models of what texts are about.
1 code implementation • ACL 2019 • Shuhei Kurita, Anders Søgaard
In Semantic Dependency Parsing (SDP), semantic relations form directed acyclic graphs, rather than trees.
no code implementations • NAACL 2019 • Mareike Hartmann, Tallulah Jansen, Isabelle Augenstein, Anders Søgaard
In online discussion fora, speakers often make arguments for or against something, say birth control, by highlighting certain aspects of the topic.
no code implementations • WS 2019 • Marcel Bollmann, Natalia Korchagina, Anders Søgaard
Historical text normalization often relies on small training datasets.
2 code implementations • NAACL 2019 • David Vilares, Mostafa Abdou, Anders Søgaard
Combining these techniques, we clearly surpass the performance of sequence tagging constituent parsers on the English and Chinese Penn Treebanks, and reduce their parsing time even further.
2 code implementations • 14 Nov 2018 • Marek Rei, Anders Søgaard
Learning to construct text representations in end-to-end systems can be difficult, as natural languages are highly compositional and task-specific annotated datasets are often limited in size.
Ranked #1 on Grammatical Error Detection on JFLEG
no code implementations • EMNLP 2018 • Mareike Hartmann, Yova Kementchedjhieva, Anders Søgaard
This paper presents a challenge to the community: Generative adversarial networks (GANs) can perfectly align independent English word embeddings induced using the same algorithm, based on distributional information alone; but fails to do so, for two different embeddings algorithms.
no code implementations • WS 2018 • Anders Søgaard, Miryam de Lhoneux, Isabelle Augenstein
Punctuation is a strong indicator of syntactic structure, and parsers trained on text with punctuation often rely heavily on this signal.
1 code implementation • CONLL 2018 • Yova Kementchedjhieva, Sebastian Ruder, Ryan Cotterell, Anders Søgaard
Most recent approaches to bilingual dictionary induction find a linear alignment between the word vector spaces of two languages.
1 code implementation • EMNLP 2018 • Sebastian Ruder, Ryan Cotterell, Yova Kementchedjhieva, Anders Søgaard
We introduce a novel discriminative latent variable model for bilingual lexicon induction.
1 code implementation • EMNLP 2018 • Miryam de Lhoneux, Johannes Bjerva, Isabelle Augenstein, Anders Søgaard
We find that sharing transition classifier parameters always helps, whereas the usefulness of sharing word and/or character LSTM parameters varies.
no code implementations • EMNLP 2018 • Ana V. González-Garduño, Isabelle Augenstein, Anders Søgaard
The best systems at the SemEval-16 and SemEval-17 community question answering shared tasks -- a task that amounts to question relevancy ranking -- involve complex pipelines and manual feature engineering.
no code implementations • ACL 2018 • Anders Søgaard, Sebastian Ruder, Ivan Vulić
Unsupervised machine translation---i. e., not assuming any cross-lingual supervision signal, whether a dictionary, translations, or comparable corpora---seems impossible, but nevertheless, Lample et al. (2018) recently proposed a fully unsupervised machine translation (MT) model.
no code implementations • NAACL 2018 • Marek Rei, Anders Søgaard
Can attention- or gradient-based visualization techniques be used to infer token-level labels for binary sequence tagging problems, using networks trained only on sentence-level labels?
1 code implementation • NAACL 2018 • Isabelle Augenstein, Sebastian Ruder, Anders Søgaard
We combine multi-task learning and semi-supervised learning by inducing a joint embedding space between disparate label spaces and learning transfer functions between label embeddings, enabling us to jointly leverage unlabelled data and auxiliary, annotated datasets.
7 code implementations • EMNLP 2017 • Bjarke Felbo, Alan Mislove, Anders Søgaard, Iyad Rahwan, Sune Lehmann
NLP tasks are often limited by scarcity of manually annotated data.
Ranked #1 on Transfer Learning on Amazon Review Polarity
no code implementations • 13 Jul 2017 • Chloé Braud, Anders Søgaard
The problem of detecting scientific fraud using machine learning was recently introduced, with initial, positive results from a model taking into account various general indicators.
no code implementations • 15 Jun 2017 • Sebastian Ruder, Ivan Vulić, Anders Søgaard
Cross-lingual representations of words enable us to reason about word meaning in multilingual contexts and are a key facilitator of cross-lingual transfer when developing natural language processing models for low-resource languages.
2 code implementations • 23 May 2017 • Sebastian Ruder, Joachim Bingel, Isabelle Augenstein, Anders Søgaard
In practice, however, MTL involves searching an enormous space of possible parameter sharing architectures to find (a) the layers or subspaces that benefit from sharing, (b) the appropriate amount of sharing, and (c) the appropriate relative weights of the different task losses.
1 code implementation • 13 Apr 2017 • Chloé Braud, Ophélie Lacroix, Anders Søgaard
Discourse segmentation is a crucial step in building end-to-end discourse parsers.
no code implementations • ACL 2017 • Isabelle Augenstein, Anders Søgaard
Keyphrase boundary classification (KBC) is the task of detecting keyphrases in scientific articles and labelling them with respect to predefined types.
1 code implementation • EACL 2017 • Joachim Bingel, Anders Søgaard
Multi-task learning (MTL) in deep neural networks for NLP has recently received increasing interest due to some compelling benefits, including its potential to efficiently regularize models and to reduce the need for labeled data.
1 code implementation • EACL 2017 • Chloé Braud, Maximin Coavoux, Anders Søgaard
Discourse parsing is an integral part of understanding information flow and argumentative structure in documents.
Ranked #10 on Discourse Parsing on RST-DT
1 code implementation • EACL 2017 • Héctor Martínez Alonso, Željko Agić, Barbara Plank, Anders Søgaard
We propose UDP, the first training-free parser for Universal Dependencies (UD).
1 code implementation • 6 Jan 2017 • Michael Sejr Schlichtkrull, Anders Søgaard
In cross-lingual dependency annotation projection, information is often lost during transfer because of early decoding.
no code implementations • 18 Nov 2016 • Anders Søgaard
We present a confidence-based single-layer feed-forward learning algorithm SPIRAL (Spike Regularized Adaptive Learning) relying on an encoding of activation spikes.
no code implementations • COLING 2016 • Marcel Bollmann, Anders Søgaard
Natural-language processing of historical documents is complicated by the abundance of variant spellings and lack of annotated data.
no code implementations • EACL 2017 • Omer Levy, Anders Søgaard, Yoav Goldberg
While cross-lingual word embeddings have been studied extensively in recent years, the qualitative differences between the different algorithms remain vague.
3 code implementations • ACL 2016 • Barbara Plank, Anders Søgaard, Yoav Goldberg
Bidirectional long short-term memory (bi-LSTM) networks have recently proven successful for various NLP sequence modeling tasks, but little is known about their reliance to input representations, target languages, data set size, and label noise.
Ranked #4 on Part-Of-Speech Tagging on UD
no code implementations • NAACL 2016 • Sigrid Klerke, Yoav Goldberg, Anders Søgaard
We show how eye-tracking corpora can be used to improve sentence compression models, presenting a novel multi-task learning algorithm based on multi-layer LSTMs.
Ranked #5 on Sentence Compression on Google Dataset
no code implementations • 9 Jan 2016 • Anders Søgaard
We instead propose to use the $k$ source language models to estimate the parameters of a Gaussian prior for learning new POS taggers.