no code implementations • CCL 2020 • Zhengwei Lv, Lei Yang, Zhizhong Shi, Xiao Liang, Tao Lei, Duoxing Liu
阅读理解问答系统是利用语义理解等自然语言处理技术, 根据输入问题, 对非结构化文档数据进行分析, 生成一个答案, 具有很高的研究和应用价值。在垂直领域应用过程中, 阅读理解问答数据标注成本高且用户问题表达复杂多样, 使得阅读理解问答系统准确率低、鲁棒性差。针对这一问题, 本文提出一种面向垂直领域的阅读理解问答数据的增强方法, 该方法基于真实用户问题, 构造阅读理解训练数据, 一方面降低标注成本, 另一方面增加训练数据多样性, 提升模型的准确率和鲁棒性。本文用汽车领域数据对该方法进行实验验证, 其结果表明该方法对垂直领域阅读理解模型的准确率和鲁棒性均能有效提升。
no code implementations • 14 Mar 2024 • Brandon McKinzie, Zhe Gan, Jean-Philippe Fauconnier, Sam Dodge, BoWen Zhang, Philipp Dufter, Dhruti Shah, Xianzhi Du, Futang Peng, Floris Weers, Anton Belyi, Haotian Zhang, Karanjeet Singh, Doug Kang, Ankur Jain, Hongyu Hè, Max Schwarzer, Tom Gunter, Xiang Kong, Aonan Zhang, Jianyu Wang, Chong Wang, Nan Du, Tao Lei, Sam Wiseman, Guoli Yin, Mark Lee, ZiRui Wang, Ruoming Pang, Peter Grasch, Alexander Toshev, Yinfei Yang
Through careful and comprehensive ablations of the image encoder, the vision language connector, and various pre-training data choices, we identified several crucial design lessons.
Ranked #21 on Visual Question Answering on MM-Vet
no code implementations • 26 Nov 2023 • Dewen Zeng, Nan Du, Tao Wang, Yuanzhong Xu, Tao Lei, Zhifeng Chen, Claire Cui
Overparameterized large-scale language models have impressive generalization performance of in-context few-shot learning.
1 code implementation • 7 Jun 2023 • Rui Sun, Tao Lei, Weichuan Zhang, Yong Wan, Yong Xia, Asoke K. Nandi
The hybrid architecture of convolution neural networks (CNN) and Transformer has been the most popular method for medical image segmentation.
1 code implementation • 6 Jun 2023 • Tao Lei, Rui Sun, Xuan Wang, Yingbo Wang, Xi He, Asoke Nandi
The hybrid architecture of convolutional neural networks (CNNs) and Transformer are very popular for medical image segmentation.
no code implementations • 3 Jun 2023 • Tao Lei, Yetong Xu, Hailong Ning, Zhiyong Lv, Chongdan Min, Yaochu Jin, Asoke K. Nandi
Popular Transformer networks have been successfully applied to remote sensing (RS) image change detection (CD) identifications and achieve better results than most convolutional neural networks (CNNs), but they still suffer from two main problems.
2 code implementations • NeurIPS 2023 • Jinhyuk Lee, Zhuyun Dai, Sai Meher Karthik Duddu, Tao Lei, Iftekhar Naim, Ming-Wei Chang, Vincent Y. Zhao
Multi-vector retrieval models such as ColBERT [Khattab and Zaharia, 2020] allow token-level interactions between queries and documents, and hence achieve state of the art on many information retrieval benchmarks.
1 code implementation • IEEE Transactions on Geoscience and Remote Sensing 2023 • Tao Lei, Xinzhe Geng, Hailong Ning, Zhiyong Lv, Maoguo Gong, Yaochu Jin, Asoke K. Nandi
First, the existing multiscale feature fusion methods often use redundant feature extraction and fusion strategies, which often lead to high computational costs and memory usage.
Ranked #2 on Change Detection on DSIFN-CD
Building change detection for remote sensing images Change Detection +1
no code implementations • 17 Mar 2023 • Joshua Ainslie, Tao Lei, Michiel de Jong, Santiago Ontañón, Siddhartha Brahma, Yury Zemlyanskiy, David Uthus, Mandy Guo, James Lee-Thorp, Yi Tay, Yun-Hsuan Sung, Sumit Sanghai
Many natural language processing tasks benefit from long inputs, but processing long documents with Transformers is expensive -- not only due to quadratic attention complexity but also from applying feedforward and projection layers to every token.
Ranked #1 on Long-range modeling on SCROLLS
1 code implementation • 4 Dec 2022 • Shu Liu, Enquan Huang, Ziyu Zhou, Yan Xu, Xiaoyan Kui, Tao Lei, Hongying Meng
The data processing is simplified to a minimum for a lightweight design, and MobileNetV2 is selected as our backbone.
no code implementations • 2 Nov 2022 • Yujie Qian, Jinhyuk Lee, Sai Meher Karthik Duddu, Zhuyun Dai, Siddhartha Brahma, Iftekhar Naim, Tao Lei, Vincent Y. Zhao
With sparsified unary saliences, we are able to prune a large number of query and document token vectors and improve the efficiency of multi-vector retrieval.
1 code implementation • 25 May 2022 • Zexuan Zhong, Tao Lei, Danqi Chen
Recent work has improved language models (LMs) remarkably by equipping them with a non-parametric memory component.
no code implementations • 23 May 2022 • Tao Lei, Ran Tian, Jasmijn Bastings, Ankur P. Parikh
In this work, we explore whether modeling recurrence into the Transformer architecture can both be beneficial and efficient, by building an extremely simple recurrent module into the Transformer.
no code implementations • 18 Feb 2022 • Yanqi Zhou, Tao Lei, Hanxiao Liu, Nan Du, Yanping Huang, Vincent Zhao, Andrew Dai, Zhifeng Chen, Quoc Le, James Laudon
Prior work allocates a fixed number of experts to each token using a top-k function regardless of the relative importance of different tokens.
no code implementations • 11 Oct 2021 • Jing Pan, Tao Lei, Kwangyoun Kim, Kyu Han, Shinji Watanabe
The Transformer architecture has been well adopted as a dominant architecture in most sequence transduction tasks including automatic speech recognition (ASR), since its attention mechanism excels in capturing long-range dependencies.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +4
no code implementations • 17 Aug 2021 • Xianyuan Liu, Shuo Zhou, Tao Lei, Haiping Lu
Finally, we propose a Channel-Temporal Attention Network (CTAN) to integrate these blocks into existing architectures.
no code implementations • 22 Jun 2021 • Xianyuan Liu, Raivo Koot, Shuo Zhou, Tao Lei, Haiping Lu
Under the team name xy9, our submission achieved 5th place in terms of top-1 accuracy for verb class and all top-5 accuracies.
no code implementations • NAACL 2021 • Darsh Shah, Lili Yu, Tao Lei, Regina Barzilay
We present a method for generating comparative summaries that highlight similarities and contradictions in input documents.
2 code implementations • 8 Apr 2021 • Darsh J Shah, Lili Yu, Tao Lei, Regina Barzilay
We present a method for generating comparative summaries that highlights similarities and contradictions in input documents.
1 code implementation • 22 Mar 2021 • Darsh J Shah, Lili Yu, Tao Lei, Regina Barzilay
We introduce \emph{Nutri-bullets}, a multi-document summarization task for health and nutrition.
1 code implementation • EMNLP 2021 • Tao Lei
In this work, we present SRU++, a highly-efficient architecture that combines fast recurrence and attention for sequence modeling.
Ranked #4 on Language Modelling on enwik8
2 code implementations • 28 Sep 2020 • Risheng Wang, Tao Lei, Ruixia Cui, Bingtao Zhang, Hongy-ing Meng, Asoke K. Nandi
Firstly, compared to traditional surveys that directly divide literatures of deep learning on medical image segmentation into many groups and introduce literatures in detail for each group, we classify currently popular literatures according to a multi-level structure from coarse to fine.
2 code implementations • EMNLP 2020 • Alexander Lin, Jeremy Wohlwend, Howard Chen, Tao Lei
The performance of autoregressive models on natural language generation tasks has dramatically improved due to the adoption of deep, self-attentive architectures.
Ranked #21 on Machine Translation on IWSLT2014 German-English
1 code implementation • ACL 2020 • Kyle Swanson, Lili Yu, Tao Lei
Selecting input features of top relevance has become a popular method for building self-explaining models.
no code implementations • 21 May 2020 • Jing Pan, Joshua Shapiro, Jeremy Wohlwend, Kyu J. Han, Tao Lei, Tao Ma
In this paper we present state-of-the-art (SOTA) performance on the LibriSpeech corpus with two novel neural network architectures, a multistream CNN for acoustic modeling and a self-attentive simple recurrent unit (SRU) for language modeling.
Ranked #7 on Speech Recognition on LibriSpeech test-clean
1 code implementation • ACL 2020 • Lili Yu, Howard Chen, Sida Wang, Tao Lei, Yoav Artzi
We study the potential for interaction in natural language classification.
1 code implementation • WS 2019 • Jeremy Wohlwend, Ethan R. Elenberg, Samuel Altschul, Shawn Henry, Tao Lei
However, in many real-world applications the label set is frequently changing.
2 code implementations • EMNLP 2020 • Ziheng Wang, Jeremy Wohlwend, Tao Lei
Large language models have recently achieved state of the art performance across a wide variety of natural language tasks.
no code implementations • WS 2019 • Kyle Swanson, Lili Yu, Christopher Fox, Jeremy Wohlwend, Tao Lei
Response suggestion is an important task for building human-computer conversation systems.
no code implementations • SEMEVAL 2019 • Zhengwei Lv, Duoxing Liu, Haifeng Sun, Xiao Liang, Tao Lei, Zhizhong Shi, Feng Zhu, Lei Yang
In order to address this task, we propose a system based on the BERT model with meta information of questions.
1 code implementation • 8 Apr 2019 • Tao Lei, Xiaohong Jia, Tongliang Liu, Shigang Liu, Hongy-ing Meng, Asoke K. Nandi
However, MR might mistakenly filter meaningful seeds that are required for generating accurate segmentation and it is also sensitive to the scale because a single-scale structuring element is employed.
1 code implementation • EMNLP 2018 • Darsh J Shah, Tao Lei, Alessandro Moschitti, Salvatore Romeo, Preslav Nakov
We address the problem of detecting duplicate questions in forums, which is an important step towards automating the process of answering new questions.
no code implementations • IEEE 2018 • Tao Lei, Xiaohong Jia, Yanning Zhang, Lifeng He, Hongy-ing Meng, Senior Member, and Asoke K. Nandi, Fellow, IEEE
However, the introduction oflocal spatial information often leads to a high computationalcomplexity, arising out of an iterative calculation of the distancebetween pixels within local spatial neighbors and clusteringcenters.
1 code implementation • ICLR 2018 • Tao Lei, Yu Zhang, Yoav Artzi
Common recurrent neural network architectures scale poorly due to the intrinsic difficulty in parallelizing their state computations.
11 code implementations • EMNLP 2018 • Tao Lei, Yu Zhang, Sida I. Wang, Hui Dai, Yoav Artzi
Common recurrent neural architectures scale poorly due to the intrinsic difficulty in parallelizing their state computations.
Ranked #32 on Question Answering on SQuAD1.1 dev
12 code implementations • NeurIPS 2017 • Tianxiao Shen, Tao Lei, Regina Barzilay, Tommi Jaakkola
We demonstrate the effectiveness of this cross-alignment method on three tasks: sentiment modification, decipherment of word substitution ciphers, and recovery of word order.
Ranked #7 on Text Style Transfer on Yelp Review Dataset (Small)
no code implementations • ICML 2017 • Tao Lei, Wengong Jin, Regina Barzilay, Tommi Jaakkola
The design of neural architectures for structured objects is typically guided by experimental insights rather than a formal process.
3 code implementations • EMNLP 2016 • Tao Lei, Regina Barzilay, Tommi Jaakkola
Our approach combines two modular components, generator and encoder, which are trained to operate well together.
1 code implementation • NAACL 2016 • Tao Lei, Hrishikesh Joshi, Regina Barzilay, Tommi Jaakkola, Katerina Tymoshenko, Alessandro Moschitti, Lluis Marquez
Question answering forums are rapidly growing in size with no effective automated ability to refer to and reuse answers already available for previous posted questions.
2 code implementations • EMNLP 2015 • Tao Lei, Regina Barzilay, Tommi Jaakkola
Moreover, we extend the n-gram convolution to non-consecutive words to recognize patterns with intervening words.
no code implementations • TACL 2014 • Yonatan Belinkov, Tao Lei, Regina Barzilay, Amir Globerson
In this paper, we show that word vector representations can yield significant PP attachment performance gains.