no code implementations • 2 Dec 2023 • Corby Rosset, Guoqing Zheng, Victor Dibia, Ahmed Awadallah, Paul Bennett
The remarkable abilities of large language models (LLMs) like GPT-4 partially stem from post-training processes like Reinforcement Learning from Human Feedback (RLHF) involving human preferences encoded in a reward model.
no code implementations • 18 Nov 2023 • Arindam Mitra, Luciano del Corro, Shweti Mahajan, Andres Codas, Clarisse Simoes, Sahaj Agarwal, Xuxi Chen, Anastasia Razdaibiedina, Erik Jones, Kriti Aggarwal, Hamid Palangi, Guoqing Zheng, Corby Rosset, Hamed Khanpour, Ahmed Awadallah
Research on training small LMs has often relied on imitation learning to replicate the output of more capable models.
Ranked #1 on Crass AI on BIG-bench
no code implementations • 4 Oct 2023 • Chen Dun, Mirian Hipolito Garcia, Guoqing Zheng, Ahmed Hassan Awadallah, Anastasios Kyrillidis, Robert Sim
Large Language Models (LLMs) have the ability to solve a variety of tasks, such as text summarization and mathematical questions, just out of the box, but they are often trained with a single task in mind.
no code implementations • 8 Aug 2023 • Menglin Xia, Xuchao Zhang, Camille Couturier, Guoqing Zheng, Saravan Rajmohan, Victor Ruhle
Retrieval augmentation enhances performance of traditional language models by incorporating additional context.
no code implementations • 14 Jun 2023 • Chen Dun, Mirian Hipolito Garcia, Guoqing Zheng, Ahmed Hassan Awadallah, Robert Sim, Anastasios Kyrillidis, Dimitrios Dimitriadis
Our gating function harnesses the knowledge of a pretrained model common expert to enhance its routing decisions on-the-fly.
no code implementations • 20 Oct 2022 • Budhaditya Deb, Guoqing Zheng, Ahmed Hassan Awadallah
Recent work has shown that language models (LMs) trained with multi-task \textit{instructional learning} (MTIL) can solve diverse NLP tasks in zero- and few-shot settings with improved performance compared to prompt tuning.
no code implementations • 26 Aug 2022 • Kaize Ding, Elnaz Nouri, Guoqing Zheng, Huan Liu, Ryen White
The success of graph neural networks on graph-based web mining highly relies on abundant human-annotated data, which is laborious to obtain in practice.
1 code implementation • 24 Aug 2022 • Yue Zhao, Guoqing Zheng, Subhabrata Mukherjee, Robert McCann, Ahmed Awadallah
In this work, we propose a method to leverage weak/noisy labels (e. g., risk scores generated by machine rules for detecting malware) that are cheaper to obtain for anomaly detection.
no code implementations • insights (ACL) 2022 • Hanjie Chen, Guoqing Zheng, Ahmed Hassan Awadallah, Yangfeng Ji
Although adapting pre-trained language models with few examples has shown promising performance on text classification, there is a lack of understanding of where the performance gain comes from.
1 code implementation • ICLR 2022 • Ruibo Liu, Guoqing Zheng, Shashank Gupta, Radhika Gaonkar, Chongyang Gao, Soroush Vosoughi, Milad Shokouhi, Ahmed Hassan Awadallah
Hence, they tend to suffer from counterfactual or hallucinatory generation when used in knowledge-intensive natural language generation (NLG) tasks.
Ranked #2 on Question Answering on KILT: ELI5
1 code implementation • 4 Nov 2021 • Subhabrata Mukherjee, Xiaodong Liu, Guoqing Zheng, Saghar Hosseini, Hao Cheng, Greg Yang, Christopher Meek, Ahmed Hassan Awadallah, Jianfeng Gao
We demonstrate that while recent models reach human performance when they have access to large amounts of labeled data, there is a huge gap in performance in the few-shot setting for most tasks.
no code implementations • Findings (EMNLP) 2021 • Budhaditya Deb, Guoqing Zheng, Milad Shokouhi, Ahmed Hassan Awadallah
We study the problem of multilingual automated reply suggestions (RS) model serving many languages simultaneously.
no code implementations • 9 Sep 2021 • Srinagesh Sharma, Guoqing Zheng, Ahmed Hassan Awadallah
In this paper, we aim to the address of the problem of few shot task learning by exploiting and transferring from a different task which admits a related but disparate label space.
no code implementations • NAACL 2022 • Guoqing Zheng, Giannis Karamanolakis, Kai Shu, Ahmed Hassan Awadallah
In this paper, we propose such a benchmark, named WALNUT (semi-WeAkly supervised Learning for Natural language Understanding Testbed), to advocate and facilitate research on weak supervision for NLU.
1 code implementation • ACL 2021 • Mozhi Zhang, Wei Wang, Budhaditya Deb, Guoqing Zheng, Milad Shokouhi, Ahmed Hassan Awadallah
Reply suggestion models help users process emails and chats faster.
2 code implementations • NAACL 2021 • Mengzhou Xia, Guoqing Zheng, Subhabrata Mukherjee, Milad Shokouhi, Graham Neubig, Ahmed Hassan Awadallah
Extensive experiments on real-world low-resource languages - without access to large-scale monolingual corpora or large amounts of labeled data - for tasks like cross-lingual sentiment analysis and named entity recognition show the effectiveness of our approach.
1 code implementation • NAACL 2021 • Giannis Karamanolakis, Subhabrata Mukherjee, Guoqing Zheng, Ahmed Hassan Awadallah
In this work, we develop a weak supervision framework (ASTRA) that leverages all the available data for a given task.
no code implementations • 26 May 2020 • Kai Shu, Subhabrata Mukherjee, Guoqing Zheng, Ahmed Hassan Awadallah, Milad Shokouhi, Susan Dumais
In this paper, we propose to leverage user actions as a source of weak supervision, in addition to a limited set of annotated examples, to detect intents in emails.
no code implementations • 3 Apr 2020 • Kai Shu, Guoqing Zheng, Yichuan Li, Subhabrata Mukherjee, Ahmed Hassan Awadallah, Scott Ruston, Huan Liu
Social media has greatly enabled people to participate in online activities at an unprecedented rate.
1 code implementation • 10 Nov 2019 • Guoqing Zheng, Ahmed Hassan Awadallah, Susan Dumais
We view the label correction procedure as a meta-process and propose a new meta-learning based framework termed MLC (Meta Label Correction) for learning with noisy labels.
Ranked #9 on Image Classification on Clothing1M (using clean data) (using extra training data)
1 code implementation • 15 Jun 2018 • Guokun Lai, Bohan Li, Guoqing Zheng, Yiming Yang
In this paper, we combine the ideas from both stochastic latent variables and dilated convolutions, and propose a new architecture to model sequential data, termed as Stochastic WaveNet, where stochastic latent variables are injected into the WaveNet structure.
1 code implementation • 20 Nov 2017 • Guoqing Zheng, Yiming Yang, Jaime Carbonell
However, freely enriching the family of variational distribution is challenging since the ELBO requires variational likelihood evaluations of the latent variables.
1 code implementation • ICLR 2018 • Guoqing Zheng, Yiming Yang, Jaime Carbonell
Variational inference provides one way to approximate the posterior distribution, however its expressive power is limited and so is the accuracy of resulting approximation.