no code implementations • 15 May 2024 • Zaitian Wang, Pengfei Wang, Kunpeng Liu, Pengyang Wang, Yanjie Fu, Chang-Tien Lu, Charu C. Aggarwal, Jian Pei, Yuanchun Zhou
Existing literature surveys only focus on a certain type of specific modality data, and categorize these methods from modality-specific and operation-centric perspectives, which lacks a consistent summary of data augmentation methods across multiple modalities and limits the comprehension of how existing data samples serve the data augmentation process.
no code implementations • 1 May 2024 • Lingze Zeng, Naili Xing, Shaofeng Cai, Gang Chen, Beng Chin Ooi, Jian Pei, Yuncheng Wu
This SQL-aware MoE technique scales up the modeling capacity, enhances effectiveness, and preserves efficiency by activating only necessary experts via the gating network during inference.
no code implementations • 15 Mar 2024 • Naili Xing, Shaofeng Cai, Zhaojing Luo, Beng Chin Ooi, Jian Pei
This transition demands an efficient and responsive anytime NAS approach that is capable of returning current optimal architectures within any given time budget while progressively enhancing architecture quality with increased budget allocation.
1 code implementation • 21 Feb 2024 • Jiahao Zhang, Rui Xue, Wenqi Fan, Xin Xu, Qing Li, Jian Pei, Xiaorui Liu
In this paper, we propose a Linear-Time Graph Neural Network (LTGNN) to scale up GNN-based recommender systems to achieve comparable scalability as classic MF approaches while maintaining GNNs' powerful expressiveness for superior prediction accuracy.
no code implementations • 26 Jan 2024 • Zicun Cong, Shi Baoxu, Shan Li, Jaewon Yang, Qi He, Jian Pei
To address the bias in node features and model parameters, FairSample is complemented by a regularization objective to optimize fairness.
1 code implementation • 10 Jan 2024 • Lichao Sun, Yue Huang, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, Xiner Li, Zhengliang Liu, Yixin Liu, Yijue Wang, Zhikun Zhang, Bertie Vidgen, Bhavya Kailkhura, Caiming Xiong, Chaowei Xiao, Chunyuan Li, Eric Xing, Furong Huang, Hao liu, Heng Ji, Hongyi Wang, huan zhang, Huaxiu Yao, Manolis Kellis, Marinka Zitnik, Meng Jiang, Mohit Bansal, James Zou, Jian Pei, Jian Liu, Jianfeng Gao, Jiawei Han, Jieyu Zhao, Jiliang Tang, Jindong Wang, Joaquin Vanschoren, John Mitchell, Kai Shu, Kaidi Xu, Kai-Wei Chang, Lifang He, Lifu Huang, Michael Backes, Neil Zhenqiang Gong, Philip S. Yu, Pin-Yu Chen, Quanquan Gu, ran Xu, Rex Ying, Shuiwang Ji, Suman Jana, Tianlong Chen, Tianming Liu, Tianyi Zhou, William Wang, Xiang Li, Xiangliang Zhang, Xiao Wang, Xing Xie, Xun Chen, Xuyu Wang, Yan Liu, Yanfang Ye, Yinzhi Cao, Yong Chen, Yue Zhao
This paper introduces TrustLLM, a comprehensive study of trustworthiness in LLMs, including principles for different dimensions of trustworthiness, established benchmark, evaluation, and analysis of trustworthiness for mainstream LLMs, and discussion of open challenges and future directions.
1 code implementation • 6 Nov 2023 • Zilin Xiao, Ming Gong, Jie Wu, Xingyao Zhang, Linjun Shou, Jian Pei, Daxin Jiang
Generative approaches powered by large language models (LLMs) have demonstrated emergent abilities in tasks that require complex reasoning abilities.
no code implementations • 6 Nov 2023 • Zilin Xiao, Linjun Shou, Xingyao Zhang, Jie Wu, Ming Gong, Jian Pei, Daxin Jiang
We propose CoherentED, an ED system equipped with novel designs aimed at enhancing the coherence of entity predictions.
no code implementations • 19 Sep 2023 • Ning Wu, Ming Gong, Linjun Shou, Jian Pei, Daxin Jiang
RUEL is the first method that connects user browsing data with typical recommendation datasets and can be generalized to various recommendation scenarios and datasets.
1 code implementation • 6 Aug 2023 • Xidong Wu, Zhengmian Hu, Jian Pei, Heng Huang
To address the above challenge, we study the serverless multi-party collaborative AUPRC maximization problem since serverless multi-party collaborative training can cut down the communications cost by avoiding the server node bottleneck, and reformulate it as a conditional stochastic optimization problem in a serverless multi-party collaborative learning setting and propose a new ServerLess biAsed sTochastic gradiEnt (SLATE) algorithm to directly optimize the AUPRC.
no code implementations • 30 May 2023 • Chen Ling, Xujiang Zhao, Jiaying Lu, Chengyuan Deng, Can Zheng, Junxiang Wang, Tanmoy Chowdhury, Yun Li, Hejie Cui, Xuchao Zhang, Tianjiao Zhao, Amit Panalkar, Dhagash Mehta, Stefano Pasquali, Wei Cheng, Haoyu Wang, Yanchi Liu, Zhengzhang Chen, Haifeng Chen, Chris White, Quanquan Gu, Jian Pei, Carl Yang, Liang Zhao
In this article, we present a comprehensive survey on domain specification techniques for large language models, an emerging direction critical for large language model applications.
1 code implementation • 9 May 2023 • Nuo Chen, Linjun Shou, Ming Gong, Jian Pei, Bowen Cao, Jianhui Chang, Daxin Jiang, Jia Li
Currently, learning better unsupervised sentence representations is the pursuit of many natural language processing communities.
1 code implementation • 17 Apr 2023 • Shengyao Zhuang, Linjun Shou, Jian Pei, Ming Gong, Houxing Ren, Guido Zuccon, Daxin Jiang
To address this challenge, we propose ToRoDer (TypOs-aware bottlenecked pre-training for RObust DEnse Retrieval), a novel re-training strategy for DRs that increases their robustness to misspelled queries while preserving their effectiveness in downstream retrieval tasks.
no code implementations • 27 Mar 2023 • Houxing Ren, Linjun Shou, Jian Pei, Ning Wu, Ming Gong, Daxin Jiang
In this paper, we propose to mine and generate self-supervised training data based on a large-scale unlabeled corpus.
1 code implementation • 14 Mar 2023 • Lianghao Xia, Yizhen Shao, Chao Huang, Yong Xu, Huance Xu, Jian Pei
In this work, we design a Disentangled Graph Neural Network (DGNN) with the integration of latent memory units, which empowers DGNN to maintain factorized representations for heterogeneous types of user and item connections.
no code implementations • 18 Feb 2023 • Ce Zhou, Qian Li, Chen Li, Jun Yu, Yixin Liu, Guangjing Wang, Kai Zhang, Cheng Ji, Qiben Yan, Lifang He, Hao Peng, JianXin Li, Jia Wu, Ziwei Liu, Pengtao Xie, Caiming Xiong, Jian Pei, Philip S. Yu, Lichao Sun
This study provides a comprehensive review of recent research advancements, challenges, and opportunities for PFMs in text, image, graph, as well as other data modalities.
no code implementations • 16 Feb 2023 • Nuo Chen, Linjun Shou, Ming Gong, Jian Pei, Chenyu You, Jianhui Chang, Daxin Jiang, Jia Li
For instance, TPLMs jointly pre-trained with table and text input could be effective for tasks also with table-text joint input like table question answering, but it may fail for tasks with only tables or text as input such as table retrieval.
no code implementations • 4 Feb 2023 • Tanmoy Chowdhury, Chen Ling, Xuchao Zhang, Xujiang Zhao, Guangji Bai, Jian Pei, Haifeng Chen, Liang Zhao
Knowledge-enhanced neural machine reasoning has garnered significant attention as a cutting-edge yet challenging research area with numerous practical applications.
1 code implementation • 3 Feb 2023 • Rui Xue, Haoyu Han, MohamadAli Torkamani, Jian Pei, Xiaorui Liu
Recent works have demonstrated the benefits of capturing long-distance dependency in graphs by deeper graph neural networks (GNNs).
1 code implementation • 5 Oct 2022 • Nian Liu, Xiao Wang, Deyu Bo, Chuan Shi, Jian Pei
Then we theoretically prove that GCL is able to learn the invariance information by contrastive invariance theorem, together with our GAME rule, for the first time, we uncover that the learned representations by GCL essentially encode the low-frequency information, which explains why GCL works.
1 code implementation • 16 Aug 2022 • Zhenan Fan, Zirui Zhou, Jian Pei, Michael P. Friedlander, Jiajie Hu, Chengliang Li, Yong Zhang
Federated learning is an emerging technique for training models from decentralized data sets.
no code implementations • 12 Jul 2022 • Mohit Bajaj, Lingyang Chu, Vittorio Romaniello, Gursimran Singh, Jian Pei, Zirui Zhou, Lanjun Wang, Yong Zhang
The key idea is to find solid evidence in the form of a group of data instances discriminated most by the model.
1 code implementation • 21 Jun 2022 • Shengyao Zhuang, Houxing Ren, Linjun Shou, Jian Pei, Ming Gong, Guido Zuccon, Daxin Jiang
This problem is further exacerbated when using DSI for cross-lingual retrieval, where document text and query text are in different languages.
no code implementations • 11 Jun 2022 • Junyi Li, Jian Pei, Heng Huang
Bilevel optimization problem is a type of optimization problem with two levels of entangled problems.
1 code implementation • 6 Jun 2022 • Lianghao Xia, Chao Huang, Yong Xu, Jian Pei
The new TGT method endows the sequential recommendation architecture to distill dedicated knowledge for type-specific behavior relational context and the implicit behavior dependencies.
no code implementations • 16 May 2022 • He Zhang, Bang Wu, Xingliang Yuan, Shirui Pan, Hanghang Tong, Jian Pei
Graph neural networks (GNNs) have emerged as a series of competent graph learning methods for diverse real-world scenarios, ranging from daily applications like recommendation systems and question answering to cutting-edge technologies such as drug discovery in life sciences and n-body simulation in astrophysics.
no code implementations • 7 May 2022 • Shining Liang, Linjun Shou, Jian Pei, Ming Gong, Wanli Zuo, Xianglin Zuo, Daxin Jiang
Despite the great success of spoken language understanding (SLU) in high-resource languages, it remains challenging in low-resource languages mainly due to the lack of labeled training data.
1 code implementation • 18 Apr 2022 • Zhonghang Li, Chao Huang, Lianghao Xia, Yong Xu, Jian Pei
Crime has become a major concern in many cities, which calls for the rising demand for timely predicting citywide crime occurrence.
no code implementations • NAACL 2022 • Nuo Chen, Linjun Shou, Ming Gong, Jian Pei, Daxin Jiang
Large-scale cross-lingual pre-trained language models (xPLMs) have shown effectiveness in cross-lingual sequence labeling tasks (xSL), such as cross-lingual machine reading comprehension (xMRC) by transferring knowledge from a high-resource language to low-resource languages.
no code implementations • 10 Mar 2022 • Saeed Ranjbar Alvar, Lanjun Wang, Jian Pei, Yong Zhang
Image-to-image translation models are shown to be vulnerable to the Membership Inference Attack (MIA), in which the adversary's goal is to identify whether a sample is used to train the model or not.
no code implementations • 7 Jan 2022 • Zhenan Fan, Huang Fang, Zirui Zhou, Jian Pei, Michael P. Friedlander, Yong Zhang
We show that VerFedSV not only satisfies many desirable properties for fairness but is also efficient to compute, and can be adapted to both synchronous and asynchronous vertical federated learning algorithms.
1 code implementation • 22 Dec 2021 • Yiming Zhang, Lingfei Wu, Qi Shen, Yitong Pang, Zhihua Wei, Fangli Xu, Bo Long, Jian Pei
As a result, we first propose a more realistic CRS learning setting, namely Multi-Interest Multi-round Conversational Recommendation, where users may have multiple interests in attribute instance combinations and accept multiple items with partially overlapped combinations of attribute instances.
no code implementations • 15 Dec 2021 • Gursimran Singh, Lingyang Chu, Lanjun Wang, Jian Pei, Qi Tian, Yong Zhang
In the real world, the frequency of occurrence of objects is naturally skewed forming long-tail class distributions, which results in poor performance on the statistically rare classes.
no code implementations • 9 Dec 2021 • Nuo Chen, Linjun Shou, Min Gong, Jian Pei, Daxin Jiang
Cross-lingual Machine Reading Comprehension (xMRC) is challenging due to the lack of training data in low-resource languages.
1 code implementation • 8 Oct 2021 • Lianghao Xia, Chao Huang, Yong Xu, Peng Dai, Xiyue Zhang, Hongsheng Yang, Jian Pei, Liefeng Bo
In particular: i) complex inter-dependencies across different types of user behaviors; ii) the incorporation of knowledge-aware item relations into the multi-behavior recommendation framework; iii) dynamic characteristics of multi-typed user-item interactions.
no code implementations • 26 Sep 2021 • Qingsong Zhang, Bin Gu, Cheng Deng, Songxiang Gu, Liefeng Bo, Jian Pei, Heng Huang
To address the challenges of communication and computation resource utilization, we propose an asynchronous stochastic quasi-Newton (AsySQN) framework for VFL, under which three algorithms, i. e. AsySQN-SGD, -SVRG and -SAGA, are proposed.
no code implementations • 19 Sep 2021 • Zhenan Fan, Huang Fang, Zirui Zhou, Jian Pei, Michael P. Friedlander, Changxin Liu, Yong Zhang
The success of federated learning depends largely on the participation of data owners.
1 code implementation • 17 Sep 2021 • Changxin Liu, Zhenan Fan, Zirui Zhou, Yang Shi, Jian Pei, Lingyang Chu, Yong Zhang
To solve it in a federated and privacy-preserving manner, we consider the equivalent dual form of the problem and develop an asynchronous gradient coordinate-descent ascent algorithm, where some active data parties perform multiple parallelized local updates per communication round to effectively reduce the number of communication rounds.
no code implementations • 13 Sep 2021 • Lingyang Chu, Lanjun Wang, Yanjie Dong, Jian Pei, Zirui Zhou, Yong Zhang
In this paper, we first propose a federated estimation method to accurately estimate the fairness of a model without infringing the data privacy of any party.
no code implementations • EMNLP 2021 • YingMei Guo, Linjun Shou, Jian Pei, Ming Gong, Mingxing Xu, Zhiyong Wu, Daxin Jiang
Although various data augmentation approaches have been proposed to synthesize training data in low-resource target languages, the augmented data sets are often noisy, and thus impede the performance of SLU models.
1 code implementation • 30 Aug 2021 • Amin Banitalebi-Dehkordi, Naveen Vedula, Jian Pei, Fei Xia, Lanjun Wang, Yong Zhang
At the same time, large amounts of input data are collected at the edge of cloud.
no code implementations • ICCV 2021 • Peter Cho-Ho Lam, Lingyang Chu, Maxim Torgonskiy, Jian Pei, Yong Zhang, Lanjun Wang
Interpreting the decision logic behind effective deep convolutional neural networks (CNN) on images complements the success of deep learning models.
no code implementations • ACL 2021 • Hao Huang, Xiubo Geng, Jian Pei, Guodong Long, Daxin Jiang
Procedural text understanding aims at tracking the states (e. g., create, move, destroy) and locations of the entities mentioned in a given paragraph.
1 code implementation • 8 Jul 2021 • Yitong Pang, Lingfei Wu, Qi Shen, Yiming Zhang, Zhihua Wei, Fangli Xu, Ethan Chang, Bo Long, Jian Pei
Additionally, existing personalized session-based recommenders capture user preference only based on the sessions of the current user, but ignore the useful item-transition patterns from other user's historical sessions.
no code implementations • NeurIPS 2021 • Mohit Bajaj, Lingyang Chu, Zi Yu Xue, Jian Pei, Lanjun Wang, Peter Cho-Ho Lam, Yong Zhang
Massive deployment of Graph Neural Networks (GNNs) in high-stake applications generates a strong demand for explanations that are robust to noise and align well with human intuition.
1 code implementation • 10 Jun 2021 • Lingfei Wu, Yu Chen, Kai Shen, Xiaojie Guo, Hanning Gao, Shucheng Li, Jian Pei, Bo Long
Deep learning has become the dominant approach in coping with various tasks in Natural LanguageProcessing (NLP).
no code implementations • 1 Jun 2021 • Shining Liang, Ming Gong, Jian Pei, Linjun Shou, Wanli Zuo, Xianglin Zuo, Daxin Jiang
Named entity recognition (NER) is a fundamental component in many applications, such as Web Search and Voice Assistants.
no code implementations • 8 Mar 2021 • Xia Hu, Lingyang Chu, Jian Pei, Weiqing Liu, Jiang Bian
Model complexity is a fundamental problem in deep learning.
no code implementations • 11 Dec 2020 • Fei Yuan, Linjun Shou, Jian Pei, Wutao Lin, Ming Gong, Yan Fu, Daxin Jiang
When multiple teacher models are available in distillation, the state-of-the-art methods assign a fixed weight to a teacher model in the whole distillation.
no code implementations • 11 Nov 2020 • Shining Liang, Linjun Shou, Jian Pei, Ming Gong, Wanli Zuo, Daxin Jiang
To tackle the challenge of lack of training data in low-resource languages, we dedicatedly develop a novel unsupervised phrase boundary recovery pre-training task to enhance the multilingual boundary detection capability of CalibreNet.
no code implementations • 1 Nov 2020 • Zicun Cong, Lingyang Chu, Yu Yang, Jian Pei
One challenge remained untouched is how we can obtain an explanation on why a test set fails the KS test.
no code implementations • COLING 2020 • Junhao Liu, Linjun Shou, Jian Pei, Ming Gong, Min Yang, Daxin Jiang
Then, we devise a multilingual distillation approach to amalgamate knowledge from multiple language branch models to a single model for all target languages.
no code implementations • COLING 2020 • Xingyao Zhang, Linjun Shou, Jian Pei, Ming Gong, Lijie Wen, Daxin Jiang
The abundant semi-structured data on the Web, such as HTML-based tables and lists, provide commercial search engines a rich information source for question answering (QA).
no code implementations • 9 Sep 2020 • Jian Pei
Data are invaluable.
1 code implementation • 5 Sep 2020 • Ziwei Zhang, Chenhao Niu, Peng Cui, Jian Pei, Bo Zhang, Wenwu Zhu
Graph neural networks (GNNs) are emerging machine learning models on graphs.
no code implementations • 18 Aug 2020 • Feihu Huang, Shangqian Gao, Jian Pei, Heng Huang
Our Acc-MDA achieves a low gradient complexity of $\tilde{O}(\kappa_y^{4. 5}\epsilon^{-3})$ without requiring large batches for finding an $\epsilon$-stationary point.
1 code implementation • ICML 2020 • Feihu Huang, Shangqian Gao, Jian Pei, Heng Huang
In particular, we present a non-adaptive version of IS-MBPG method, i. e., IS-MBPG*, which also reaches the best known sample complexity of $O(\epsilon^{-3})$ without any large batches.
1 code implementation • 7 Jul 2020 • Yutao Huang, Lingyang Chu, Zirui Zhou, Lanjun Wang, Jiangchuan Liu, Jian Pei, Yong Zhang
Non-IID data present a tough challenge for federated learning.
no code implementations • 5 Jul 2020 • Xiao Wang, Meiqi Zhu, Deyu Bo, Peng Cui, Chuan Shi, Jian Pei
We tackle the challenge and propose an adaptive multi-channel graph convolutional networks for semi-supervised classification (AM-GCN).
no code implementations • 16 Jun 2020 • Xia Hu, Weiqing Liu, Jiang Bian, Jian Pei
Our results demonstrate that the occurrence of overfitting is positively correlated with the increase of model complexity during training.
no code implementations • 13 Jun 2020 • Linjun Shou, Shining Bo, Feixiang Cheng, Ming Gong, Jian Pei, Daxin Jiang
In this paper, we make the first study to explore the correlation between user behavior and passage relevance, and propose a novel approach for mining training data for Web QA.
no code implementations • 8 Jun 2020 • Ziwei Zhang, Peng Cui, Jian Pei, Xin Wang, Wenwu Zhu
Graph Neural Networks (GNNs) are emerging machine learning models on graphs.
1 code implementation • 2020 • Mingdong Ou, Peng Cui, Jian Pei, Ziwei Zhang, Wenwu Zhu
In particular, we develop a novel graph embedding algorithm, High-Order Proximity preserved Embedding (HOPE for short), which is scalable to preserve high-order proximities of large scale graphs and capable of capturing the asymmetric transitivity.
no code implementations • 30 Jul 2019 • Feihu Huang, Shangqian Gao, Jian Pei, Heng Huang
Zeroth-order (a. k. a, derivative-free) methods are a class of effective optimization methods for solving complex machine learning problems, where gradients of the objective functions are not available or computationally prohibitive.
1 code implementation • 17 Jun 2019 • Zicun Cong, Lingyang Chu, Lanjun Wang, Xia Hu, Jian Pei
More and more AI services are provided through APIs on cloud where predictive models are hidden behind APIs.
no code implementations • NAACL 2019 • Wei Yang, Luchen Tan, Chunwei Lu, Anqi Cui, Han Li, Xi Chen, Kun Xiong, Muzi Wang, Ming Li, Jian Pei, Jimmy Lin
Consumers dissatisfied with the normal dispute resolution process provided by an e-commerce company{'}s customer service agents have the option of escalating their complaints by filing grievances with a government authority.
no code implementations • 2 May 2019 • Wenhui Yu, Xiangnan He, Jian Pei, Xu Chen, Li Xiong, Jinfei Liu, Zheng Qin
While recent developments on visually-aware recommender systems have taken the product image into account, none of them has considered the aesthetic aspect.
no code implementations • 17 Feb 2018 • Lingyang Chu, Xia Hu, Juhua Hu, Lanjun Wang, Jian Pei
Strong intelligent machines powered by deep neural networks are increasingly deployed as black boxes to make decisions in risk-sensitive domains, such as finance and medical.
1 code implementation • 27 Nov 2017 • Ziwei Zhang, Peng Cui, Jian Pei, Xiao Wang, Wenwu Zhu
By setting a maximum tolerated error as a threshold, we can trigger SVD restart automatically when the margin exceeds this threshold. We prove that the time complexity of our method is linear with respect to the number of local dynamic changes, and our method is general across different types of dynamic networks.
Social and Information Networks
no code implementations • 23 Nov 2017 • Peng Cui, Xiao Wang, Jian Pei, Wenwu Zhu
Network embedding assigns nodes in a network to low-dimensional representations and effectively preserves the network structure.
Social and Information Networks
no code implementations • 23 Sep 2017 • Lingyang Chu, Zhefeng Wang, Jian Pei, Yanyan Zhang, Yu Yang, Enhong Chen
Given a database network where each vertex is associated with a transaction database, we are interested in finding theme communities.
2 code implementations • AAAI 2017 • Xiao Wang, Peng Cui, Jing Wang, Jian Pei, Wenwu Zhu, Shiqiang Yang
While previous network embedding methods primarily preserve the microscopic structure, such as the first- and second-order proximities of nodes, the mesoscopic community structure, which is one of the most prominent feature of networks, is largely ignored.
1 code implementation • 13 Dec 2015 • Shixia Liu, Jialun Yin, Xiting Wang, Weiwei Cui, Kelei Cao, Jian Pei
To this end, we learn a set of streaming tree cuts from topic trees based on user-selected focus nodes.
1 code implementation • 30 Nov 2015 • Kui Yu, Xindong Wu, Wei Ding, Jian Pei
Feature selection is important in many big data applications.