no code implementations • 13 May 2024 • Lixi Zhu, Xiaowen Huang, Jitao Sang
Through experiments and case studies in two conversational recommendation scenarios, we show that our framework can adapt to a variety of conversational recommendation settings and effectively simulate users' personalized preferences.
1 code implementation • 27 Apr 2024 • Lei Cheng, Xiaowen Huang, Jitao Sang, Jian Yu
In the adversarial robustness, we introduce the fundamental principles and classical methods of recommender system adversarial attacks and defenses.
no code implementations • 16 Apr 2024 • Rui Hu, Yahan Tu, Jitao Sang
In this paper, we propose a targeted instruction data generation framework named DFTG that tailored to the hallucination specificity of different models.
no code implementations • 7 Apr 2024 • Yi Zhang, Jitao Sang
Machine learning models often make predictions based on biased features such as gender, race, and other social attributes, posing significant fairness risks, especially in societal applications, such as hiring, banking, and criminal justice.
no code implementations • 27 Mar 2024 • YuQi Yang, Xiaowen Huang, Jitao Sang
Large language models (LLMs), renowned for their impressive capabilities in various tasks, have significantly advanced artificial intelligence.
no code implementations • 25 Mar 2024 • Lixi Zhu, Xiaowen Huang, Jitao Sang
Through multiple experiments on two widely-used datasets in the field of conversational recommendation, we highlight several issues with the current evaluation methods for user simulators based on LLMs: (1) Data leakage, which occurs in conversational history and the user simulator's replies, results in inflated evaluation results.
no code implementations • 13 Mar 2024 • YiFei Gao, Jiaqi Wang, Zhiyu Lin, Jitao Sang
Remarkably, our findings shed light on a consistent AIGC \textbf{hallucination bias}: the object hallucinations induced by synthetic images are characterized by a greater quantity and a more uniform position distribution, even these synthetic images do not manifest unrealistic or additional relevant visual features compared to natural images.
1 code implementation • 1 Feb 2024 • Jitao Sang, Yuhang Wang, Jing Zhang, Yanxu Zhu, Chao Kong, Junhong Ye, Shuyu Wei, Jinlin Xiao
In the first phase, based on human supervision, the quality of weak supervision is enhanced through a combination of scalable oversight and ensemble learning, reducing the capability gap between weak teachers and strong students.
1 code implementation • 29 Jan 2024 • Junyang Wang, Haiyang Xu, Jiabo Ye, Ming Yan, Weizhou Shen, Ji Zhang, Fei Huang, Jitao Sang
To assess the performance of Mobile-Agent, we introduced Mobile-Eval, a benchmark for evaluating mobile device operations.
no code implementations • 9 Dec 2023 • Chaoquan Jiang, Jinqiang Wang, Rui Hu, Jitao Sang
To address this issue, We propose a language-assisted diagnostic method that uses texts instead of images to diagnose bugs in vision models based on multi-modal models (eg CLIP).
1 code implementation • 28 Nov 2023 • Yuhang Wang, Yanxu Zhu, Chao Kong, Shuyu Wei, Xiaoyuan Yi, Xing Xie, Jitao Sang
This benchmark serves as a valuable resource for cultural studies in LLMs, paving the way for more culturally aware and sensitive models.
1 code implementation • 19 Nov 2023 • Jiaming Zhang, Xingjun Ma, Xin Wang, Lingyu Qiu, Jiaqi Wang, Yu-Gang Jiang, Jitao Sang
With the rapid advancement of multimodal learning, pre-trained Vision-Language Models (VLMs) such as CLIP have demonstrated remarkable capacities in bridging the gap between visual and language modalities.
1 code implementation • 13 Nov 2023 • Junyang Wang, Yuhang Wang, Guohai Xu, Jing Zhang, Yukai Gu, Haitao Jia, Jiaqi Wang, Haiyang Xu, Ming Yan, Ji Zhang, Jitao Sang
Despite making significant progress in multi-modal tasks, current Multi-modal Large Language Models (MLLMs) encounter the significant challenge of hallucinations, which may lead to harmful consequences.
1 code implementation • 29 Aug 2023 • Junyang Wang, Yiyang Zhou, Guohai Xu, Pengcheng Shi, Chenlin Zhao, Haiyang Xu, Qinghao Ye, Ming Yan, Ji Zhang, Jihua Zhu, Jitao Sang, Haoyu Tang
In this paper, we propose Hallucination Evaluation based on Large Language Models (HaELM), an LLM-based hallucination evaluation framework.
no code implementations • 13 Aug 2023 • Yi Zhang, Jitao Sang, Junyang Wang, Dongmei Jiang, YaoWei Wang
To this end, we propose \emph{Shortcut Debiasing}, to first transfer the target task's learning of bias attributes from bias features to shortcut features, and then employ causal intervention to eliminate shortcut features during inference.
no code implementations • 4 Aug 2023 • Yuhang Wang, Yuxiang Zhang, Dongyuan Lu, Jitao Sang
Many news comment mining studies are based on the assumption that comment is explicitly linked to the corresponding news.
1 code implementation • 19 Jul 2023 • Guohai Xu, Jiayi Liu, Ming Yan, Haotian Xu, Jinghui Si, Zhuoran Zhou, Peng Yi, Xing Gao, Jitao Sang, Rong Zhang, Ji Zhang, Chao Peng, Fei Huang, Jingren Zhou
In this paper, we present CValues, the first Chinese human values evaluation benchmark to measure the alignment ability of LLMs in terms of both safety and responsibility criteria.
no code implementations • 13 Jul 2023 • Jiaming Zhang, Jitao Sang, Qi Yi, Changsheng Xu
Harnessing the concept of non-robust features, we elaborate on two guiding principles for surrogate model selection to explain why the foundational model is an optimal choice for this role.
1 code implementation • 6 Jun 2023 • Yuhang Wang, Dongyuan Lu, Chao Kong, Jitao Sang
Many works employed prompt tuning methods to automatically optimize prompt queries and extract the factual knowledge stored in Pretrained Language Models.
no code implementations • 3 Jun 2023 • YiFei Gao, Zhiyu Lin, Yunfan Yang, Jitao Sang
Black-box attack, which is a more realistic threat and has led to various black-box adversarial training-based defense methods, however, does not attract considerable attention in adversarial example detection.
1 code implementation • 6 May 2023 • Rui Hu, Yahan Tu, Jitao Sang
This paper first presents experimental analyses revealing that the existing biased models overfit to bias-conflicting samples in the training data, which negatively impacts the debiasing performance of the target models.
1 code implementation • 26 Apr 2023 • Junyang Wang, Ming Yan, Yi Zhang, Jitao Sang
Although previous works have created generation capacity for CLIP through additional language models, a modality gap between the CLIP representations of different modalities and the inability of CLIP to model the offset of this gap, which fails the concept to transfer across modalities.
1 code implementation • ICCV 2023 • Junyang Wang, Yuanhong Xu, Juhua Hu, Ming Yan, Jitao Sang, Qi Qian
Fine-tuning a visual pre-trained model can leverage the semantic information from large-scale pre-training data and mitigate the over-fitting problem on downstream vision tasks with limited training examples.
no code implementations • 1 Mar 2023 • Shangxi Wu, Qiuyang He, Fangzhao Wu, Jitao Sang, YaoWei Wang, Changsheng Xu
In this work, we found that the backdoor attack can construct an artificial bias similar to the model bias derived in standard training.
1 code implementation • CVPR 2023 • Jiaming Zhang, Xingjun Ma, Qi Yi, Jitao Sang, Yu-Gang Jiang, YaoWei Wang, Changsheng Xu
Furthermore, we propose to leverage VisionandLanguage Pre-trained Models (VLPMs) like CLIP as the surrogate model to improve the transferability of the crafted UCs to diverse domains.
no code implementations • 7 Dec 2022 • Yinpeng Dong, Peng Chen, Senyou Deng, Lianji L, Yi Sun, Hanyu Zhao, Jiaxing Li, Yunteng Tan, Xinyu Liu, Yangyi Dong, Enhui Xu, Jincai Xu, Shu Xu, Xuelin Fu, Changfeng Sun, Haoliang Han, Xuchong Zhang, Shen Chen, Zhimin Sun, Junyi Cao, Taiping Yao, Shouhong Ding, Yu Wu, Jian Lin, Tianpeng Wu, Ye Wang, Yu Fu, Lin Feng, Kangkang Gao, Zeyu Liu, Yuanzhe Pang, Chengqi Duan, Huipeng Zhou, Yajie Wang, Yuhang Zhao, Shangbo Wu, Haoran Lyu, Zhiyu Lin, YiFei Gao, Shuang Li, Haonan Wang, Jitao Sang, Chen Ma, Junhao Zheng, Yijia Li, Chao Shen, Chenhao Lin, Zhichao Cui, Guoshuai Liu, Huafeng Shi, Kun Hu, Mengxin Zhang
The security of artificial intelligence (AI) is an important research area towards safe, reliable, and trustworthy AI systems.
no code implementations • 14 Nov 2022 • Junyang Wang, Yi Zhang, Ming Yan, Ji Zhang, Jitao Sang
We further propose Anchor Augment to guide the generative model's attention to the fine-grained information in the representation of CLIP.
no code implementations • 2 Nov 2022 • Yi Zhang, Jitao Sang, Junyang Wang
To this end, we propose \emph{Proxy Debiasing}, to first transfer the target task's learning of bias information from bias features to artificial proxy features, and then employ causal intervention to eliminate proxy features in inference.
no code implementations • 26 Oct 2022 • Junyang Wang, Yi Zhang, Jitao Sang
Although FairCLIP is used to eliminate bias in image retrieval, it achieves the neutralization of the representation which is common to all CLIP downstream tasks.
1 code implementation • 3 Jul 2022 • Yi Zhang, Junyang Wang, Jitao Sang
Vision-Language Pre-training (VLP) models have achieved state-of-the-art performance in numerous cross-modal tasks.
1 code implementation • 19 Jun 2022 • Jiaming Zhang, Qi Yi, Jitao Sang
While vision-language pre-training model (VLP) has shown revolutionary improvements on various vision-language (V+L) tasks, the studies regarding its adversarial robustness remain largely unexplored.
no code implementations • 19 Jun 2022 • Jiaming Zhang, Qi Yi, Dongyuan Lu, Jitao Sang
In light of the growing concerns regarding the unauthorized use of facial recognition systems and its implications on individual privacy, the exploration of adversarial perturbations as a potential countermeasure has gained traction.
1 code implementation • 6 May 2022 • Zhiyu Lin, YiFei Gao, Jitao Sang
Specifically, our investigations verify that the spectral density of datasets mainly affects the learning priority, while the class consistency mainly affects the feature discrimination.
no code implementations • CVPR 2022 • Yaogong Feng, Xiaowen Huang, Pengbo Yang, Jian Yu, Jitao Sang
Synthesizing pseudo samples is currently the most effective way to solve the Generalized Zero-Shot Learning (GZSL) problem.
no code implementations • 18 Feb 2022 • Shangxi Wu, Qiuyang He, Yi Zhang, Jitao Sang
Backdoor attack is a new AI security risk that has emerged in recent years.
no code implementations • 17 Nov 2021 • Jitao Sang, Jinqiang Wang, Rui Hu, Chaoquan Jiang
Deep network models perform excellently on In-Distribution (ID) data, but can significantly fail on Out-Of-Distribution (OOD) data.
no code implementations • 18 Oct 2021 • Xiaowen Huang, Jitao Sang, Jian Yu, Changsheng Xu
The cold-start recommendation is an urgent problem in contemporary online applications.
1 code implementation • 13 Oct 2021 • Mengyuan Zhao, Xiaowen Huang, Lixi Zhu, Jitao Sang, Jian Yu
Then, two samplers are designed to enhance knowledge by sampling fuzzy samples with high uncertainty for obtaining user preferences and reliable negative samples for updating recommender to achieve efficient acquisition of user preferences and model updating, and thus provide a powerful solution for CRS to deal with E&E problem.
no code implementations • 29 Sep 2021 • Guanhua Zheng, Jitao Sang, Wang Haonan, Changsheng Xu
Recently, backpropagation(BP)-based feature attribution methods have been widely adopted to interpret the internal mechanisms of convolutional neural networks (CNNs), and expected to be human-understandable (lucidity) and faithful to decision-making processes (fidelity).
no code implementations • 26 Jul 2021 • Jitao Sang, Xian Zhao, Jiaming Zhang, Zhiyu Lin
In spite of the successful application in many fields, machine learning models today suffer from notorious problems like vulnerability to adversarial examples.
1 code implementation • 21 Jun 2021 • Jiaming Zhang, Jitao Sang, Qi Yi, Yunfan Yang, Huiwen Dong, Jian Yu
ImageNet pre-training has enabled state-of-the-art results on many tasks.
no code implementations • 19 Nov 2020 • Shangxi Wu, Jitao Sang, Xian Zhao, Lizhang Chen
Deep learning models suffer from the problem of semantic discontinuity: small perturbations in the input space tend to cause semantic-level interference to the model output.
no code implementations • 27 Jul 2020 • Yi Zhang, Jitao Sang
Our data analysis on facial attribute recognition demonstrates (1) the attribution of model bias from imbalanced training data distribution and (2) the potential of adversarial examples in balancing data distribution.
2 code implementations • 25 Jul 2020 • Jiaming Zhang, Jitao Sang, Xian Zhao, Xiaowen Huang, Yanfeng Sun, Yongli Hu
While widely adopted in practical applications, face recognition has been critically discussed regarding the malicious use of face images and the potential privacy problems, e. g., deceiving payment system and causing personal sabotage.
no code implementations • 18 Jun 2020 • Guanhua Zheng, Jitao Sang, Changsheng Xu
Since the basic assumption of conventional manifold learning fails in case of sparse and uneven data distribution, we introduce a new target, Minimum Manifold Coding (MMC), for manifold learning to encourage simple and unfolded manifold.
no code implementations • 25 May 2020 • Shangxi Wu, Jitao Sang, Kaiyuan Xu, Guanhua Zheng, Changsheng Xu
Specifically, AALP consists of an adaptive feature optimization module with Guided Dropout to systematically pursue fewer high-contribution features, and an adaptive sample weighting module by setting sample-specific training weights to balance between logits pairing loss and classification loss.
no code implementations • 28 Nov 2019 • Guanhua Zheng, Jitao Sang, Houqiang Li, Jian Yu, Changsheng Xu
The derived generalization bound based on the ITID assumption identifies the significance of hypothesis invariance in guaranteeing generalization performance.
no code implementations • 22 Apr 2019 • Jiaming Zhang, Jitao Sang, Kaiyuan Xu, Shangxi Wu, Yongli Hu, Yanfeng Sun, Jian Yu
Turing test was originally proposed to examine whether machine's behavior is indistinguishable from a human.
no code implementations • 24 Nov 2018 • Shangxi Wu, Jitao Sang, Kaiyuan Xu, Jiaming Zhang, Jian Yu
This study provides a new understanding of the adversarial attack problem by examining the correlation between adversarial attack and visual attention change.
no code implementations • ICLR 2018 • Guanhua Zheng, Jitao Sang, Changsheng Xu
DNN is then regarded as approximating the feature conditions with multilayer feature learning, and proved to be a recursive solution towards maximum entropy principle.