1 code implementation • 21 Jun 2023 • Chengchao Shen, Dawei Liu, Hao Tang, Zhe Qu, Jianxin Wang
In this paper, we propose a novel image mix method, PatchMix, for contrastive learning in Vision Transformer (ViT), to model inter-instance similarities among images.
1 code implementation • 5 Jun 2023 • Chengchao Shen, Jianzhong Chen, Shu Wang, Hulin Kuang, Jin Liu, Jianxin Wang
Asymmetric appearance between positive pair effectively reduces the risk of representation degradation in contrastive learning.
no code implementations • CVPR 2023 • Zhe Qu, Xingyu Li, Xiao Han, Rui Duan, Chengchao Shen, Lixing Chen
Intuitively, these poor clients may come from biased universal information shared with others.
1 code implementation • 17 Dec 2022 • Tao Sheng, Chengchao Shen, YuAn Liu, Yeyu Ou, Zhe Qu, Jianxin Wang
It introduces a global Generative Adversarial Network to model the global data distribution without access to local datasets, so the global model can be trained using the global information of data distribution without privacy leakage.
1 code implementation • 23 Nov 2021 • Tongya Zheng, Zunlei Feng, Yu Wang, Chengchao Shen, Mingli Song, Xingen Wang, Xinyu Wang, Chun Chen, Hao Xu
Our proposed Dynamic Preference Structure (DPS) framework consists of two stages: structure sampling and graph fusion.
2 code implementations • NeurIPS 2021 • Gongfan Fang, Yifan Bao, Jie Song, Xinchao Wang, Donglin Xie, Chengchao Shen, Mingli Song
Knowledge distillation~(KD) aims to craft a compact student model that imitates the behavior of a pre-trained teacher in a target domain.
3 code implementations • 18 May 2021 • Gongfan Fang, Jie Song, Xinchao Wang, Chengchao Shen, Xingen Wang, Mingli Song
In this paper, we propose Contrastive Model Inversion~(CMI), where the data diversity is explicitly modeled as an optimizable objective, to alleviate the mode collapse issue.
1 code implementation • CVPR 2021 • Chengchao Shen, Youtan Yin, Xinchao Wang, Xubin Li, Jie Song, Mingli Song
Based on the adversarial losses of the generator and discriminator, we categorize GANs into two classes, Symmetric GANs and Asymmetric GANs, and introduce a novel gradient decomposition method to unify the two, allowing us to train both classes in one stage and hence alleviate the training effort.
2 code implementations • 9 Dec 2020 • Chengchao Shen, Xinchao Wang, Youtan Yin, Jie Song, Sihui Luo, Mingli Song
In this paper, we investigate the practical few-shot knowledge distillation scenario, where we assume only a few samples without human annotations are available for each category.
1 code implementation • CVPR 2020 • Jie Song, Yixin Chen, Jingwen Ye, Xinchao Wang, Chengchao Shen, Feng Mao, Mingli Song
In this paper, we propose the DEeP Attribution gRAph (DEPARA) to investigate the transferability of knowledge learned from PR-DNNs.
3 code implementations • 23 Dec 2019 • Gongfan Fang, Jie Song, Chengchao Shen, Xinchao Wang, Da Chen, Mingli Song
Knowledge Distillation (KD) has made remarkable progress in the last few years and become a popular paradigm for model compression and knowledge transfer.
2 code implementations • NeurIPS 2019 • Jie Song, Yixin Chen, Xinchao Wang, Chengchao Shen, Mingli Song
Exploring the transferability between heterogeneous tasks sheds light on their intrinsic interconnections, and consequently enables knowledge transfer from one task to another so as to reduce the training effort of the latter.
2 code implementations • ICCV 2019 • Chengchao Shen, Mengqi Xue, Xinchao Wang, Jie Song, Li Sun, Mingli Song
To this end, we introduce a dual-step strategy that first extracts the task-specific knowledge from the heterogeneous teachers sharing the same sub-task, and then amalgamates the extracted knowledge to build the student network.
1 code implementation • 7 Nov 2018 • Chengchao Shen, Xinchao Wang, Jie Song, Li Sun, Mingli Song
We propose in this paper to study a new model-reusing task, which we term as \emph{knowledge amalgamation}.
no code implementations • ECCV 2018 • Jie Song, Chengchao Shen, Jie Lei, An-Xiang Zeng, Kairi Ou, DaCheng Tao, Mingli Song
We propose a selective zero-shot classifier based on both the human defined and the automatically discovered residual attributes.
no code implementations • CVPR 2018 • Jie Song, Chengchao Shen, Yezhou Yang, Yang Liu, Mingli Song
Most existing Zero-Shot Learning (ZSL) methods have the strong bias problem, in which instances of unseen (target) classes tend to be categorized as one of the seen (source) classes.