no code implementations • 24 Mar 2024 • Libo Huang, Zhulin An, Yan Zeng, Chuanguang Yang, Xinqiang Yu, Yongjun Xu
Exemplar-Free Class Incremental Learning (efCIL) aims to continuously incorporate the knowledge from new classes while retaining previously learned information, without storing any old-class exemplars (i. e., samples).
1 code implementation • 24 Jul 2023 • Chuanguang Yang, Zhulin An, Libo Huang, Junyu Bi, Xinqiang Yu, Han Yang, Boyu Diao, Yongjun Xu
The unified method is applied to distill several student models trained on CC3M+12M.
no code implementations • 19 Jun 2023 • Chuanguang Yang, Xinqiang Yu, Zhulin An, Yongjun Xu
Knowledge Distillation (KD) aims to optimize a lightweight network from the perspective of over-parameterized training.
no code implementations • 15 Jun 2023 • Yuqi Li, Yizhi Luo, Xiaoshuai Hao, Chuanguang Yang, Zhulin An, Dantong Song, Wei Yi
In this report, we describe the technical details of our submission to the EPIC-SOUNDS Audio-Based Interaction Recognition Challenge 2023, by Team "AcieLee" (username: Yuqi\_Li).
no code implementations • 20 Apr 2023 • Libo Huang, Yan Zeng, Chuanguang Yang, Zhulin An, Boyu Diao, Yongjun Xu
Most successful CIL methods incrementally train a feature extractor with the aid of stored exemplars, or estimate the feature distribution with the stored prototypes.
no code implementations • ICCV 2023 • Junyu Bi, Daixuan Cheng, Ping Yao, Bochen Pang, Yuefeng Zhan, Chuanguang Yang, Yujing Wang, Hao Sun, Weiwei Deng, Qi Zhang
Vision-Language Pretraining (VLP) has significantly improved the performance of various vision-language tasks with the matching of images and texts.
1 code implementation • 11 Aug 2022 • Chuanguang Yang, Zhulin An, Helong Zhou, Linhang Cai, Xiang Zhi, Jiwen Wu, Yongjun Xu, Qian Zhang
MixSKD mutually distills feature maps and probability distributions between the random pair of original images and their mixup images in a meaningful way.
2 code implementations • 23 Jul 2022 • Chuanguang Yang, Zhulin An, Helong Zhou, Fuzhen Zhuang, Yongjun Xu, Qian Zhan
This enables each network to learn extra contrastive knowledge from others, leading to better feature representations, thus improving the performance of visual recognition tasks.
1 code implementation • 7 Jun 2022 • Chuanguang Yang, Zhulin An, Yongjun Xu
This ensures the exact mapping from a high-level spatial location to the specific input image patch.
1 code implementation • CVPR 2022 • Chuanguang Yang, Helong Zhou, Zhulin An, Xue Jiang, Yongjun Xu, Qian Zhang
Current Knowledge Distillation (KD) methods for semantic segmentation often guide the student to mimic the teacher's structured information generated from individual data samples.
1 code implementation • AAAI 2022 • Linhang Cai, Zhulin An, Chuanguang Yang, Yangchun Yan, Yongjun Xu
In detail, the proposed PGMPF selectively suppresses the gradient of those ”unimportant” parameters via a prior gradient mask generated by the pruning criterion during fine-tuning.
1 code implementation • 7 Sep 2021 • Chuanguang Yang, Zhulin An, Linhang Cai, Yongjun Xu
Each auxiliary branch is guided to learn self-supervision augmented task and distill this distribution from teacher to student.
1 code implementation • 29 Jul 2021 • Chuanguang Yang, Zhulin An, Linhang Cai, Yongjun Xu
We therefore adopt an alternative self-supervised augmented task to guide the network to learn the joint distribution of the original recognition task and self-supervised auxiliary task.
Ranked #20 on Knowledge Distillation on ImageNet
1 code implementation • 26 Apr 2021 • Chuanguang Yang, Zhulin An, Linhang Cai, Yongjun Xu
We present a collaborative learning method called Mutual Contrastive Learning (MCL) for general visual representation learning.
no code implementations • 13 Dec 2020 • Kun Zhang, Rui Wu, Ping Yao, Kai Deng, Ding Li, Renbiao Liu, Chuanguang Yang, Ge Chen, Min Du, Tianyao Zheng
We note that 2D pose estimation task is highly dependent on the contextual relationship between image patches, thus we introduce a self-supervised method for pretraining 2D pose estimation networks.
no code implementations • 19 Oct 2020 • Linhang Cai, Zhulin An, Chuanguang Yang, Yongjun Xu
Network pruning is widely used to compress Deep Neural Networks (DNNs).
1 code implementation • 7 Jun 2020 • Chuanguang Yang, Zhulin An, Yongjun Xu
Previous Online Knowledge Distillation (OKD) often carries out mutually exchanging probability distributions, but neglects the useful representational knowledge.
no code implementations • 31 Jan 2020 • Chuanguang Yang, Zhulin An, Xiaolong Hu, Hui Zhu, Yongjun Xu
Deep convolutional neural networks (CNN) always depend on wider receptive field (RF) and more complex non-linearity to achieve state-of-the-art performance, while suffering the increased difficult to interpret how relevant patches contribute the final prediction.
no code implementations • 20 Nov 2019 • Xiaolong Hu, Zhulin An, Chuanguang Yang, Hui Zhu, Kaiqaing Xu, Yongjun Xu
For VGG16 pre-trained on ImageNet, our method averagely gains 14. 29\% accuracy promotion for two-classes sub-tasks.
no code implementations • 4 Sep 2019 • Hui Zhu, Zhulin An, Chuanguang Yang, Xiaolong Hu, Kaiqiang Xu, Yongjun Xu
In this paper, we propose a method for efficient automatic architecture search which is special to the widths of networks instead of the connections of neural architecture.
1 code implementation • 26 Aug 2019 • Chuanguang Yang, Zhulin An, Hui Zhu, Xiaolong Hu, Kun Zhang, Kaiqiang Xu, Chao Li, Yongjun Xu
We propose a simple yet effective method to reduce the redundancy of DenseNet by substantially decreasing the number of stacked modules by replacing the original bottleneck by our SMG module, which is augmented by local residual.
Ranked #60 on Image Classification on CIFAR-10
no code implementations • 2 Jun 2019 • Chuanguang Yang, Zhulin An, Chao Li, Boyu Diao, Yongjun Xu
In this work, we propose a heuristic genetic algorithm (GA) for pruning convolutional neural networks (CNNs) according to the multi-objective trade-off among error, computation and sparsity.
1 code implementation • 10 May 2019 • Hui Zhu, Zhulin An, Chuanguang Yang, Kaiqiang Xu, Erhu Zhao, Yongjun Xu
Latest algorithms for automatic neural architecture search perform remarkable but are basically directionless in search space and computational expensive in training of every intermediate architecture.