no code implementations • 2 May 2024 • Tianyi Chen, Zhi-Qin John Xu
In scientific applications, the scale of neural networks is generally moderate-size, mainly to ensure the speed of inference during application.
1 code implementation • 12 Apr 2024 • Tianyu Ding, Jinxin Zhou, Tianyi Chen, Zhihui Zhu, Ilya Zharkov, Luming Liang
Existing angle-based contour descriptors suffer from lossy representation for non-starconvex shapes.
no code implementations • 11 Apr 2024 • Guangzhi Wang, Tianyi Chen, Kamran Ghasedi, HsiangTao Wu, Tianyu Ding, Chris Nuesmeyer, Ilya Zharkov, Mohan Kankanhalli, Luming Liang
S3Editor is model-agnostic and compatible with various editing approaches.
no code implementations • 9 Mar 2024 • Spencer Hutchinson, Tianyi Chen, Mahnoosh Alizadeh
In this work, we consider a version of this problem with static linear constraints that the player receives noisy feedback of and must always satisfy.
no code implementations • 10 Feb 2024 • Han Shen, Zhuoran Yang, Tianyi Chen
But bilevel problems such as incentive design, inverse reinforcement learning (RL), and RL from human feedback (RLHF) are often modeled as dynamic objective functions that go beyond the simple static objective structures, which pose significant challenges of using existing bilevel solutions.
2 code implementations • 6 Feb 2024 • Jongwoo Ko, Sungnyun Kim, Tianyi Chen, Se-Young Yun
Knowledge distillation (KD) is widely used for compressing a teacher model to a smaller student model, reducing its inference cost and memory footprint while preserving model capabilities.
1 code implementation • 22 Jan 2024 • Momin Abbas, Yi Zhou, Parikshit Ram, Nathalie Baracaldo, Horst Samulowitz, Theodoros Salonidis, Tianyi Chen
However, applying ICL in real cases does not scale with the number of samples, and lacks robustness to different prompt templates and demonstration permutations.
1 code implementation • 13 Jan 2024 • A F M Saif, Xiaodong Cui, Han Shen, Songtao Lu, Brian Kingsbury, Tianyi Chen
In this paper, we present a novel bilevel optimization-based training approach to training acoustic models for automatic speech recognition (ASR) tasks that we term {bi-level joint unsupervised and supervised training (BL-JUST)}.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +2
1 code implementation • 7 Jan 2024 • Tianyi Chen, Nan Hao, Yingzhou Lu, Capucine van Rechem
Selective classification, encompassing a spectrum of methods for uncertainty quantification, empowers the model to withhold decision-making in the face of samples marked by ambiguity or low confidence, thereby amplifying the accuracy of predictions for the instances it chooses to classify.
1 code implementation • 15 Dec 2023 • Tianyi Chen, Tianyu Ding, Zhihui Zhu, Zeyu Chen, HsiangTao Wu, Ilya Zharkov, Luming Liang
Compressing a predefined deep neural network (DNN) into a compact sub-network with competitive performance is crucial in the efficient machine learning realm.
no code implementations • 3 Dec 2023 • Fan Yang, Tianyi Chen, Xiaosheng He, Zhongang Cai, Lei Yang, Si Wu, Guosheng Lin
We propose AttriHuman-3D, an editable 3D human generation model, which address the aforementioned problems with attribute decomposition and indexing.
1 code implementation • 1 Dec 2023 • Tianyu Ding, Tianyi Chen, Haidong Zhu, Jiachen Jiang, Yiqi Zhong, Jinxin Zhou, Guangzhi Wang, Zhihui Zhu, Ilya Zharkov, Luming Liang
The rapid growth of Large Language Models (LLMs) has been a driving force in transforming various domains, reshaping the artificial general intelligence landscape.
1 code implementation • 30 Nov 2023 • Jinxin Zhou, Tianyu Ding, Tianyi Chen, Jiachen Jiang, Ilya Zharkov, Zhihui Zhu, Luming Liang
We present DREAM, a novel training framework representing Diffusion Rectification and Estimation Adaptive Models, requiring minimal code changes (just three lines) yet significantly enhancing the alignment of training with sampling in diffusion models.
1 code implementation • 27 Nov 2023 • Haidong Zhu, Tianyu Ding, Tianyi Chen, Ilya Zharkov, Ram Nevatia, Luming Liang
CaesarNeRF explicitly models pose differences of reference views to combine scene-level semantic representations, providing a calibrated holistic understanding.
1 code implementation • 24 Oct 2023 • Tianyi Chen, Tianyu Ding, Badal Yadav, Ilya Zharkov, Luming Liang
Large Language Models (LLMs) have transformed the landscape of artificial intelligence, while their enormous size presents significant challenges in terms of computational costs.
no code implementations • 27 Jun 2023 • Zhehua Zhong, Tianyi Chen, Zhen Wang
Fine-tuning large-scale pre-trained language models has been demonstrated effective for various natural language processing (NLP) tasks.
no code implementations • 4 Jun 2023 • Quan Xiao, Songtao Lu, Tianyi Chen
Bilevel optimization has recently regained interest owing to its applications in emerging machine learning fields such as hyperparameter optimization, meta-learning, and reinforcement learning.
1 code implementation • 25 May 2023 • Tianyi Chen, Luming Liang, Tianyu Ding, Ilya Zharkov
To search an optimal sub-network within a general deep neural network (DNN), existing neural architecture search (NAS) methods typically rely on handcrafting a search space beforehand.
1 code implementation • 17 Apr 2023 • Chaoyue Song, Tianyi Chen, YiWen Chen, Jiacheng Wei, Chuan Sheng Foo, Fayao Liu, Guosheng Lin
To solve this problem, we propose neural dual quaternion blend skinning (NeuDBS) to achieve 3D point deformation, which can perform rigid transformation without skin-collapsing artifacts.
1 code implementation • 13 Mar 2023 • Tianyi Chen, Luming Liang, Tianyu Ding, Zhihui Zhu, Ilya Zharkov
We propose the second generation of Only-Train-Once (OTOv2), which first automatically trains and compresses a general DNN only once from scratch to produce a more compact model with competitive performance without fine-tuning.
1 code implementation • 10 Feb 2023 • Han Shen, Quan Xiao, Tianyi Chen
Bilevel optimization enjoys a wide range of applications in hyper-parameter optimization, meta-learning and reinforcement learning.
no code implementations • CVPR 2023 • Yunfei Zhang, Xiaoyang Huo, Tianyi Chen, Si Wu, Hau San Wong
Semi-supervised class-conditional image synthesis is typically performed by inferring and injecting class labels into a conditional Generative Adversarial Network (GAN).
no code implementations • 17 Dec 2022 • Bo Ji, Tianyi Chen
Convolution neural networks (CNNs) have achieved remarkable success, but typically accompany high computation cost and numerous redundant weight parameters.
1 code implementation • 14 Nov 2022 • Quan Xiao, Han Shen, Wotao Yin, Tianyi Chen
By leveraging the special structure of the equality constraints problem, the paper first presents an alternating implicit projected SGD approach and establishes the $\tilde{\cal O}(\epsilon^{-2})$ sample complexity that matches the state-of-the-art complexity of ALSET \citep{chen2021closing} for unconstrained bilevel problems.
1 code implementation • 23 Oct 2022 • Heshan Fernando, Han Shen, Miao Liu, Subhajit Chaudhury, Keerthiram Murugesan, Tianyi Chen
Machine learning problems with multiple objective functions appear either in learning with multiple criteria where learning has to make a trade-off between multiple performance metrics such as fairness, safety and accuracy; or, in multi-task learning where multiple tasks are optimized jointly, sharing inductive bias between them.
no code implementations • 3 Oct 2022 • Lisha Chen, Sharu Theresa Jose, Ivana Nikoloska, Sangwoo Park, Tianyi Chen, Osvaldo Simeone
This review monograph provides an introduction to meta-learning by covering principles, algorithms, theory, and engineering applications.
no code implementations • 25 Sep 2022 • Youbang Sun, Heshan Fernando, Tianyi Chen, Shahin Shahrampour
We consider the open federated learning (FL) systems, where clients may join and/or leave the system during the FL process.
1 code implementation • 9 Sep 2022 • Tianyu Ding, Luming Liang, Zhihui Zhu, Tianyi Chen, Ilya Zharkov
As a result, we achieve a considerable performance gain with a quarter of the size of the original AdaCoF.
no code implementations • 1 Jul 2022 • Duowei Li, Jianping Wu, Feng Zhu, Tianyi Chen, Yiik Diew Wong
The simulation results demonstrate that the model is able to: (1) achieve satisfactory convergence performances; (2) adaptively determine platoon size in response to varying traffic conditions; and (3) completely avoid deadlocks at the intersection.
no code implementations • 27 Jun 2022 • Lisha Chen, Songtao Lu, Tianyi Chen
While the conventional statistical learning theory suggests that overparameterized models tend to overfit, empirical evidence reveals that overparameterized meta learning methods still work well -- a phenomenon often called "benign overfitting."
no code implementations • 24 Jun 2022 • Duowei Li, Jianping Wu, Feng Zhu, Tianyi Chen, Yiik Diew Wong
As a strategy to reduce travel delay and enhance energy efficiency, platooning of connected and autonomous vehicles (CAVs) at non-signalized intersections has become increasingly popular in academia.
no code implementations • 21 Jun 2022 • Han Shen, Tianyi Chen
Stochastic approximation (SA) with multiple coupled sequences has found broad applications in machine learning such as bilevel learning and reinforcement learning (RL).
no code implementations • 14 Jun 2022 • Quan Xiao, Qing Ling, Tianyi Chen
A major challenge of applying zeroth-order (ZO) methods is the high query complexity, especially when queries are costly.
1 code implementation • 8 Jun 2022 • Momin Abbas, Quan Xiao, Lisha Chen, Pin-Yu Chen, Tianyi Chen
Model-agnostic meta learning (MAML) is currently one of the dominating approaches for few-shot meta-learning.
1 code implementation • 5 Apr 2022 • Xuanang Xu, Hannah H. Deng, Tianyi Chen, Tianshu Kuang, Joshua C. Barber, Daeseung Kim, Jaime Gateno, James J. Xia, Pingkun Yan
In this paper, we first conduct a theoretical analysis on the FL algorithm to reveal the problem of model aggregation during training on non-iid data.
1 code implementation • 6 Mar 2022 • Lisha Chen, Tianyi Chen
In this paper, we aim to provide theoretical justifications for Bayesian MAML's advantageous performance by comparing the meta test risks of MAML and Bayesian MAML.
1 code implementation • IJCAI 2021 • Lianghao Xia, Chao Huang, Yong Xu, Peng Dai, Liefeng Bo, Xiyue Zhang, Tianyi Chen
Crime prediction is crucial for public safety and resource optimization, yet is very challenging due to two aspects: i) the dynamics of criminal patterns across time and space, crime events are distributed unevenly on both spatial and temporal domains; ii) time-evolving dependencies between different types of crimes (e. g., Theft, Robbery, Assault, Damage) which reveal fine-grained semantics of crimes.
no code implementations • CVPR 2022 • Tianyi Chen, Yunfei Zhang, Xiaoyang Huo, Si Wu, Yong Xu, Hau San Wong
To reduce the dependence of generative models on labeled data, we propose a semi-supervised hyper-spherical GAN for class-conditional fine-grained image generation, and our model is referred to as SphericGAN.
1 code implementation • NeurIPS 2021 • Xiao Jin, Pin-Yu Chen, Chia-Yi Hsu, Chia-Mu Yu, Tianyi Chen
We name our proposed method as catastrophic data leakage in vertical federated learning (CAFE).
no code implementations • NeurIPS 2021 • Tianyi Chen, Yuejiao Sun, Wotao Yin
By leveraging the hidden smoothness of the problem, this paper presents a tighter analysis of ALSET for stochastic nested problems.
1 code implementation • 26 Oct 2021 • Xiao Jin, Pin-Yu Chen, Chia-Yi Hsu, Chia-Mu Yu, Tianyi Chen
We name our proposed method as catastrophic data leakage in vertical federated learning (CAFE).
no code implementations • 11 Oct 2021 • Siliang Zeng, Tianyi Chen, Alfredo Garcia, Mingyi Hong
The flexibility in our design allows the proposed MARL-CAC algorithm to be used in a {\it fully decentralized} setting, where the agents can only communicate with their neighbors, as well as a {\it federated} setting, where the agents occasionally communicate with a server while optimizing their (partially personalized) local models.
1 code implementation • NeurIPS 2021 • Tianyi Chen, Bo Ji, Tianyu Ding, Biyi Fang, Guanyi Wang, Zhihui Zhu, Luming Liang, Yixin Shi, Sheng Yi, Xiao Tu
Structured pruning is a commonly used technique in deploying deep neural networks (DNNs) onto resource-constrained devices.
no code implementations • 25 Jun 2021 • Tianyi Chen, Yuejiao Sun, Wotao Yin
By leveraging the hidden smoothness of the problem, this paper presents a tighter analysis of ALSET for stochastic nested problems.
no code implementations • CVPR 2021 • Yi Liu, Xiaoyang Huo, Tianyi Chen, Xiangping Zeng, Si Wu, Zhiwen Yu, Hau-San Wong
Semi-supervised generative learning (SSGL) makes use of unlabeled data to achieve a trade-off between the data collection/annotation effort and generation performance, when adequate labeled data are not available.
no code implementations • 28 Mar 2021 • Tianyi Chen, Meng Wang, Siyuan Gong, Yang Zhou, Bin Ran
In this study, we propose a rotation-based connected automated vehicle (CAV) distributed cooperative control strategy for an on-ramp merging scenario.
no code implementations • 9 Feb 2021 • Tianyi Chen, Yuejiao Sun, Quan Xiao, Wotao Yin
This paper develops a new optimization method for a class of stochastic bilevel problems that we term Single-Timescale stochAstic BiLevEl optimization (STABLE) method.
no code implementations • ICCV 2021 • Tianyi Chen, Yi Liu, Yunfei Zhang, Si Wu, Yong Xu, Feng Liangbing, Hau San Wong
To ensure disentanglement among the variables, we maximize mutual information between the class-independent variable and synthesized images, map real images to the latent space of a generator to perform consistency regularization of cross-class attributes, and incorporate class semantic-based regularization into a discriminator's feature space.
no code implementations • 1 Jan 2021 • Xiao Jin, Ruijie Du, Pin-Yu Chen, Tianyi Chen
In this paper, we revisit this defense premise and propose an advanced data leakage attack to efficiently recover batch data from the shared aggregated gradients.
no code implementations • 1 Jan 2021 • Tianyi Chen, Guanyi Wang, Tianyu Ding, Bo Ji, Sheng Yi, Zhihui Zhu
Optimizing with group sparsity is significant in enhancing model interpretability in machining learning applications, e. g., feature selection, compressed sensing and model compression.
1 code implementation • 31 Dec 2020 • Tianyi Chen, Ziye Guo, Yuejiao Sun, Wotao Yin
This paper proposes an adaptive stochastic gradient descent method for distributed machine learning, which can be viewed as the communication-adaptive counterpart of the celebrated Adam method - justifying its name CADA.
no code implementations • 31 Dec 2020 • Han Shen, Kaiqing Zhang, Mingyi Hong, Tianyi Chen
Asynchronous and parallel implementation of standard reinforcement learning (RL) algorithms is a key enabler of the tremendous success of modern RL.
no code implementations • 22 Dec 2020 • Xinwei Zhang, Wotao Yin, Mingyi Hong, Tianyi Chen
To the best of our knowledge, this is the first formulation and algorithm developed for the hybrid FL.
no code implementations • 24 Nov 2020 • Xiupeng Shi, Yiik Diew Wong, Chen Chai, Michael Zhi-Feng Li, Tianyi Chen, Zeng Zeng
Secondly, we propose balanced Silhouette Index (bSI) to evaluate the internal quality of imbalanced clustering.
no code implementations • 10 Nov 2020 • Tianyi Chen, Bo Ji, Yixin Shi, Tianyu Ding, Biyi Fang, Sheng Yi, Xiao Tu
The compression of deep neural networks (DNNs) to reduce inference cost becomes increasingly important to meet realistic deployment requirements of various applications.
no code implementations • 25 Oct 2020 • Carey E. Priebe, Cencheng Shen, Ningyuan Huang, Tianyi Chen
Neural networks have achieved remarkable successes in machine learning tasks.
2 code implementations • 17 Sep 2020 • Jie Peng, Zhaoxian Wu, Qing Ling, Tianyi Chen
We prove that the proposed method reaches a neighborhood of the optimal solution at a linear convergence rate and the learning error is determined by the number of Byzantine workers.
no code implementations • 25 Aug 2020 • Tianyi Chen, Yuejiao Sun, Wotao Yin
In particular, we apply Adam to SCSC, and the exhibited rate of convergence matches that of the original Adam on non-compositional stochastic optimization.
1 code implementation • 21 Jul 2020 • Jinxiu Liang, Jingwen Wang, Yuhui Quan, Tianyi Chen, Jiaying Liu, Haibin Ling, Yong Xu
REG produces progressively and efficiently intermediate images corresponding to various exposure settings, and such pseudo-exposures are then fused by MED to detect faces across different lighting conditions.
no code implementations • 12 Jul 2020 • Tianyi Chen, Xiao Jin, Yuejiao Sun, Wotao Yin
Horizontal Federated learning (FL) handles multi-client data that share the same set of features, and vertical FL trains a better predictor that combine all the features from different clients.
no code implementations • 17 Jun 2020 • Yanjie Dong, Georgios B. Giannakis, Tianyi Chen, Julian Cheng, Md. Jahangir Hossain, Victor C. M. Leung
For strongly convex loss functions, FRPG and LFRPG have provably faster convergence rates than a benchmark robust stochastic aggregation algorithm.
1 code implementation • 7 Apr 2020 • Tianyi Chen, Tianyu Ding, Bo Ji, Guanyi Wang, Jing Tian, Yixin Shi, Sheng Yi, Xiao Tu, Zhihui Zhu
Sparsity-inducing regularization problems are ubiquitous in machine learning applications, ranging from feature selection to model compression.
1 code implementation • 26 Feb 2020 • Tianyi Chen, Yuejiao Sun, Wotao Yin
The new algorithms adaptively choose between fresh and stale stochastic gradients and have convergence rates comparable to the original SGD.
no code implementations • 20 Feb 2020 • Tao Sun, Han Shen, Tianyi Chen, Dongsheng Li
Typically, the performance of TD(0) and TD($\lambda$) is very sensitive to the choice of stepsizes.
no code implementations • 29 Dec 2019 • Zhaoxian Wu, Qing Ling, Tianyi Chen, Georgios B. Giannakis
This motivates us to reduce the variance of stochastic gradients as a means of robustifying SGD in the presence of Byzantine attacks.
1 code implementation • NeurIPS 2019 • Jun Sun, Tianyi Chen, Georgios B. Giannakis, Zaiyue Yang
The present paper develops a novel aggregated gradient approach for distributed machine learning that adaptively compresses the gradient communication.
1 code implementation • 9 Sep 2019 • Weiyu Li, Tianyi Chen, Liping Li, Zhaoxian Wu, Qing Ling
Specifically, in CSGD, the latest mini-batch stochastic gradient at a worker will be transmitted to the server if and only if it is sufficiently informative.
1 code implementation • 27 Jul 2019 • Bo Ji, Tianyi Chen
The main features of the new framework include: (i) A discriminator consists of an integrated CNN-Long-Short-Term- Memory (LSTM) based feature extraction with Path Signature Features (PSF) as input and a Feedforward Neural Network (FNN) based binary classifier; (ii) A recurrent latent variable model as generator for synthesizing sequential handwritten data.
no code implementations • 7 Dec 2018 • Tianyi Chen, Kaiqing Zhang, Georgios B. Giannakis, Tamer Başar
This paper deals with distributed policy optimization in reinforcement learning, which involves a central controller and a group of learners.
1 code implementation • 9 Nov 2018 • Liping Li, Wei Xu, Tianyi Chen, Georgios B. Giannakis, Qing Ling
In this paper, we propose a class of robust stochastic subgradient methods for distributed learning from heterogeneous datasets at presence of an unknown number of Byzantine workers.
no code implementations • 9 Jul 2018 • Bingcong Li, Tianyi Chen, Georgios B. Giannakis
This paper deals with bandit online learning problems involving feedback of unknown delay that can emerge in multi-armed bandit (MAB) and bandit convex optimization (BCO) settings.
1 code implementation • NeurIPS 2018 • Tianyi Chen, Georgios B. Giannakis, Tao Sun, Wotao Yin
This paper presents a new class of gradient methods for distributed machine learning that adaptively skip the gradient calculations to learn with reduced communication and computation.
no code implementations • 9 May 2018 • Bingcong Li, Tianyi Chen, Georgios B. Giannakis
To accommodate heterogeneous tasks in Internet of Things (IoT), a new communication and computing paradigm termed mobile edge computing emerges that extends computing services from the cloud to edge, but at the same time exposes new challenges on security.
no code implementations • 28 Dec 2017 • Yanning Shen, Tianyi Chen, Georgios B. Giannakis
To further boost performance in dynamic environments, an adaptive multi-kernel learning scheme (termed AdaRaker) is developed.
no code implementations • 27 Jul 2017 • Tianyi Chen, Georgios B. Giannakis
Tailored for such human-in-the-loop systems where the loss functions are hard to model, a family of bandit online saddle-point (BanSaP) schemes are developed, which adaptively adjust the online operations based on (possibly multiple) bandit feedback of the loss functions, and the changing environment.
no code implementations • 14 Jan 2017 • Tianyi Chen, Qing Ling, Georgios B. Giannakis
Performance of an online algorithm in this setting is assessed by: i) the difference of its losses relative to the best dynamic solution with one-slot-ahead information of the loss function and the constraint (that is here termed dynamic regret); and, ii) the accumulated amount of constraint violations (that is here termed dynamic fit).
no code implementations • 7 Oct 2016 • Tianyi Chen, Aryan Mokhtari, Xin Wang, Alejandro Ribeiro, Georgios B. Giannakis
Existing approaches to resource allocation for nowadays stochastic networks are challenged to meet fast convergence and tolerable delay requirements.