no code implementations • 3 May 2024 • Kaiyuan Chen, Xingzhuo Guo, Yu Zhang, Jianmin Wang, Mingsheng Long
The precision weighting mechanism posits that the brain allocates more attention to signals with lower precision, contributing to the cognitive ability of human brains.
no code implementations • 24 Apr 2024 • Jialong Wu, Chaoyi Deng, Jianmin Wang, Mingsheng Long
Effective code optimization in compilers plays a central role in computer and software engineering.
1 code implementation • 20 Mar 2024 • Peng Zhou, Jianmin Wang, Chunyan Li, Zixu Wang, Yiping Liu, Siqi Sun, Jianxin Lin, Longyue Wang, Xiangxiang Zeng
While various models and computational tools have been proposed for structure and property analysis of molecules, generating molecules that conform to all desired structures and properties remains a challenge.
1 code implementation • 14 Mar 2024 • Kaichao You, Runsheng Bai, Meng Cao, Jianmin Wang, Ion Stoica, Mingsheng Long
PyTorch \texttt{2. x} introduces a compiler designed to accelerate deep learning programs.
no code implementations • 29 Feb 2024 • Yuxuan Wang, Haixu Wu, Jiaxiang Dong, Yong liu, Yunzhong Qiu, Haoran Zhang, Jianmin Wang, Mingsheng Long
Experimentally, TimeXer significantly improves time series forecasting with exogenous variables and achieves consistent state-of-the-art performance in twelve real-world forecasting benchmarks.
1 code implementation • 4 Feb 2024 • Yong liu, Haoran Zhang, Chenyu Li, Xiangdong Huang, Jianmin Wang, Mingsheng Long
Continuous progresses have been achieved as the emergence of large language models, exhibiting unprecedented ability in few-shot generalization, scalability, and task generality, which is however absent in time series models.
1 code implementation • 4 Feb 2024 • Yong liu, Guo Qin, Xiangdong Huang, Jianmin Wang, Mingsheng Long
Foundation models of time series have not been fully developed due to the limited availability of large-scale time series and the underexploration of scalable pre-training.
no code implementations • 4 Feb 2024 • Qilong Ma, Haixu Wu, Lanxiang Xing, Jianmin Wang, Mingsheng Long
Accurately predicting the future fluid is important to extensive areas, such as meteorology, oceanology and aerodynamics.
no code implementations • 4 Feb 2024 • Jiaxiang Dong, Haixu Wu, Yuxuan Wang, Yunzhong Qiu, Li Zhang, Jianmin Wang, Mingsheng Long
To emphasize temporal correlation modeling, this paper proposes TimeSiam as a simple but effective self-supervised pre-training framework for Time series based on Siamese networks.
no code implementations • 4 Feb 2024 • Haixu Wu, Huakun Luo, Haowen Wang, Jianmin Wang, Mingsheng Long
Transformers have empowered many milestones across various fields and have recently been applied to solve partial differential equations (PDEs).
no code implementations • 16 Oct 2023 • Lanxiang Xing, Haixu Wu, Yuezhou Ma, Jianmin Wang, Mingsheng Long
Compared with previous velocity estimating methods, HelmFluid is faithfully derived from Helmholtz theorem and ravels out complex fluid dynamics with physically interpretable evidence.
no code implementations • 30 Sep 2023 • Haoyu Ma, Jialong Wu, Ningya Feng, Chenjun Xiao, Dong Li, Jianye Hao, Jianmin Wang, Mingsheng Long
Model-based reinforcement learning (MBRL) holds the promise of sample-efficient learning by utilizing a world model, which models how the environment works and typically encompasses components for two tasks: observation modeling and reward modeling.
Ranked #4 on Atari Games 100k on Atari 100k
2 code implementations • NeurIPS 2023 • Yong liu, Chenyu Li, Jianmin Wang, Mingsheng Long
While previous models suffer from complicated series variations induced by changing temporal distribution, we tackle non-stationary time series with modern Koopman theory that fundamentally considers the underlying time-variant dynamics.
1 code implementation • NeurIPS 2023 • Jiaxiang Dong, Haixu Wu, Haoran Zhang, Li Zhang, Jianmin Wang, Mingsheng Long
By relating masked modeling to manifold learning, SimMTM proposes to recover masked time points by the weighted aggregation of multiple neighbors outside the manifold, which eases the reconstruction task by assembling ruined but complementary temporal variations from multiple masked series.
1 code implementation • 2 Feb 2023 • Yang Shu, Xingzhuo Guo, Jialong Wu, Ximei Wang, Jianmin Wang, Mingsheng Long
This paper aims at generalizing CLIP to out-of-distribution test data on downstream tasks.
1 code implementation • 30 Jan 2023 • Haixu Wu, Tengge Hu, Huakun Luo, Jianmin Wang, Mingsheng Long
A burgeoning paradigm is learning neural operators to approximate the input-output mappings of PDEs.
3 code implementations • 5 Oct 2022 • Haixu Wu, Tengge Hu, Yong liu, Hang Zhou, Jianmin Wang, Mingsheng Long
TimesBlock can discover the multi-periodicity adaptively and extract the complex temporal variations from transformed 2D tensors by a parameter-efficient inception block.
no code implementations • 3 Oct 2022 • Zhongyi Pei, Lin Liu, Chen Wang, Jianmin Wang
Today, many industrial processes are undergoing digital transformation, which often requires the integration of well-understood domain models and state-of-the-art machine learning technology in business processes.
no code implementations • 8 Jun 2022 • Yang Shu, Zhangjie Cao, Ziyang Zhang, Jianmin Wang, Mingsheng Long
The proposed framework can be trained end-to-end with the target task-specific loss, where it learns to explore better pathway configurations and exploit the knowledge in pre-trained models for each target datum.
2 code implementations • 28 May 2022 • Yong liu, Haixu Wu, Jianmin Wang, Mingsheng Long
However, their performance can degenerate terribly on non-stationary real-world data in which the joint distribution changes over time.
no code implementations • CVPR 2021 • Chao Huang, Zhangjie Cao, Yunbo Wang, Jianmin Wang, Mingsheng Long
It is a challenging problem due to the substantial geometry shift from simulated to real data, such that most existing 3D models underperform due to overfitting the complete geometries in the source domain.
1 code implementation • 14 Mar 2022 • Zhangjie Cao, Kaichao You, Ziyang Zhang, Jianmin Wang, Mingsheng Long
Still, the common requirement of identical class space shared across domains hinders applications of domain adaptation to partial-set domains.
1 code implementation • 15 Feb 2022 • Baixu Chen, Junguang Jiang, Ximei Wang, Pengfei Wan, Jianmin Wang, Mingsheng Long
Yet these datasets are time-consuming and labor-exhaustive to obtain on realistic tasks.
1 code implementation • 13 Feb 2022 • Haixu Wu, Jialong Wu, Jiehui Xu, Jianmin Wang, Mingsheng Long
By respectively conserving the incoming flow of sinks for source competition and the outgoing flow of sources for sink allocation, Flow-Attention inherently generates informative attentions without using specific inductive biases.
Ranked #4 on D4RL on D4RL
3 code implementations • 13 Feb 2022 • Jialong Wu, Haixu Wu, Zihan Qiu, Jianmin Wang, Mingsheng Long
Policy constraint methods to offline reinforcement learning (RL) typically utilize parameterization or regularization that constrains the policy to perform actions within the support set of the behavior policy.
no code implementations • 18 Jan 2022 • Li Lin, Yixin Cao, Lifu Huang, Shu'ang Li, Xuming Hu, Lijie Wen, Jianmin Wang
To alleviate the knowledge forgetting issue, we design two modules, Im and Gm, for each type of knowledge, which are combined via prompt tuning.
1 code implementation • 15 Jan 2022 • Junguang Jiang, Yang Shu, Jianmin Wang, Mingsheng Long
The success of deep learning algorithms generally depends on large-scale data, while humans appear to have inherent ability of knowledge transfer, by recognizing and applying relevant knowledge from previous learning experiences when encountering and solving unseen tasks.
1 code implementation • 20 Oct 2021 • Kaichao You, Yong liu, Ziyang Zhang, Jianmin Wang, Michael I. Jordan, Mingsheng Long
(2) The best ranked PTM can either be fine-tuned and deployed if we have no preference for the model's architecture or the target PTM can be tuned by the top $K$ ranked PTMs via a Bayesian procedure that we propose.
no code implementations • 14 Oct 2021 • Yang Shu, Zhangjie Cao, Jinghan Gao, Jianmin Wang, Philip S. Yu, Mingsheng Long
While pre-training and meta-training can create deep models powerful for few-shot generalization, we find that pre-training and meta-training focuses respectively on cross-domain transferability and cross-task transferability, which restricts their data efficiency in the entangled settings of domain shift and task shift.
no code implementations • ICLR 2022 • Ximei Wang, Xinyang Chen, Jianmin Wang, Mingsheng Long
To take the power of both worlds, we propose a novel X-model by simultaneously encouraging the invariance to {data stochasticity} and {model stochasticity}.
1 code implementation • 8 Oct 2021 • Zhiyu Yao, Yunbo Wang, Haixu Wu, Jianmin Wang, Mingsheng Long
To this end, we propose ModeRNN, which introduces a novel method to learn structured hidden representations between recurrent states.
3 code implementations • ICLR 2022 • Jiehui Xu, Haixu Wu, Jianmin Wang, Mingsheng Long
Unsupervised detection of anomaly points in time series is a challenging problem, which requires the model to derive a distinguishable criterion.
2 code implementations • ICLR 2022 • Junguang Jiang, Baixu Chen, Jianmin Wang, Mingsheng Long
Besides, previous methods focused on category adaptation but ignored another important part for object detection, i. e., the adaptation on bounding box regression.
no code implementations • 18 Sep 2021 • Yanwei Fu, Feng Li, Paula boned Fustel, Lei Zhao, Lijie Jia, Haojie Zheng, Qiang Sun, Shisong Rong, Haicheng Tang, xiangyang xue, Li Yang, Hong Li, Jiao Xie Wenxuan Wang, Yuan Li, Wei Wang, Yantao Pei, Jianmin Wang, Xiuqi Wu, Yanhua Zheng, Hongxia Tian, Mengwei Gu
The image-level performance of COVID-19 prescreening model in the China-Spain multicenter study achieved an AUC of 0. 913 (95% CI, 0. 898-0. 927), with a sensitivity of 0. 695 (95% CI, 0. 643-0. 748), a specificity of 0. 904 (95% CI, 0. 891 -0. 919), an accuracy of 0. 875(0. 861-0. 889), and a F1 of 0. 611(0. 568-0. 655).
no code implementations • 29 Jun 2021 • Yang Shu, Zhi Kou, Zhangjie Cao, Jianmin Wang, Mingsheng Long
We propose \emph{Zoo-Tuning} to address these challenges, which learns to adaptively transfer the parameters of pretrained models to the target task.
2 code implementations • NeurIPS 2021 • Haixu Wu, Jiehui Xu, Jianmin Wang, Mingsheng Long
Going beyond Transformers, we design Autoformer as a novel decomposition architecture with an Auto-Correlation mechanism.
no code implementations • CVPR 2021 • Bo Fu, Zhangjie Cao, Jianmin Wang, Mingsheng Long
Due to the domain shift, the query selection criteria of prior active learning methods may be ineffective to select the most informative target samples for annotation.
no code implementations • 12 Jun 2021 • Yanwei Fu, Lei Zhao, Haojie Zheng, Qiang Sun, Li Yang, Hong Li, Jiao Xie, xiangyang xue, Feng Li, Yuan Li, Wei Wang, Yantao Pei, Jianmin Wang, Xiuqi Wu, Yanhua Zheng, Hongxia Tian Mengwei Gu1
It is still nontrivial to develop a new fast COVID-19 screening method with the easier access and lower cost, due to the technical and cost limitations of the current testing methods in the medical resource-poor districts.
no code implementations • CVPR 2021 • Chao Huang, Zhangjie Cao, Yunbo Wang, Jianmin Wang, Mingsheng Long
It is a challenging problem due to the substantial geometry shift from simulated to real data, such that most existing 3D models underperform due to overfitting the complete geometries in the source domain.
no code implementations • CVPR 2021 • Yang Shu, Zhangjie Cao, Chenyu Wang, Jianmin Wang, Mingsheng Long
Leveraging datasets available to learn a model with high generalization ability to unseen domains is important for computer vision, especially when the unseen domain's annotated data are unavailable.
3 code implementations • 17 Mar 2021 • Yunbo Wang, Haixu Wu, Jianjin Zhang, Zhifeng Gao, Jianmin Wang, Philip S. Yu, Mingsheng Long
This paper models these structures by presenting PredRNN, a new recurrent network, in which a pair of memory cells are explicitly decoupled, operate in nearly independent transition manners, and finally form unified representations of the complex environment.
Ranked #1 on Video Prediction on KTH (Cond metric)
2 code implementations • CVPR 2021 • Junguang Jiang, Yifei Ji, Ximei Wang, Yufeng Liu, Jianmin Wang, Mingsheng Long
First, based on our observation that the probability density of the output space is sparse, we introduce a spatial probability distribution to describe this sparsity and then use it to guide the learning of the adversarial regressor.
1 code implementation • NeurIPS 2021 • Hong Liu, Jianmin Wang, Mingsheng Long
In the forward step, CST generates target pseudo-labels with a source-trained classifier.
1 code implementation • CVPR 2021 • Haixu Wu, Zhiyu Yao, Jianmin Wang, Mingsheng Long
With high flexibility, this framework can adapt to a series of models for deterministic spatiotemporal prediction.
2 code implementations • 25 Feb 2021 • Ximei Wang, Jinghan Gao, Mingsheng Long, Jianmin Wang
Deep learning has made revolutionary advances to diverse applications in the presence of large-scale labeled datasets.
1 code implementation • 22 Feb 2021 • Kaichao You, Yong liu, Jianmin Wang, Mingsheng Long
In pursuit of a practical assessment method, we propose to estimate the maximum value of label evidence given features extracted by pre-trained models.
Ranked #3 on Transferability on classification benchmark
no code implementations • 23 Dec 2020 • Xiaohe Li, Lijie Wen, Chen Qian, Jianmin Wang
Heterogeneous network embedding aims to embed nodes into low-dimensional vectors which capture rich intrinsic information of heterogeneous networks.
2 code implementations • NeurIPS 2020 • Zhi Kou, Kaichao You, Mingsheng Long, Jianmin Wang
During training, two branches are stochastically selected to avoid over-depending on some sample statistics, resulting in a strong regularization effect, which we interpret as ``architecture regularization.''
2 code implementations • NeurIPS 2020 • Kaichao You, Zhi Kou, Mingsheng Long, Jianmin Wang
Fine-tuning pre-trained deep neural networks (DNNs) to a target dataset, also known as transfer learning, is widely used in computer vision and NLP.
Ranked #1 on Transfer Learning on COCO70
1 code implementation • NeurIPS 2020 • Hong Liu, Mingsheng Long, Jianmin Wang, Yu Wang
(2) Since the target data arrive online, the agent should also maintain competence on previous target domains, i. e. to adapt without forgetting.
no code implementations • 12 Nov 2020 • Jincheng Zhong, Ximei Wang, Zhi Kou, Jianmin Wang, Mingsheng Long
It is common within the deep learning community to first pre-train a deep neural network from a large-scale dataset and then fine-tune the pre-trained model to a specific downstream task.
1 code implementation • 8 Dec 2019 • Zhiyu Yao, Yunbo Wang, Jianmin Wang, Philip S. Yu, Mingsheng Long
This paper introduces video domain generalization where most video classification networks degenerate due to the lack of exposure to the target domains of divergent distributions.
2 code implementations • International Conference on Machine Learning 2019 • Xinyang Chen, Sinan Wang, Mingsheng Long, Jianmin Wang
In this paper, a series of experiments based on spectral analysis of the feature representations have been conducted, revealing an unexpected deterioration of the discriminability while learning transferable features adversarially.