Associate Professor
(with tenure and with endowed professorship)
Machine Learning Group
School of Software
Tsinghua University
longmingsheng@gmail.com, mingsheng@tsinghua.edu.cn
Room 11-413, East Main Building, Tsinghua University, Beijing 100084, China
My research spans machine learning theory, algorithms and models, with persistent commitment to creating strong learning machines from big data that adapt to the real world. I am working on deep learning and foundation models, scientific learning and world models, transfer learning and model adaptation.
Our Machine Learning Group is interested in powering machine learning for representation, perception, prediction, and generation of big data with a good tradeoff between accuracy, efficiency, generalizability, and transferability. Our mission is to solve open problems and enable major applications of Artificial Intelligence (AI), including physical sciences, hybrid AI systems, and industrial software.“Everything should be made as simple as possible, but no simpler.” --Albert Einstein
PhD in Computer Science, 2008-2014
Department of Computer Science and Technology, Tsinghua University, Beijing, China
Supervisor: Prof. Jianmin Wang
Bachelor in Electrical Engineering, 2004-2008
Department of Electrical Engineering, Tsinghua University, Beijing, China
Postdoc in Machine Learning, 2014-2015
Department of Computer Science, UC Berkeley
Advisor: Prof. Michael I. Jordan
Postdoc in Data Science, 2014-2016
School of Software, Tsinghua University, Beijing, China
Advisor: Prof. Jianmin Wang
Big nowcasting model for extreme precipitation (NowcastNet) was reported in News and Views and published in Nature 2023
Unified forecasting model for worldwide stations (Corrformer) was published as the Cover Article in Nat Mach Intell 2023
Base forecasting model for time series (Autoformer) was ranked 10th of the most influential papers in NeurIPS 2021
Conditional Domain Adversarial Network (CDAN) was ranked 6th of the most influential papers in NeurIPS 2018
Joint Adaptation Network (JAN) was ranked 10th of the most influential papers in ICML 2017
Deep Adaptation Network (DAN) was ranked 5th of the most influential papers in ICML 2015, Test of Time Award at FTL-IJCAI
Joint Distribution Adaptation (JDA) was ranked 2nd of the most influential papers in ICCV 2013
(✉ Corresponding Author)
Mingsheng Long. Transfer Learning: Problems and Methods. 1-127, 2014 [PDF] (In Chinese)
Skilful Nowcasting of Extreme Precipitation with NowcastNet
Yuchen Zhang, Mingsheng Long✉, Kaiyuan Chen, Lanxiang Xing, Ronghua Jin, Michael I. Jordan✉, Jianmin Wang✉
Nature 619, 1-7, 2023 [Link] [Code]
⚑ Hot Paper
⚑ WAIC Youth Outstanding Paper Award
Interpretable Weather Forecasting for Worldwide Stations with a Unified Deep Model
Haixu Wu, Hang Zhou, Mingsheng Long✉, Jianmin Wang✉
Nature Machine Intelligence (Nat Mach Intell) 5, 602-611, 2023 [Link] [Code]
⚑ Cover Article
⚑ WAIC Youth Outstanding Paper Award Honorable Mention
One Fits Many: Class Confusion Loss for Versatile Domain Adaptation
Ying Jin, Zhangjie Cao, Ximei Wang, Jianmin Wang, and Mingsheng Long✉
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2024 [Link] [Code]
Omni-Training: Bridging Pre-Training and Meta-Training for Few-Shot Learning
Yang Shu, Zhangjie Cao, Jinghan Gao, Jianmin Wang, Philip S. Yu, Mingsheng Long✉
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 45(12):15275-15291, 2023 [Link] [Code]
ModeRNN: Harnessing Spatiotemporal Mode Collapse in Unsupervised Predictive Learning
Zhiyu Yao, Yunbo Wang, Haixu Wu, Jianmin Wang, Mingsheng Long✉
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 45(11):13281-13296, 2023 [Link] [Code]
PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning
Yunbo Wang, Haixu Wu, Jianjin Zhang, Zhifeng Gao, Jianmin Wang, Philip S. Yu, Mingsheng Long✉
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 45(2):2208-2225, 2023 [Link] [Code]
⚑ Highly Cited Paper
From Big to Small: Adaptive Learning to Partial-Set Domains
Zhangjie Cao, Kaichao You, Ziyang Zhang, Jianmin Wang, Mingsheng Long✉
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 45(2):1766-1780, 2023 [Link] [Code]
VideoDG: Generalizing Temporal Relations in Videos to Novel Domains
Zhiyu Yao, Yunbo Wang, Jianmin Wang, Philip S. Yu, Mingsheng Long✉
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 44(11):7989-8004, 2022 [Link] [Code]
Ranking and Tuning Pre-trained Models: A New Paradigm for Exploiting Model Hubs
Kaichao You, Yong Liu, Ziyang Zhang, Jianmin Wang, Michael I. Jordan, Mingsheng Long✉
Journal of Machine Learning Research (JMLR), 23(209):1-47, 2022 [Link] [PDF] [Code]
Transferable Representation Learning with Deep Adaptation Networks
Mingsheng Long, Yue Cao, Zhangjie Cao, Jianmin Wang, Michael I. Jordan
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 41(12):3071-3085, 2019 [Link] [Code]
⚑ Highly Cited Paper
Transolver: A Fast Transformer Solver for PDEs on General Geometries
Haixu Wu, Huakun Luo, Haowen Wang, Jianmin Wang, Mingsheng Long✉
International Conference on Machine Learning (ICML), 2024 [arXiv] (Accepted)
⚑ Spotlight Paper
HelmFluid: Learning Helmholtz Dynamics for Interpretable Fluid Prediction
Lanxiang Xing, Haixu Wu, Yuezhou Ma, Jianmin Wang, Mingsheng Long✉
International Conference on Machine Learning (ICML), 2024 [arXiv] (Accepted)
Timer: Generative Pre-trained Transformers are Large Time Series Models
Yong Liu, Haoran Zhang, Chenyu Li, Xiangdong Huang, Jianmin Wang, Mingsheng Long✉
International Conference on Machine Learning (ICML), 2024 [arXiv] (Accepted)
TimeSiam: A Pre-Training Framework for Siamese Time-Series Modeling
Jiaxiang Dong, Haixu Wu, Yuxuan Wang, Yunzhong Qiu, Li Zhang, Jianmin Wang, Mingsheng Long✉
International Conference on Machine Learning (ICML), 2024 [arXiv] (Accepted)
CogDPM: Diffusion Probabilistic Models via Cognitive Predictive Coding
Kaiyuan Chen, Xingzhuo Guo, Yu Zhang, Jianmin Wang, Mingsheng Long✉
International Conference on Machine Learning (ICML), 2024 [arXiv] (Accepted)
HarmonyDream: Task Harmonization Inside World Models
Haoyu Ma, Jialong Wu, Ningya Feng, Chenjun Xiao, Dong Li, Jianye Hao, Jianmin Wang, Mingsheng Long✉
International Conference on Machine Learning (ICML), 2024 [arXiv] (Accepted)
Mobile Attention: Mobile-Friendly Linear-Attention for Vision Transformers
Zhiyu Yao, Jian Wang, Haixu Wu, Jingdong Wang, Mingsheng Long✉
International Conference on Machine Learning (ICML), 2024 (Accepted)
On the Embedding Collapse when Scaling up Recommendation Models
Xingzhuo Guo, Junwei Pan, Ximei Wang, Baixu Chen, Jie Jiang, Mingsheng Long✉
International Conference on Machine Learning (ICML), 2024 [arXiv] (Accepted)
iTransformer: Inverted Transformers Are Effective for Time Series Forecasting
Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, Mingsheng Long✉
International Conference on Learning Representations (ICLR), 2024 [arXiv] (Accepted)
⚑ Spotlight Paper
Efficient ConvBN Blocks for Transfer Learning and Beyond
Kaichao You, Guo Qin, Anchang Bao, Meng Cao, Ping Huang, Jiulong Shan, Mingsheng Long✉
International Conference on Learning Representations (ICLR), 2024 [arXiv] (Accepted)
⚑ Spotlight Paper
Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learning
Jialong Wu, Haoyu Ma, Chaoyi Deng, Mingsheng Long✉
Neural Information Processing Systems (NeurIPS), 2023 [PDF] [arXiv] [Code]
ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learning
Junguang Jiang, Baixu Chen, Junwei Pan, Ximei Wang, Liu Dapeng, Jie Jiang, Mingsheng Long✉
Neural Information Processing Systems (NeurIPS), 2023 [PDF] [arXiv] [Code]
Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictors
Yong Liu, Chenyu Li, Jianmin Wang, Mingsheng Long✉
Neural Information Processing Systems (NeurIPS), 2023 [PDF] [arXiv] [Code]
SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modeling
Jiaxiang Dong, Haixu Wu, Haoran Zhang, Li Zhang, Jianmin Wang, Mingsheng Long✉
Neural Information Processing Systems (NeurIPS), 2023 [PDF] [arXiv] [Code]
⚑ Spotlight Paper
Solving High-Dimensional PDEs with Latent Spectral Models
Haixu Wu, Tengge Hu, Huakun Luo, Jianmin Wang, Mingsheng Long✉
International Conference on Machine Learning (ICML), 2023 [PDF] [arXiv] [Code]
CLIPood: Generalizing CLIP to Out-of-Distributions
Yang Shu, Xingzhuo Guo, Jialong Wu, Ximei Wang, Jianmin Wang, Mingsheng Long✉
International Conference on Machine Learning (ICML), 2023 [PDF] [arXiv] [Code]
Estimating Heterogeneous Treatment Effects: Mutual Information Bounds and Learning Algorithms
Xingzhuo Guo, Yuchen Zhang, Jianmin Wang, Mingsheng Long✉
International Conference on Machine Learning (ICML), 2023 [PDF] [Code]
TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis
Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, Mingsheng Long✉
International Conference on Learning Representations (ICLR), 2023 [PDF] [arXiv] [Code]
Debiased Self-Training for Semi-Supervised Learning
Baixu Chen, Junguang Jiang, Ximei Wang, Pengfei Wan, Jianmin Wang, Mingsheng Long✉
Neural Information Processing Systems (NeurIPS), 2022 [PDF] [arXiv] [Slides] [Code]
⚑ Oral Paper
Hub-Pathway: Transfer Learning from A Hub of Pre-trained Models
Yang Shu, Zhangjie Cao, Ziyang Zhang, Jianmin Wang, Mingsheng Long✉
Neural Information Processing Systems (NeurIPS), 2022 [PDF] [arXiv]
Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting
Yong Liu, Haixu Wu, Jianmin Wang, Mingsheng Long✉
Neural Information Processing Systems (NeurIPS), 2022 [PDF] [arXiv] [Code]
Supported Policy Optimization for Offline Reinforcement Learning
Jialong Wu, Haixu Wu, Zihan Qiu, Jianmin Wang, Mingsheng Long✉
Neural Information Processing Systems (NeurIPS), 2022 [PDF] [arXiv] [Code]
Flowformer: Linearizing Transformers with Conservation Flows
Haixu Wu, Jialong Wu, Jiehui Xu, Jianmin Wang, Mingsheng Long✉
International Conference on Machine Learning (ICML), 2022 [PDF] [arXiv] [Code]
Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy
Jiehui Xu, Haixu Wu, Jianmin Wang, Mingsheng Long✉
International Conference on Learning Representations (ICLR), 2022 [PDF] [arXiv] [Code]
⚑ Spotlight Paper
Decoupled Adaptation for Cross-Domain Object Detection
Junguang Jiang, Baixu Chen, Jianmin Wang, Mingsheng Long✉
International Conference on Learning Representations (ICLR), 2022 [PDF] [arXiv] [Code]
X-model: Improving Data Efficiency in Deep Learning with A Minimax Model
Ximei Wang, Xinyang Chen, Jianmin Wang, Mingsheng Long✉
International Conference on Learning Representations (ICLR), 2022 [PDF] [arXiv] [Code]
Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting
Haixu Wu, Jiehui Xu, Jianmin Wang, Mingsheng Long✉
Neural Information Processing Systems (NeurIPS), 2021 [PDF] [Appendix] [arXiv] [Code]
⚑ Ranks 10th in NeurIPS 2021
Cycle Self-Training for Domain Adaptation
Hong Liu, Jianmin Wang, Mingsheng Long✉
Neural Information Processing Systems (NeurIPS), 2021 [PDF] [Appendix] [Code]
LogME: Practical Assessment of Pre-trained Models for Transfer Learning
Kaichao You, Yong Liu, Jianmin Wang, Mingsheng Long✉
International Conference on Machine Learning (ICML), 2021 [PDF] [Appendix] [Code]
Zoo-Tuning: Adaptive Transfer from A Zoo of Models
Yang Shu, Zhi Kou, Zhangjie Cao, Jianmin Wang, Mingsheng Long✉
International Conference on Machine Learning (ICML), 2021 [PDF] [Appendix] [Code]
Self-Tuning for Data-Efficient Deep Learning
Ximei Wang, Jinghan Gao, Jianmin Wang, Mingsheng Long✉
International Conference on Machine Learning (ICML), 2021 [PDF] [Appendix] [Code]
Representation Subspace Distance for Domain Adaptation Regression
Xinyang Chen, Sinan Wang, Jianmin Wang, Mingsheng Long✉
International Conference on Machine Learning (ICML), 2021 [PDF] [Appendix] [Code]
Transferable Calibration with Lower Bias and Variance in Domain Adaptation
Ximei Wang, Mingsheng Long✉, Jianmin Wang, Michael I. Jordan
Neural Information Processing Systems (NeurIPS), 2020 [PDF] [Appendix] [Code]
Co-Tuning for Transfer Learning
Kaichao You, Zhi Kou, Mingsheng Long✉, Jianmin Wang
Neural Information Processing Systems (NeurIPS), 2020 [PDF] [Code]
Stochastic Normalization
Zhi Kou, Kaichao You, Mingsheng Long✉, Jianmin Wang
Neural Information Processing Systems (NeurIPS), 2020 [PDF] [Code]
Learning to Adapt to Evolving Domains
Hong Liu, Mingsheng Long✉, Jianmin Wang, Yu Wang
Neural Information Processing Systems (NeurIPS), 2020 [PDF] [Code]
Unsupervised Transfer Learning for Spatiotemporal Predictive Networks
Zhiyu Yao, Yunbo Wang, Mingsheng Long✉, Jianmin Wang
International Conference on Machine Learning (ICML), 2020 [PDF] [Code]
Catastrophic Forgetting Meets Negative Transfer: Batch Spectral Shrinkage for Safe Transfer Learning
Xinyang Chen, Sinan Wang, Bo Fu, Mingsheng Long✉, Jianmin Wang
Neural Information Processing Systems (NeurIPS), 2019 [PDF] [Code]
Transferable Normalization: Towards Improving Transferability of Deep Neural Networks
Ximei Wang, Ying Jin, Mingsheng Long✉, Jianmin Wang, Michael I. Jordan
Neural Information Processing Systems (NeurIPS), 2019 [PDF] [Code]
Bridging Theory and Algorithm for Domain Adaptation
Yuchen Zhang, Tianle Liu, Mingsheng Long✉, Michael I. Jordan
International Conference on Machine Learning (ICML), 2019 [PDF] [Appendix] [Code]
⚑ Long Oral Paper
Transferable Adversarial Training: A General Approach to Adapting Deep Classifiers
Hong Liu, Mingsheng Long✉, Jianmin Wang, Michael I. Jordan
International Conference on Machine Learning (ICML), 2019 [PDF] [Code]
⚑ Long Oral Paper
Towards Accurate Model Selection in Deep Unsupervised Domain Adaptation
Kaichao You, Ximei Wang, Mingsheng Long✉, Michael I. Jordan
International Conference on Machine Learning (ICML), 2019 [PDF] [Code]
Transferability vs. Discriminability: Batch Spectral Penalization for Adversarial Domain Adaptation
Xinyang Chen, Sinan Wang, Mingsheng Long✉, Jianmin Wang
International Conference on Machine Learning (ICML), 2019 [PDF] [Code]
Conditional Adversarial Domain Adaptation
Mingsheng Long, Zhangjie Cao, Jianmin Wang, Michael I. Jordan
Neural Information Processing Systems (NeurIPS), 2018 [PDF] [Poster] [Code]
⚑ Ranks 6th in NeurIPS 2018
Generalized Zero-Shot Learning with Deep Calibration Network
Shichen Liu, Mingsheng Long✉, Jianmin Wang, Michael I. Jordan
Neural Information Processing Systems (NeurIPS), 2018 [PDF] [Code]
PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning
Yunbo Wang, Zhifeng Gao, Mingsheng Long✉, Jianmin Wang, Philip S. Yu
International Conference on Machine Learning (ICML), 2018 [PDF] [Code]
PredRNN: Recurrent Neural Networks for Predictive Learning using Spatiotemporal LSTMs
Yunbo Wang, Mingsheng Long✉, Jianmin Wang, Zhifeng Gao, Philip S. Yu
Neural Information Processing Systems (NeurIPS), 2017 [PDF] [Code]
Learning Multiple Tasks with Multilinear Relationship Networks
Mingsheng Long, Zhangjie Cao, Jianmin Wang, Philip S. Yu
Neural Information Processing Systems (NeurIPS), 2017 [PDF] [Poster] [Code]
Deep Transfer Learning with Joint Adaptation Networks
Mingsheng Long, Han Zhu, Jianmin Wang, Michael I. Jordan
International Conference on Machine Learning (ICML), 2017 [PDF] [Slides] [Poster] [Code]
⚑ Ranks 10th in ICML 2017
Unsupervised Domain Adaptation with Residual Transfer Networks
Mingsheng Long, Han Zhu, Jianmin Wang, Michael I. Jordan
Neural Information Processing Systems (NeurIPS), 2016 [PDF] [Poster] [Code]
Learning Transferable Features with Deep Adaptation Networks
Mingsheng Long, Yue Cao, Jianmin Wang, Michael I. Jordan
International Conference on Machine Learning (ICML), 2015
[PDF] [Slides] [Poster] [Code]
⚑ Ranks 5th in ICML 2015
⚑ Test of Time Award at FTL-IJCAI 2021