Mingsheng Long


Mingsheng Long

Associate Professor
(with tenure and with endowed professorship)

Machine Learning Group
School of Software
Tsinghua University

longmingsheng@gmail.com, mingsheng@tsinghua.edu.cn
Room 11-413, East Main Building, Tsinghua University, Beijing 100084, China

[Publications] [Google Scholar] [THUML @ GitHub] [Subscription @ WeChat] [Instruction to Students]

Biography


My research spans machine learning theory, algorithms and models, with persistent dedication to create strong learning machines that adapt to complex real world. I am actively working on transfer learning and domain adaptation, deep learning and foundation models, scientific learning and world models.

Our Machine Learning Group (THUML) is interested in empowering machine learning for representation, perception, prediction, and decision making with a good tradeoff between accuracy, efficiency, generalizability, and transferability. Our mission is to solve open problems and enable major applications of Artificial Intelligence, including physical sciences, industrial IoT, and intelligent software.

My Quote

Everything should be made as simple as possible, but no simpler.” --Albert Einstein

Research Interests


Education


Research Experience


Highlights


Selected Publications [Full List] [DBLP] [Google Scholar]

(✉ Corresponding Author)


Thesis

  1. Mingsheng Long. Transfer Learning: Problems and Methods. 1-127, 2014 [PDF] (In Chinese)

Journal Articles

  1. Skilful Nowcasting of Extreme Precipitation with NowcastNet
    Yuchen Zhang, Mingsheng Long, Kaiyuan Chen, Lanxiang Xing, Ronghua Jin, Michael I. Jordan, Jianmin Wang
    Nature 619, 1-7, 2023 [Link] [Code]

  2. Interpretable Weather Forecasting for Worldwide Stations with a Unified Deep Model
    Haixu Wu, Hang Zhou, Mingsheng Long, Jianmin Wang
    Nature Machine Intelligence (Nat. Mach. Intell.) 5, 602-611, 2023 [Link] [Code]
    Cover Article

  3. Omni-Training: Bridging Pre-Training and Meta-Training for Few-Shot Learning
    Yang Shu, Zhangjie Cao, Jinghan Gao, Jianmin Wang, Philip S. Yu, Mingsheng Long
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 45(12):15275-15291, 2023 [Link] [Code]

  4. ModeRNN: Harnessing Spatiotemporal Mode Collapse in Unsupervised Predictive Learning
    Zhiyu Yao, Yunbo Wang, Haixu Wu, Jianmin Wang, Mingsheng Long
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 45(11):13281-13296, 2023 [Link] [Code]

  5. PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning
    Yunbo Wang, Haixu Wu, Jianjin Zhang, Zhifeng Gao, Jianmin Wang, Philip S. Yu, Mingsheng Long
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 45(2):2208-2225, 2023 [Link] [Code]
    Highly Cited Paper

  6. From Big to Small: Adaptive Learning to Partial-Set Domains
    Zhangjie Cao, Kaichao You, Ziyang Zhang, Jianmin Wang, Mingsheng Long
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 45(2):1766-1780, 2023 [Link] [Code]

  7. VideoDG: Generalizing Temporal Relations in Videos to Novel Domains
    Zhiyu Yao, Yunbo Wang, Jianmin Wang, Philip S. Yu, Mingsheng Long
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 44(11):7989-8004, 2022 [Link] [Code]

  8. Ranking and Tuning Pre-trained Models: A New Paradigm for Exploiting Model Hubs
    Kaichao You, Yong Liu, Ziyang Zhang, Jianmin Wang, Michael I. Jordan, Mingsheng Long
    Journal of Machine Learning Research (JMLR), 23(209):1-47, 2022 [Link] [PDF] [Code]

  9. Transferable Representation Learning with Deep Adaptation Networks
    Mingsheng Long, Yue Cao, Zhangjie Cao, Jianmin Wang, Michael I. Jordan
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 41(12):3071-3085, 2019 [Link] [Code]
    Highly Cited Paper

Conference Proceedings

  1. iTransformer: Inverted Transformers Are Effective for Time Series Forecasting
    Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, Mingsheng Long
    International Conference on Learning Representations (ICLR), 2024 [arXiv] (Accepted)
    Spotlight Paper

  2. Efficient ConvBN Blocks For Transfer Learning and Beyond
    Kaichao You, Guo Qin, Anchang Bao, Meng Cao, Ping Huang, Jiulong Shan, Mingsheng Long
    International Conference on Learning Representations (ICLR), 2024 [arXiv] (Accepted)
    Spotlight Paper

  3. Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learning
    Jialong Wu, Haoyu Ma, Chaoyi Deng, Mingsheng Long
    Neural Information Processing Systems (NeurIPS), 2023 [PDF] [arXiv] [Code]

  4. ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learning
    Junguang Jiang, Baixu Chen, Junwei Pan, Ximei Wang, Liu Dapeng, Jie Jiang, Mingsheng Long
    Neural Information Processing Systems (NeurIPS), 2023 [PDF] [arXiv] [Code]

  5. Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictors
    Yong Liu, Chenyu Li, Jianmin Wang, Mingsheng Long
    Neural Information Processing Systems (NeurIPS), 2023 [PDF] [arXiv] [Code]

  6. SimMTM: A Simple Pre-Training Framework for Masked Time-Series Modeling
    Jiaxiang Dong, Haixu Wu, Haoran Zhang, Li Zhang, Jianmin Wang, Mingsheng Long
    Neural Information Processing Systems (NeurIPS), 2023 [PDF] [arXiv] [Code]
    Spotlight Paper

  7. Solving High-Dimensional PDEs with Latent Spectral Models
    Haixu Wu, Tengge Hu, Huakun Luo, Jianmin Wang, Mingsheng Long
    International Conference on Machine Learning (ICML), 2023 [PDF] [arXiv] [Code]

  8. CLIPood: Generalizing CLIP to Out-of-Distributions
    Yang Shu, Xingzhuo Guo, Jialong Wu, Ximei Wang, Jianmin Wang, Mingsheng Long
    International Conference on Machine Learning (ICML), 2023 [PDF] [arXiv] [Code]

  9. Estimating Heterogeneous Treatment Effects: Mutual Information Bounds and Learning Algorithms
    Xingzhuo Guo, Yuchen Zhang, Jianmin Wang, Mingsheng Long
    International Conference on Machine Learning (ICML), 2023 [PDF] [Code]

  10. TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis
    Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, Mingsheng Long
    International Conference on Learning Representations (ICLR), 2023 [PDF] [arXiv] [Code]

  11. Debiased Self-Training for Semi-Supervised Learning
    Baixu Chen, Junguang Jiang, Ximei Wang, Pengfei Wan, Jianmin Wang, Mingsheng Long
    Neural Information Processing Systems (NeurIPS), 2022 [PDF] [arXiv] [Slides] [Code]
    Oral Paper

  12. Hub-Pathway: Transfer Learning from A Hub of Pre-trained Models
    Yang Shu, Zhangjie Cao, Ziyang Zhang, Jianmin Wang, Mingsheng Long
    Neural Information Processing Systems (NeurIPS), 2022 [PDF] [arXiv]

  13. Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting
    Yong Liu, Haixu Wu, Jianmin Wang, Mingsheng Long
    Neural Information Processing Systems (NeurIPS), 2022 [PDF] [arXiv] [Code]

  14. Supported Policy Optimization for Offline Reinforcement Learning
    Jialong Wu, Haixu Wu, Zihan Qiu, Jianmin Wang, Mingsheng Long
    Neural Information Processing Systems (NeurIPS), 2022 [PDF] [arXiv] [Code]

  15. Flowformer: Linearizing Transformers with Conservation Flows
    Haixu Wu, Jialong Wu, Jiehui Xu, Jianmin Wang, Mingsheng Long
    International Conference on Machine Learning (ICML), 2022 [PDF] [arXiv] [Code]

  16. Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy
    Jiehui Xu, Haixu Wu, Jianmin Wang, Mingsheng Long
    International Conference on Learning Representations (ICLR), 2022 [PDF] [arXiv] [Code]
    Spotlight Paper

  17. Decoupled Adaptation for Cross-Domain Object Detection
    Junguang Jiang, Baixu Chen, Jianmin Wang, Mingsheng Long
    International Conference on Learning Representations (ICLR), 2022 [PDF] [arXiv] [Code]

  18. X-model: Improving Data Efficiency in Deep Learning with A Minimax Model
    Ximei Wang, Xinyang Chen, Jianmin Wang, Mingsheng Long
    International Conference on Learning Representations (ICLR), 2022 [PDF] [arXiv] [Code]

  19. Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting
    Haixu Wu, Jiehui Xu, Jianmin Wang, Mingsheng Long
    Neural Information Processing Systems (NeurIPS), 2021 [PDF] [Appendix] [arXiv] [Code]
    Ranks 14th in NeurIPS 2021

  20. Cycle Self-Training for Domain Adaptation
    Hong Liu, Jianmin Wang, Mingsheng Long
    Neural Information Processing Systems (NeurIPS), 2021 [PDF] [Appendix] [Code]

  21. LogME: Practical Assessment of Pre-trained Models for Transfer Learning
    Kaichao You, Yong Liu, Jianmin Wang, Mingsheng Long
    International Conference on Machine Learning (ICML), 2021 [PDF] [Appendix] [Code]

  22. Zoo-Tuning: Adaptive Transfer from A Zoo of Models
    Yang Shu, Zhi Kou, Zhangjie Cao, Jianmin Wang, Mingsheng Long
    International Conference on Machine Learning (ICML), 2021 [PDF] [Appendix] [Code]

  23. Self-Tuning for Data-Efficient Deep Learning
    Ximei Wang, Jinghan Gao, Jianmin Wang, Mingsheng Long
    International Conference on Machine Learning (ICML), 2021 [PDF] [Appendix] [Code]

  24. Representation Subspace Distance for Domain Adaptation Regression
    Xinyang Chen, Sinan Wang, Jianmin Wang, Mingsheng Long
    International Conference on Machine Learning (ICML), 2021 [PDF] [Appendix] [Code]

  25. Transferable Calibration with Lower Bias and Variance in Domain Adaptation
    Ximei Wang, Mingsheng Long, Jianmin Wang, Michael I. Jordan
    Neural Information Processing Systems (NeurIPS), 2020 [PDF] [Appendix] [Code]

  26. Co-Tuning for Transfer Learning
    Kaichao You, Zhi Kou, Mingsheng Long, Jianmin Wang
    Neural Information Processing Systems (NeurIPS), 2020 [PDF] [Code]

  27. Stochastic Normalization
    Zhi Kou, Kaichao You, Mingsheng Long, Jianmin Wang
    Neural Information Processing Systems (NeurIPS), 2020 [PDF] [Code]

  28. Learning to Adapt to Evolving Domains
    Hong Liu, Mingsheng Long, Jianmin Wang, Yu Wang
    Neural Information Processing Systems (NeurIPS), 2020 [PDF] [Code]

  29. Unsupervised Transfer Learning for Spatiotemporal Predictive Networks
    Zhiyu Yao, Yunbo Wang, Mingsheng Long, Jianmin Wang
    International Conference on Machine Learning (ICML), 2020 [PDF] [Code]

  30. Catastrophic Forgetting Meets Negative Transfer: Batch Spectral Shrinkage for Safe Transfer Learning
    Xinyang Chen, Sinan Wang, Bo Fu, Mingsheng Long, Jianmin Wang
    Neural Information Processing Systems (NeurIPS), 2019 [PDF] [Code]

  31. Transferable Normalization: Towards Improving Transferability of Deep Neural Networks
    Ximei Wang, Ying Jin, Mingsheng Long, Jianmin Wang, Michael I. Jordan
    Neural Information Processing Systems (NeurIPS), 2019 [PDF] [Code]

  32. Bridging Theory and Algorithm for Domain Adaptation
    Yuchen Zhang, Tianle Liu, Mingsheng Long, Michael I. Jordan
    International Conference on Machine Learning (ICML), 2019 [PDF] [Appendix] [Code]
    Long Oral Paper

  33. Transferable Adversarial Training: A General Approach to Adapting Deep Classifiers
    Hong Liu, Mingsheng Long, Jianmin Wang, Michael I. Jordan
    International Conference on Machine Learning (ICML), 2019 [PDF] [Code]
    Long Oral Paper

  34. Towards Accurate Model Selection in Deep Unsupervised Domain Adaptation
    Kaichao You, Ximei Wang, Mingsheng Long, Michael I. Jordan
    International Conference on Machine Learning (ICML), 2019 [PDF] [Code]

  35. Transferability vs. Discriminability: Batch Spectral Penalization for Adversarial Domain Adaptation
    Xinyang Chen, Sinan Wang, Mingsheng Long, Jianmin Wang
    International Conference on Machine Learning (ICML), 2019 [PDF] [Code]

  36. Conditional Adversarial Domain Adaptation
    Mingsheng Long, Zhangjie Cao, Jianmin Wang, Michael I. Jordan
    Neural Information Processing Systems (NeurIPS), 2018 [PDF] [Poster] [Code]
    Ranks 6th in NeurIPS 2018

  37. Generalized Zero-Shot Learning with Deep Calibration Network
    Shichen Liu, Mingsheng Long, Jianmin Wang, Michael I. Jordan
    Neural Information Processing Systems (NeurIPS), 2018 [PDF] [Code]

  38. PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning
    Yunbo Wang, Zhifeng Gao, Mingsheng Long, Jianmin Wang, Philip S. Yu
    International Conference on Machine Learning (ICML), 2018 [PDF] [Code]

  39. PredRNN: Recurrent Neural Networks for Predictive Learning using Spatiotemporal LSTMs
    Yunbo Wang, Mingsheng Long, Jianmin Wang, Zhifeng Gao, Philip S. Yu
    Neural Information Processing Systems (NeurIPS), 2017 [PDF] [Code]

  40. Learning Multiple Tasks with Multilinear Relationship Networks
    Mingsheng Long, Zhangjie Cao, Jianmin Wang, Philip S. Yu
    Neural Information Processing Systems (NeurIPS), 2017 [PDF] [Poster] [Code]

  41. Deep Transfer Learning with Joint Adaptation Networks
    Mingsheng Long, Han Zhu, Jianmin Wang, Michael I. Jordan
    International Conference on Machine Learning (ICML), 2017 [PDF] [Slides] [Poster] [Code]
    Ranks 12th in ICML 2017

  42. Unsupervised Domain Adaptation with Residual Transfer Networks
    Mingsheng Long, Han Zhu, Jianmin Wang, Michael I. Jordan
    Neural Information Processing Systems (NeurIPS), 2016 [PDF] [Poster] [Code]

  43. Learning Transferable Features with Deep Adaptation Networks
    Mingsheng Long, Yue Cao, Jianmin Wang, Michael I. Jordan
    International Conference on Machine Learning (ICML), 2015 [PDF] [Slides] [Poster] [Code]
    Ranks 5th in ICML 2015
    Test of Time Award at FTL-IJCAI 2021

Software


Talks


Selected Awards


Professional Services


Senior Area Chair

Area Chair

Associate Editor

Guest Editor

Program Chair

(Senior) PC Member | Reviewer

Research Grants


Teaching


Students


PhDs

Masters

Undergraduates