Mingsheng Long

Mingsheng Long

Associate Professor
(with tenure and with endowed professorship)

Machine Learning Group
School of Software
Tsinghua University

longmingsheng@gmail.com, mingsheng@tsinghua.edu.cn
Room 11-413, East Main Building, Tsinghua University, Beijing 100084, China

[Publications] [Google Scholar] [GitHub] [Prospective students: No opening positions]

Biography

My research spans machine learning theory, algorithms and models, with persistent commitment to creating strong learning machines from big data that adapt to the real world. I am working on deep learning and foundation models, scientific learning and world models, transfer learning and model adaptation.

Our Machine Learning Group is interested in powering machine learning for representation, perception, prediction, and generation of big data with a good tradeoff between accuracy, efficiency, generalizability, and transferability. Our mission is to solve open problems and enable major applications in science and engineering, including AI for science, AI for industrial software, and physics-AI systems.

My Quote

“Everything should be made as simple as possible, but no simpler.” --Albert Einstein

Research Interests

Education

Research Experience

Highlights

Selected Publications [Full List] [DBLP] [Google Scholar]

(✉ Corresponding Author)

Thesis

  1. Mingsheng Long. Transfer Learning: Problems and Methods. 1-127, 2014 [PDF] (In Chinese)

Journal Articles

  1. Skilful Nowcasting of Extreme Precipitation with NowcastNet
    Yuchen Zhang, Mingsheng Long✉, Kaiyuan Chen, Lanxiang Xing, Ronghua Jin, Michael I. Jordan✉, Jianmin Wang✉
    Nature 619, 526–532, 2023 [Link] [Code]
    ⚑ Hot Paper
    ⚑ Youth Outstanding Paper Award, World AI Conference

  2. Interpretable Weather Forecasting for Worldwide Stations with a Unified Deep Model
    Haixu Wu, Hang Zhou, Mingsheng Long✉, Jianmin Wang✉
    Nature Machine Intelligence (Nat Mach Intell) 5, 602-611, 2023 [Link] [Code]
    ⚑ Cover Article
    ⚑ Youth Outstanding Paper Award Honorable Mention, World AI Conference

  3. PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning
    Yunbo Wang, Haixu Wu, Jianjin Zhang, Zhifeng Gao, Jianmin Wang, Philip S. Yu, Mingsheng Long✉
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 45(2):2208-2225, 2023 [Link] [Code]
    ⚑ Hot Paper

  4. Ranking and Tuning Pre-trained Models: A New Paradigm for Exploiting Model Hubs
    Kaichao You, Yong Liu, Ziyang Zhang, Jianmin Wang, Michael I. Jordan, Mingsheng Long✉
    Journal of Machine Learning Research (JMLR), 23(209):1-47, 2022 [Link] [PDF] [Code]

  5. Transferable Representation Learning with Deep Adaptation Networks
    Mingsheng Long, Yue Cao, Zhangjie Cao, Jianmin Wang, Michael I. Jordan
    IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 41(12):3071-3085, 2019 [Link] [Code]
    ⚑ Highly Cited Paper

Conference Proceedings

  1. RoPINN: Region Optimized Physics-Informed Neural Networks
    Haixu Wu, Huakun Luo, Yuezhou Ma, Jianmin Wang, Mingsheng Long✉
    Neural Information Processing Systems (NeurIPS), 2024 [PDF] [arXiv] [Code]

  2. iVideoGPT: Interactive VideoGPTs are Scalable World Models
    Jialong Wu, Shaofeng Yin, Ningya Feng, Xu He, Dong Li, Jianye Hao, Mingsheng Long✉
    Neural Information Processing Systems (NeurIPS), 2024 [PDF] [arXiv] [Code]

  3. Transolver: A Fast Transformer Solver for PDEs on General Geometries
    Haixu Wu, Huakun Luo, Haowen Wang, Jianmin Wang, Mingsheng Long✉
    International Conference on Machine Learning (ICML), 2024 [PDF] [arXiv] [Code]
    ⚑ Spotlight Paper

  4. Timer: Generative Pre-trained Transformers are Large Time Series Models
    Yong Liu, Haoran Zhang, Chenyu Li, Xiangdong Huang, Jianmin Wang, Mingsheng Long✉
    International Conference on Machine Learning (ICML), 2024 [PDF] [arXiv] [Code]

  5. CogDPM: Diffusion Probabilistic Models via Cognitive Predictive Coding
    Kaiyuan Chen, Xingzhuo Guo, Yu Zhang, Jianmin Wang, Mingsheng Long✉
    International Conference on Machine Learning (ICML), 2024 [PDF] [arXiv]

  6. HarmonyDream: Task Harmonization Inside World Models
    Haoyu Ma, Jialong Wu, Ningya Feng, Chenjun Xiao, Dong Li, Jianye Hao, Jianmin Wang, Mingsheng Long✉
    International Conference on Machine Learning (ICML), 2024 [PDF] [arXiv] [Code]

  7. On the Embedding Collapse when Scaling up Recommendation Models
    Xingzhuo Guo, Junwei Pan, Ximei Wang, Baixu Chen, Jie Jiang, Mingsheng Long✉
    International Conference on Machine Learning (ICML), 2024 [PDF] [arXiv] [Code]

  8. iTransformer: Inverted Transformers Are Effective for Time Series Forecasting
    Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, Mingsheng Long✉
    International Conference on Learning Representations (ICLR), 2024 [PDF] [arXiv] [Code]
    ⚑ Spotlight Paper

  9. Efficient ConvBN Blocks for Transfer Learning and Beyond
    Kaichao You, Guo Qin, Anchang Bao, Meng Cao, Ping Huang, Jiulong Shan, Mingsheng Long✉
    International Conference on Learning Representations (ICLR), 2024 [PDF] [arXiv] [Code]
    ⚑ Spotlight Paper

  10. Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learning
    Jialong Wu, Haoyu Ma, Chaoyi Deng, Mingsheng Long✉
    Neural Information Processing Systems (NeurIPS), 2023 [PDF] [arXiv] [Code]

  11. TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis
    Haixu Wu, Tengge Hu, Yong Liu, Hang Zhou, Jianmin Wang, Mingsheng Long✉
    International Conference on Learning Representations (ICLR), 2023 [PDF] [arXiv] [Code]

  12. Debiased Self-Training for Semi-Supervised Learning
    Baixu Chen, Junguang Jiang, Ximei Wang, Pengfei Wan, Jianmin Wang, Mingsheng Long✉
    Neural Information Processing Systems (NeurIPS), 2022 [PDF] [arXiv] [Slides] [Code]
    ⚑ Oral Paper

  13. Flowformer: Linearizing Transformers with Conservation Flows
    Haixu Wu, Jialong Wu, Jiehui Xu, Jianmin Wang, Mingsheng Long✉
    International Conference on Machine Learning (ICML), 2022 [PDF] [arXiv] [Code]

  14. Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy
    Jiehui Xu, Haixu Wu, Jianmin Wang, Mingsheng Long✉
    International Conference on Learning Representations (ICLR), 2022 [PDF] [arXiv] [Code]
    ⚑ Spotlight Paper

  15. Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting
    Haixu Wu, Jiehui Xu, Jianmin Wang, Mingsheng Long✉
    Neural Information Processing Systems (NeurIPS), 2021 [PDF] [Appendix] [arXiv] [Code]
    ⚑ Ranks 10th in NeurIPS 2021

  16. Cycle Self-Training for Domain Adaptation
    Hong Liu, Jianmin Wang, Mingsheng Long✉
    Neural Information Processing Systems (NeurIPS), 2021 [PDF] [Appendix] [Code]

  17. LogME: Practical Assessment of Pre-trained Models for Transfer Learning
    Kaichao You, Yong Liu, Jianmin Wang, Mingsheng Long✉
    International Conference on Machine Learning (ICML), 2021 [PDF] [Appendix] [Code]

  18. Bridging Theory and Algorithm for Domain Adaptation
    Yuchen Zhang, Tianle Liu, Mingsheng Long✉, Michael I. Jordan
    International Conference on Machine Learning (ICML), 2019 [PDF] [Appendix] [Code]
    ⚑ Long Oral Paper

  19. Conditional Adversarial Domain Adaptation
    Mingsheng Long, Zhangjie Cao, Jianmin Wang, Michael I. Jordan
    Neural Information Processing Systems (NeurIPS), 2018 [PDF] [Poster] [Code]
    ⚑ Ranks 6th in NeurIPS 2018

  20. PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning
    Yunbo Wang, Zhifeng Gao, Mingsheng Long✉, Jianmin Wang, Philip S. Yu
    International Conference on Machine Learning (ICML), 2018 [PDF] [Code]

  21. PredRNN: Recurrent Neural Networks for Predictive Learning using Spatiotemporal LSTMs
    Yunbo Wang, Mingsheng Long✉, Jianmin Wang, Zhifeng Gao, Philip S. Yu
    Neural Information Processing Systems (NeurIPS), 2017 [PDF] [Code]

  22. Learning Multiple Tasks with Multilinear Relationship Networks
    Mingsheng Long, Zhangjie Cao, Jianmin Wang, Philip S. Yu
    Neural Information Processing Systems (NeurIPS), 2017 [PDF] [Poster] [Code]

  23. Deep Transfer Learning with Joint Adaptation Networks
    Mingsheng Long, Han Zhu, Jianmin Wang, Michael I. Jordan
    International Conference on Machine Learning (ICML), 2017 [PDF] [Slides] [Poster] [Code]
    ⚑ Ranks 10th in ICML 2017

  24. Unsupervised Domain Adaptation with Residual Transfer Networks
    Mingsheng Long, Han Zhu, Jianmin Wang, Michael I. Jordan
    Neural Information Processing Systems (NeurIPS), 2016 [PDF] [Poster] [Code]

  25. Learning Transferable Features with Deep Adaptation Networks
    Mingsheng Long, Yue Cao, Jianmin Wang, Michael I. Jordan
    International Conference on Machine Learning (ICML), 2015 [PDF] [Slides] [Poster] [Code]
    ⚑ Ranks 5th in ICML 2015
    ⚑ Test of Time Award at FTL-IJCAI 2021

Software

Talks

Selected Awards

Professional Services

Senior Area Chair

Area Chair

Associate Editor

Program Co-Chair

(Senior) PC Member | Reviewer

Research Grants

Teaching

Students

PhDs

Masters

Undergraduates