Biography

---

My research spans machine learning theory, algorithms, and models, with persistent dedication to create stronger learning machines that adapt versatilely to the complex real world. I am actively working on transfer learning and domain adaptation, deep learning and foundation models, scientific learning and world models.

Our Machine Learning Group (THUML) is interested in empowering machine learning for representation, perception, prediction, and decision making with good tradeoff between accuracy, efficiency, generalizability, and transferability. We focus on major science and engineering problems of Artificial Intelligence (AI), including analysis and understanding of big data from IoT, Software, and Sciences.

My Quote

“Everything should be made as simple as possible, but no simpler.” --Albert Einstein

Research Interests

---

• Machine Learning: Transfer Learning, Deep Learning, Scientific Learning

Education

---

• PhD in Computer Science, 2008-2014
  Department of Computer Science and Technology, Tsinghua University, Beijing, China
  Advisor: Prof. Jianmin Wang

• Bachelor in Electrical Engineering, 2004-2008
  Department of Electrical Engineering, Tsinghua University, Beijing, China

Research Experience

---

• Researcher in Machine Learning, 2014-2015
  Department of Computer Science, UC Berkeley
  Advisor: Prof. Michael I. Jordan

• Postdoc in Data Science, 2014-2016
  School of Software, Tsinghua University, Beijing, China
  Advisor: Prof. Jianmin Wang

Recent News

---

• 2022.09: Four papers got accepted by NeurIPS, congrats! (more)
• 2022.06: One paper got accepted by JMLR, congrats! (more)
• 2022.05: One paper got accepted by ICML, congrats! (more)
• 2022.03: Two papers got accepted by TPAMI, congrats! (more)
• 2022.01: Three papers got accepted by ICLR, congrats! (more)
• 2021.09: Two papers got accepted by NeurIPS, congrats! (more)
• 2021.09: One paper got accepted by TPAMI, congrats! (more)
• 2021.05: Four papers got accepted by ICML, congrats! (more)
• 2021.03: Five papers got accepted by CVPR, congrats! (more)
• 2020.09: Four papers got accepted by NeurIPS, congrats! (more)

Selected Publications [Full List] [DBLP] [Google Scholar]

(✉ Corresponding Author)

---

Thesis

1. Mingsheng Long. Transfer Learning: Problems and Methods. 1-127, 2014 [PDF] (In Chinese)
   

Journal Articles

1. Ranking and Tuning Pre-trained Models: A New Paradigm for Exploiting Model Hubs
   Kaichao You, Yong Liu, Ziyang Zhang, Jianmin Wang, Michael I. Jordan, Mingsheng Long^✉
   Journal of Machine Learning Research (JMLR), 2022 [PDF] [arXiv] [Code]

2. PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning
   Yunbo Wang, Haixu Wu, Jianjin Zhang, Zhifeng Gao, Jianmin Wang, Philip S. Yu, Mingsheng Long^✉
   IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 99(Preprint), 2022 [arXiv] [Code]

3. From Big to Small: Adaptive Learning to Partial-Set Domains
   Zhangjie Cao, Kaichao You, Ziyang Zhang, Jianmin Wang, Mingsheng Long^✉
   IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 99(Preprint), 2022 [arXiv] [Code]

4. VideoDG: Generalizing Temporal Relations in Videos to Novel Domains
   Zhiyu Yao, Yunbo Wang, Jianmin Wang, Philip S. Yu, Mingsheng Long^✉
   IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 99(Preprint), 2021 [Link] [Code]

5. Transferable Representation Learning with Deep Adaptation Networks
   Mingsheng Long, Yue Cao, Zhangjie Cao, Jianmin Wang, Michael I. Jordan
   IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 41(12):3071-3085, 2019 [Link] [Code]

Conference Proceedings

1. Debiased Self-Training for Semi-Supervised Learning
   Baixu Chen, Junguang Jiang, Ximei Wang, Pengfei Wan, Jianmin Wang, Mingsheng Long^✉
   Neural Information Processing Systems (NeurIPS), 2022 [arXiv] (Oral)

2. Hub-Pathway: Transfer Learning from A Hub of Pre-trained Models
   Yang Shu, Zhangjie Cao, Ziyang Zhang, Jianmin Wang, Mingsheng Long^✉
   Neural Information Processing Systems (NeurIPS), 2022 [arXiv]

3. Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting
   Yong Liu, Haixu Wu, Jianmin Wang, Mingsheng Long^✉
   Neural Information Processing Systems (NeurIPS), 2022 [arXiv]

4. Supported Policy Optimization for Offline Reinforcement Learning
   Jialong Wu, Haixu Wu, Zihan Qiu, Jianmin Wang, Mingsheng Long^✉
   Neural Information Processing Systems (NeurIPS), 2022 [arXiv]

5. Flowformer: Linearizing Transformers with Conservation Flows
   Haixu Wu, Jialong Wu, Jiehui Xu, Jianmin Wang, Mingsheng Long^✉
   International Conference on Machine Learning (ICML), 2022 [PDF] [arXiv] [Code]

6. Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy
   Jiehui Xu, Haixu Wu, Jianmin Wang, Mingsheng Long^✉
   International Conference on Learning Representations (ICLR), 2022 [PDF] [OpenReview] [Code] (Spotlight)

7. X-model: Improving Data Efficiency in Deep Learning with A Minimax Model
   Ximei Wang, Xinyang Chen, Jianmin Wang, Mingsheng Long^✉
   International Conference on Learning Representations (ICLR), 2022 [PDF] [OpenReview] [Code]

8. Decoupled Adaptation for Cross-Domain Object Detection
   Junguang Jiang, Baixu Chen, Jianmin Wang, Mingsheng Long^✉
   International Conference on Learning Representations (ICLR), 2022 [PDF] [OpenReview] [Code]

9. Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting
   Haixu Wu, Jiehui Xu, Jianmin Wang, Mingsheng Long^✉
   Neural Information Processing Systems (NeurIPS), 2021 [PDF] [Appendix] [arXiv] [Code]

10. Cycle Self-Training for Domain Adaptation
    Hong Liu, Jianmin Wang, Mingsheng Long^✉
    Neural Information Processing Systems (NeurIPS), 2021 [PDF] [Appendix] [Code]

11. LogME: Practical Assessment of Pre-trained Models for Transfer Learning
    Kaichao You, Yong Liu, Jianmin Wang, Mingsheng Long^✉
    International Conference on Machine Learning (ICML), 2021 [PDF] [Appendix] [Code]

12. Zoo-Tuning: Adaptive Transfer from A Zoo of Models
    Yang Shu, Zhi Kou, Zhangjie Cao, Jianmin Wang, Mingsheng Long^✉
    International Conference on Machine Learning (ICML), 2021 [PDF] [Appendix] [Code]

13. Self-Tuning for Data-Efficient Deep Learning
    Ximei Wang, Jinghan Gao, Jianmin Wang, Mingsheng Long^✉
    International Conference on Machine Learning (ICML), 2021 [PDF] [Appendix] [Code]

14. Representation Subspace Distance for Domain Adaptation Regression
    Xinyang Chen, Sinan Wang, Jianmin Wang, Mingsheng Long^✉
    International Conference on Machine Learning (ICML), 2021 [PDF] [Appendix] [Code]

15. Transferable Calibration with Lower Bias and Variance in Domain Adaptation
    Ximei Wang, Mingsheng Long^✉, Jianmin Wang, Michael I. Jordan
    Neural Information Processing Systems (NeurIPS), 2020 [PDF] [Appendix] [Code]

16. Co-Tuning for Transfer Learning
    Kaichao You, Zhi Kou, Mingsheng Long^✉, Jianmin Wang
    Neural Information Processing Systems (NeurIPS), 2020 [PDF] [Code]

17. Stochastic Normalization
    Zhi Kou, Kaichao You, Mingsheng Long^✉, Jianmin Wang
    Neural Information Processing Systems (NeurIPS), 2020 [PDF] [Code]

18. Learning to Adapt to Evolving Domains
    Hong Liu, Mingsheng Long^✉, Jianmin Wang, Yu Wang
    Neural Information Processing Systems (NeurIPS), 2020 [PDF] [Code]

19. Unsupervised Transfer Learning for Spatiotemporal Predictive Networks
    Zhiyu Yao, Yunbo Wang, Mingsheng Long^✉, Jianmin Wang
    International Conference on Machine Learning (ICML), 2020 [PDF] [Code]

20. Catastrophic Forgetting Meets Negative Transfer: Batch Spectral Shrinkage for Safe Transfer Learning
    Xinyang Chen, Sinan Wang, Bo Fu, Mingsheng Long^✉, Jianmin Wang
    Neural Information Processing Systems (NeurIPS), 2019 [PDF] [Code]

21. Transferable Normalization: Towards Improving Transferability of Deep Neural Networks
    Ximei Wang, Ying Jin, Mingsheng Long^✉, Jianmin Wang, Michael I. Jordan
    Neural Information Processing Systems (NeurIPS), 2019 [PDF] [Code]

22. Bridging Theory and Algorithm for Domain Adaptation
    Yuchen Zhang, Tianle Liu, Mingsheng Long^✉, Michael I. Jordan
    International Conference on Machine Learning (ICML), 2019 [PDF] [Appendix] [Code] (Long)

23. Transferable Adversarial Training: A General Approach to Adapting Deep Classifiers
    Hong Liu, Mingsheng Long^✉, Jianmin Wang, Michael I. Jordan
    International Conference on Machine Learning (ICML), 2019 [PDF] [Code] (Long)

24. Towards Accurate Model Selection in Deep Unsupervised Domain Adaptation
    Kaichao You, Ximei Wang, Mingsheng Long^✉, Michael I. Jordan
    International Conference on Machine Learning (ICML), 2019 [PDF] [Code]

25. Transferability vs. Discriminability: Batch Spectral Penalization for Adversarial Domain Adaptation
    Xinyang Chen, Sinan Wang, Mingsheng Long^✉, Jianmin Wang
    International Conference on Machine Learning (ICML), 2019 [PDF] [Code]

26. Conditional Adversarial Domain Adaptation
    Mingsheng Long, Zhangjie Cao, Jianmin Wang, Michael I. Jordan
    Neural Information Processing Systems (NeurIPS), 2018 [PDF] [Poster] [Code] (Ranks 5th in NeurIPS 2018)

27. Generalized Zero-Shot Learning with Deep Calibration Network
    Shichen Liu, Mingsheng Long^✉, Jianmin Wang, Michael I. Jordan
    Neural Information Processing Systems (NeurIPS), 2018 [PDF] [Code]

28. PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning
    Yunbo Wang, Zhifeng Gao, Mingsheng Long^✉, Jianmin Wang, Philip S. Yu
    International Conference on Machine Learning (ICML), 2018 [PDF] [Code]

29. PredRNN: Recurrent Neural Networks for Predictive Learning using Spatiotemporal LSTMs
    Yunbo Wang, Mingsheng Long^✉, Jianmin Wang, Zhifeng Gao, Philip S. Yu
    Neural Information Processing Systems (NeurIPS), 2017 [PDF] [Code]

30. Learning Multiple Tasks with Multilinear Relationship Networks
    Mingsheng Long, Zhangjie Cao, Jianmin Wang, Philip S. Yu
    Neural Information Processing Systems (NeurIPS), 2017 [PDF] [Poster] [Code]

31. Deep Transfer Learning with Joint Adaptation Networks
    Mingsheng Long, Han Zhu, Jianmin Wang, Michael I. Jordan
    International Conference on Machine Learning (ICML), 2017 [PDF] [Slides] [Poster] [Code] (Ranks 13rd in ICML 2017)

32. Unsupervised Domain Adaptation with Residual Transfer Networks
    Mingsheng Long, Han Zhu, Jianmin Wang, Michael I. Jordan
    Neural Information Processing Systems (NeurIPS), 2016 [PDF] [Poster] [Code]

33. Learning Transferable Features with Deep Adaptation Networks
    Mingsheng Long, Yue Cao, Jianmin Wang, Michael I. Jordan
    International Conference on Machine Learning (ICML), 2015 [PDF] [Slides] [Poster] [Code] (Ranks 5th in ICML 2015)

Invited Talks

---

• Adaptive Transfer Learning from Pre-trained Models, Vision And Learning SEminar, VALSE 2022 [PDF]
• Deep Learning Models for Sequential Data Analysis, Chinese Conference on Data Mining, CCDM 2022 [PDF]
• Transfer Learning: Theories and Algorithms, Chinese Conference on Data Mining, CCDM 2020 [PDF]
• Transfer Learning: Theories and Algorithms, Conference on Pattern Recognition and Computer Vision, PRCV 2019 [PDF]
• Transfer Learning: Theories and Algorithms, CCF Conference on Artificial Intelligence, CCFAI 2019 [PDF]
• Transfer Learning: From Algorithms to Theories and Back, Vision And Learning SEminar, VALSE 2019 [PDF]
• Predictive Learning for Video Prediction, Chinese Conference on Data Mining, CCDM 2018 [PDF]
• Deep Hashing for Multimedia Retrieval, International Conference on Data Science, 2017 [PDF]

Software

---

• Transfer-Learning-Library (TLlib): An open-source and easy-to-use library for transfer learning [Code @ GitHub] [Documentation]
• Predictive-Learning-Library (Pred-Learn): An open-source and easy-to-use package for predictive learning [Code @ GitHub]
• DeepHash: An open-source package for deep learning to hash [Code @ GitHub]

Awards

---

• Highlighted Area Chair, ICLR 2022
• Test of Time Award, FTL-IJCAI, 2021
• AI 2000 Most Influential Scholar Award in Machine Learning (rank 1 in China universities), 2021
• First Prize, Science and Technology Progress Award, Beijing, 2021
• Excellent Young Scholar, National Natural Science Foundation, China, 2020
• Rising Star in Science and Technology, Beijing, 2020
• Popular Dissertation Top 20, CNKI, China, 2019
• First Prize, Technical Invention Award, Ministry of Education (MOE), China, 2018
• Distinguished Dissertation Award, China Association for Artificial Intelligence (CAAI), 2016
• Distinguished Dissertation Nominee, China Computer Federation (CCF), 2014
• Outstanding PhD Graduate, Tsinghua University, 2014
• Distinguished Dissertation Award, Tsinghua University, 2014
• Best Paper Nominee, SIAM SDM, 2012

Professional Services

---

PC Chair

• Transfer and Multi-Task Learning Workshop, NeurIPS 2015
• Transferring and Adapting Source Knowledge in Computer Vision Workshop and Challenge (TASK-CV), ICCV 2017

Area Chair

• International Conference on Machine Learning (ICML) 2021, 2022
• Neural Information Processing Systems (NeurIPS) 2020, 2021, 2022
• International Conference on Learning Representations (ICLR) 2020, 2021, 2022, 2023
• AAAI Conference on Artificial Intelligence (AAAI) 2022, 2023
• International Joint Conference on Artificial Intelligence (IJCAI) 2021

Associate Editor

• IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 2021, 2022
• Transactions on Machine Learning Research (TMLR) 2022

Guest Editor

• IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 2020, 2021, 2022

(Senior) PC Member | Reviewer

• International Conference on Machine Learning (ICML) 2014, 2017, 2018, 2019, 2020
• Neural Information Processing Systems (NeurIPS) 2016, 2017, 2018, 2019
• IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2018, 2019, 2020, 2021, 2022
• ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) 2018, 2019, 2020
• Artificial Intelligence Journal (AIJ) 2016, 2018, 2022
• Journal of Machine Learning Research (JMLR) 2018, 2020, 2021, 2022
• IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 2016, 2017, 2018, 2019, 2020, 2021, 2022
• IEEE Transactions on Knowledge and Data Engineering (TKDE) 2014, 2015, 2016, 2017, 2018, 2019, 2020
• IEEE Transactions on Neural Networks and Learning Systems (TNNLS) 2016, 2017, 2018, 2019, 2020
• ACM Computing Surveys 2019, 2020
• Nature Communications 2019, 2020
• Nature Human Behavior 2019, 2020

Research Grants

---

• Principal Investigator, Transfer Learning, National Natural Science Foundation for Excellent Young Scholars, 2021-2023
• Principal Investigator, Industrial Big Data Transfer Learning: Theories and Algorithms, Beijing Nova Program, 2021-2023
• Principal Investigator, Adaptive Perception and Intelligent Scheduling for Manufacturing Tasks, National Major Program for New Generation AI, 2020-2023
• Principal Investigator, Robust Transfer Learning for OOD Data in Open Dynamic Environment, MOE Strategic Research Project on Artificial Intelligence Algorithms (Project PI: Prof. Yang Yu @ Nanjing University), 2020-2021
• Principal Investigator, Recommendation Algorithm based on Multi-Task Learning, Alibaba Innovative Research (AIR), 2020-2021
• Principal Investigator, Recommendation Algorithm based on Deep Learning, Alibaba Innovative Research (AIR), 2018-2019
• Principal Investigator, Intelligent Severe Weather Nowcasting System, China Meteorological Administration, 2019-2020
• Principal Investigator, Spatiotemporal Deep Learning for Severe Weather Nowcasting, National Key R&D Plan, 2018-2022
• Principal Investigator, Spatiotemporal Deep Learning: Algorithms and Applications, Natural Science Foundation of China, 2018-2021
• Principal Investigator, Safe Transfer Learning for Big Data Analytics, Natural Science Foundation of China, 2016-2018
• Principal Investigator, Scalable Transfer Learning: Theory and Algorithms, China Postdoc Science Foundation, 2015-2016

Teaching

---

• Instructor: Deep Learning, 2018- (清华大学年度教学优秀奖, Top 25% every year)
• Instructor: Machine Learning, 2019- (Top 25% every year)
• Instructor: Introduction to Artificial Intelligence, 2021- (Top 25% every year)
• Instructor: Introduction to Data Science, 2018-2020
• Co-Instructor: Introduction to Big Data Technology, 2017-2020
• Co-Instructor: Foundations of Big Data Systems (A), 2016-2020
• Co-Instructor: Foundations of Big Data Systems (B), 2015-2020

Students

---

PhDs

• Yue Cao (2014-2019): Co-supervised with Prof. Jianmin Wang. 清华大学特等奖学金, 清华大学优秀博士学位论文, 清华大学优秀博士毕业生 -> Senior Researcher, MSRA
• Yunbo Wang (2015-2020): Co-supervised with Prof. Philip S. Yu. 麻省理工学院访问学者, 清华大学优秀博士学位论文, 中国计算机学会优秀博士学位论文奖 -> Assistant Professor, Shanghai Jiao Tong University
• Zhongyi Pei (2015-2020): Co-supervised with Prof. Philip S. Yu -> THU Postdoc
• Ximei Wang (2017-2022): Co-supervised with Prof. Jianmin Wang. 北京市优秀毕业生 -> Daka, WeChat
• Xinyang Chen (2017-2022): 全球AI华人百强新星 -> Assistant Professor, Harbin Institute of Technology (Shenzhen)
• Yang Shu (2018-2023): ->
• Zhiyu Yao (2019-2024): ->
• Yuchen Zhang (2020-2024): Co-supervised with Prof. Jianmin Wang ->
• Kaichao You (2020-2025): ->
• Yong Liu (2021-2026): ->
• Jincheng Zhong (2022-2026): ->
• Jialong Wu (2022-2027): ->

Masters

• Han Zhu (2014-2017): Co-supervised with Prof. Jianmin Wang. 清华大学优秀硕士学位论文, 清华大学优秀硕士毕业生 -> Alimama
• Feng Li (2014-2017): Co-supervised with Prof. Jianmin Wang -> Alimama
• Qiaoan Chen (2014-2017): Co-supervised with Prof. Jianmin Wang -> WeChat
• Qingfu Wen (2015-2018): Co-supervised with Prof. Jianmin Wang -> Tencent
• Jue Wang (2015-2018): Co-supervised with Prof. Jianmin Wang -> Government
• Ziru Xu (2016-2019): Co-supervised with Prof. Jianmin Wang. 清华大学优秀硕士学位论文, 清华大学优秀硕士毕业生 -> Alimama
• Jianjin Zhang (2016-2019): -> Microsoft
• Zhifeng Gao (2016-2019): -> Alimama
• Yuchen Zhang (2017-2020): Co-supervised with Prof. Jianmin Wang. 清华大学优秀硕士学位论文 -> THU PhD
• Jintao Liu (2017-2020): Co-supervised with Prof. Jianmin Wang -> ByteDance
• Liang Li (2017-2020): Co-supervised with Prof. Jianmin Wang -> Microsoft
• Sinan Wang (2017-2020): 清华大学优秀硕士学位论文, 清华大学优秀硕士毕业生 -> Alibaba
• Ying Jin (2018-2021): Co-supervised with Prof. Jianmin Wang -> PhD @ MMLab, CUHK
• Xingqiang Du (2018-2021): Co-supervised with Prof. Jianmin Wang -> Kuaishou
• Bo Fu (2018-2021): Co-supervised with Prof. Jianmin Wang. 清华大学优秀硕士学位论文, 清华大学优秀硕士毕业生 -> WeChat
• Bin Liu (2018-2021): 北京市优秀毕业生 -> Alimama
• Chao Huang (2019-2022): -> ByteDance
• Zhi Kou (2019-2022): -> Alimama
• Jincheng Zhong (2019-2022): -> THU PhD
• Jinghan Gao (2019-2022): -> Alibaba
• Haixu Wu (2020-2023): ->
• Junguang Jiang (2020-2023): ->
• Jiehui Xu (2020-2023): ->
• Kaiyuan Chen (2021-2024): ->
• Tengge Hu (2021-2024): ->
• Haoyu Ma (2022-2025): ->
• Lanxiang Xing (2022-2025): ->
• Baixu Chen (2022-2025): ->
• Qilong Ma (2022-2025): ->

Undergraduates

• Zhangjie Cao (2014-2018): 清华大学优良毕业生 -> Stanford PhD
• Shichen Liu (2014-2018): 清华大学优良毕业生 -> USC PhD
• Bin Liu (2014-2018): 清华大学优良毕业生 -> THU Master
• Lijia Ma (2015-2019): -> UW PhD
• Hongyu Zhu (2015-2019): -> CMU Master
• Zhiyu Yao (2015-2019): -> THU PhD
• Chao Huang (2015-2019): -> THU Master
• Zhi Kou (2015-2019): -> THU Master
• Kaichao You (2016-2020): 清华大学特等奖学金, 清华大学优秀毕业生 -> THU PhD
• Haixu Wu (2016-2020): 清华大学优良毕业生 -> THU Master
• Junguang Jiang (2016-2020): -> THU Master
• Weirui Ye (2016-2020): -> THU PhD
• Ziping Sun (2016-2020): -> THU Master
• Tianle Liu (2016-2020): 清华大学优良毕业生 -> Harvard PhD
• Hong Liu (2017-2021): 清华大学特等奖学金, 清华大学优秀毕业生 -> Stanford PhD
• Yong Liu (2017-2021): 清华大学优秀毕业生, 北京市普通高等学校优秀本科生毕业论文 -> THU PhD
• Yifei Ji (2017-2021): 北京市优秀毕业生 -> THU Master
• Shijie Wang (2017-2021): -> Brown PhD
• Haoyu Niu (2017-2021): -> CMU Master
• Jialong Wu (2018-2022): 清华大学优秀毕业生 -> THU PhD
• Haoyu Ma (2018-2022): 清华大学优良毕业生 -> THU Master
• Lanxiang Xing (2018-2022): 清华大学优良毕业生 -> THU Master
• Baixu Chen (2018-2022): -> THU Master
• Chenyu Wang (2018-2022): 清华大学优秀毕业生 -> MIT PhD

---