Mingsheng Long

Associate Professor, PhD Supervisor

Machine Learning Group
School of Software
Tsinghua University

longmingsheng@gmail.com
mingsheng@tsinghua.edu.cn
Room 11-413, East Main Building, Tsinghua University, Beijing 100084, China

[Publications] [Google Scholar] [THUML @ Github] [Subscription @ WeChat] [Instruction to Applicants]

Biography

My research spans theories, algorithms, and applications of machine learning, with persistent dedication to enable adaptive machine learning in the open, dynamic and non-stationary world. My current research is to promote advances in transfer, adaptation, and data-efficient learning, foundation deep models, learning with spatiotemporal and scientific knowledge.

Our Machine Learning Group is interested in developing machine learning technology for systematic generalization in representation, inference, and decision making with good trade-off between accuracy, efficiency, and stability. We focus on industrial applications of AI and data science, including analysis and understanding of vision, language, IoT, and science data.

My Quote

“Everything should be made as simple as possible, but no simpler.” --Albert Einstein

Research Interests

Machine Learning: Transfer Learning, Deep Learning, Knowledge Learning

Education

PhD in Computer Science, 2008-2014
Department of Computer Science and Technology, Tsinghua University, Beijing, China
Advisor: Prof. Jianmin Wang
Bachelor in Electrical Engineering, 2004-2008
Department of Electrical Engineering, Tsinghua University, Beijing, China

Research Experience

Postdoc in Machine Learning, 2014-2015
Department of Computer Science, UC Berkeley
Advisor: Prof. Michael I. Jordan
Postdoc in Machine Learning, 2014-2016
School of Software, Tsinghua University, Beijing, China
Advisor: Prof. Jianmin Wang

Selected Publications [Full List] [DBLP] [Google Scholar]

(* Corresponding Author)

Thesis

Mingsheng Long. Transfer Learning: Problems and Methods. 1-127, 2014 [PDF] (In Chinese)

Journal Articles

VideoDG: Generalizing Temporal Relations in Videos to Novel Domains
Zhiyu Yao, Yunbo Wang, Jianmin Wang, Philip S. Yu, Mingsheng Long*
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 99(Preprint), 2021 [arXiv] [Code]
Transferable Representation Learning with Deep Adaptation Networks
Mingsheng Long, Yue Cao, Zhangjie Cao, Jianmin Wang, Michael I. Jordan
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 41(12):3071-3085, 2019 [Link] [Code]

Conference Proceedings

LogME: Practical Assessment of Pre-trained Models for Transfer Learning
Kaichao You, Yong Liu, Jianmin Wang, Mingsheng Long*
International Conference on Machine Learning (ICML), 2021 [PDF] [Appendix] [Code]
Zoo-Tuning: Adaptive Transfer from A Zoo of Models
Yang Shu, Zhi Kou, Zhangjie Cao, Jianmin Wang, Mingsheng Long*
International Conference on Machine Learning (ICML), 2021 [PDF] [Appendix] [Code]
Self-Tuning for Data-Efficient Deep Learning
Ximei Wang, Jinghan Gao, Jianmin Wang, Mingsheng Long*
International Conference on Machine Learning (ICML), 2021 [PDF] [Appendix] [Code]
Representation Subspace Distance for Domain Adaptation Regression
Xinyang Chen, Sinan Wang, Jianmin Wang, Mingsheng Long*
International Conference on Machine Learning (ICML), 2021 [PDF] [Appendix] [Code]
Transferable Calibration with Lower Bias and Variance in Domain Adaptation
Ximei Wang, Mingsheng Long*, Jianmin Wang, Michael I. Jordan
Neural Information Processing Systems (NeurIPS), 2020 [PDF] [Appendix] [Code]
Co-Tuning for Transfer Learning
Kaichao You, Zhi Kou, Mingsheng Long*, Jianmin Wang
Neural Information Processing Systems (NeurIPS), 2020 [PDF] [Code]
Stochastic Normalization
Zhi Kou, Kaichao You, Mingsheng Long*, Jianmin Wang
Neural Information Processing Systems (NeurIPS), 2020 [PDF] [Code]
Learning to Adapt to Evolving Domains
Hong Liu, Mingsheng Long*, Jianmin Wang, Yu Wang
Neural Information Processing Systems (NeurIPS), 2020 [PDF] [Code]
Unsupervised Transfer Learning for Spatiotemporal Predictive Networks
Zhiyu Yao, Yunbo Wang, Mingsheng Long*, Jianmin Wang
International Conference on Machine Learning (ICML), 2020 [PDF] [Code]
Catastrophic Forgetting Meets Negative Transfer: Batch Spectral Shrinkage for Safe Transfer Learning
Xinyang Chen, Sinan Wang, Bo Fu, Mingsheng Long*, Jianmin Wang
Neural Information Processing Systems (NeurIPS), 2019 [PDF] [Code]
Transferable Normalization: Towards Improving Transferability of Deep Neural Networks
Ximei Wang, Ying Jin, Mingsheng Long*, Jianmin Wang, Michael I. Jordan
Neural Information Processing Systems (NeurIPS), 2019 [PDF] [Code]
Bridging Theory and Algorithm for Domain Adaptation
Yuchen Zhang, Tianle Liu, Mingsheng Long*, Michael I. Jordan
International Conference on Machine Learning (ICML), 2019 [PDF] [Appendix] [Code] (Long Oral)
Towards Accurate Model Selection in Deep Unsupervised Domain Adaptation
Kaichao You, Ximei Wang, Mingsheng Long*, Michael I. Jordan
International Conference on Machine Learning (ICML), 2019 [PDF] [Code]
Transferable Adversarial Training: A General Approach to Adapting Deep Classifiers
Hong Liu, Mingsheng Long*, Jianmin Wang, Michael I. Jordan
International Conference on Machine Learning (ICML), 2019 [PDF] [Code] (Long Oral)
Transferability vs. Discriminability: Batch Spectral Penalization for Adversarial Domain Adaptation
Xinyang Chen, Sinan Wang, Mingsheng Long*, Jianmin Wang
International Conference on Machine Learning (ICML), 2019 [PDF] [Code]
Conditional Adversarial Domain Adaptation
Mingsheng Long, Zhangjie Cao, Jianmin Wang, Michael I. Jordan
Neural Information Processing Systems (NeurIPS), 2018 [PDF] [Poster] [Code]
PaperDigest Most Influential Papers
Generalized Zero-Shot Learning with Deep Calibration Network
Shichen Liu, Mingsheng Long*, Jianmin Wang, Michael I. Jordan
Neural Information Processing Systems (NeurIPS), 2018 [PDF] [Code]
PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning
Yunbo Wang, Zhifeng Gao, Mingsheng Long*, Jianmin Wang, Philip S. Yu
International Conference on Machine Learning (ICML), 2018 [PDF] [Code]
PredRNN: Recurrent Neural Networks for Predictive Learning using Spatiotemporal LSTMs
Yunbo Wang, Mingsheng Long*, Jianmin Wang, Zhifeng Gao, Philip S. Yu
Neural Information Processing Systems (NeurIPS), 2017 [PDF] [Code]
Learning Multiple Tasks with Multilinear Relationship Networks
Mingsheng Long, Zhangjie Cao, Jianmin Wang, Philip S. Yu
Neural Information Processing Systems (NeurIPS), 2017 [PDF] [Poster] [Code]
Deep Transfer Learning with Joint Adaptation Networks
Mingsheng Long, Han Zhu, Jianmin Wang, Michael I. Jordan
International Conference on Machine Learning (ICML), 2017 [PDF] [Slides] [Poster] [Code] (Long Oral)
PaperDigest Most Influential Papers
Unsupervised Domain Adaptation with Residual Transfer Networks
Mingsheng Long, Han Zhu, Jianmin Wang, Michael I. Jordan
Neural Information Processing Systems (NeurIPS), 2016 [PDF] [Poster] [Code]
Learning Transferable Features with Deep Adaptation Networks
Mingsheng Long, Yue Cao, Jianmin Wang, Michael I. Jordan
International Conference on Machine Learning (ICML), 2015 [PDF] [Slides] [Poster] [Code] (Long Oral)
PaperDigest Most Influential Papers

Invited Talks

Transfer Learning: Theories, Algorithms, and Open Library, FTL-IJCAI 2021 [PDF]
Transfer Learning: Theories and Algorithms, Chinese Conference on Data Mining, CCDM 2020 [PDF]
Transfer Learning: Theories and Algorithms, Conference on Pattern Recognition and Computer Vision, PRCV 2019 [PDF]
Transfer Learning: Theories and Algorithms, CCF Conference on Artificial Intelligence, CCFAI 2019 [PDF]
Transfer Learning: From Algorithms to Theories and Back, Vision And Learning SEminar, VALSE 2019 [PDF]
Predictive Learning for Video Prediction, Chinese Conference on Data Mining, CCDM 2018 [PDF]
Deep Hashing for Multimedia Retrieval, International Conference on Data Science, 2017 [PDF]

Software

Transfer-Learning-Library: An open-source and high-quality package for deep transfer learning [Code @ GitHub]
Predictive-Learning-Library: An open-source and high-quality package for deep predictive learning [Code @ GitHub]
DeepHash: An open-source package for deep learning to hash [Code @ GitHub]

Awards

Test of Time Award, FTL-IJCAI, 2021
AI 2000 Most Influential Scholar Award in Machine Learning (rank 1 in domestic universities), 2021
Excellent Young Scholar, National Natural Science Foundation, China, 2020
Rising Star in Science and Technology, Beijing, 2020
Popular Dissertation Top 20, CNKI, China, 2019
First Prize, Technical Invention Award, Ministry of Education (MOE), China, 2018
Distinguished Dissertation Award, China Association for Artificial Intelligence (CAAI), 2016
Distinguished Dissertation Nominee, China Computer Federation (CCF), 2014
Outstanding PhD Graduate, Tsinghua University, 2014
Distinguished Dissertation Award, Tsinghua University, 2014
Best Paper Nominee, SIAM SDM, 2012

Professional Services

PC Chair

Transfer and Multi-Task Learning Workshop, NeurIPS 2015
Transferring and Adapting Source Knowledge in Computer Vision Workshop and Challenge (TASK-CV), ICCV 2017

Area Chair

International Conference on Machine Learning (ICML) 2021
Neural Information Processing Systems (NeurIPS) 2020, 2021
International Conference on Learning Representations (ICLR) 2020, 2021
AAAI Conference on Artificial Intelligence (AAAI) 2022
International Joint Conference on Artificial Intelligence (IJCAI) 2021

Associate Editor

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 2021

Guest Editor

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 2020, 2021

(Senior) PC Member | Reviewer

International Conference on Machine Learning (ICML) 2014, 2017, 2018, 2019, 2020
Neural Information Processing Systems (NeurIPS) 2016, 2017, 2018, 2019
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2018, 2019, 2020, 2021
IEEE International Conference on Computer Vision (ICCV) 2019
European Conference on Computer Vision (ECCV) 2018, 2020
ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) 2018, 2019, 2020
AAAI Conference on Artificial Intelligence (AAAI) 2018, 2019, 2020, 2021
International Joint Conference on Artificial Intelligence (IJCAI) 2017, 2018, 2019, 2020
Artificial Intelligence Journal (AIJ) 2016, 2018
Journal of Machine Learning Research (JMLR) 2018
International Journal of Computer Vision (IJCV) 2019, 2020
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 2016, 2017, 2018, 2019, 2020, 2021
IEEE Transactions on Knowledge and Data Engineering (TKDE) 2014, 2015, 2016, 2017, 2018, 2019, 2020
IEEE Transactions on Neural Networks and Learning Systems (TNNLS) 2016, 2017, 2018, 2019
IEEE Transactions on Image Processing (TIP) 2014, 2015, 2016, 2017, 2018, 2019
ACM Computing Surveys 2019, 2020
Nature Communications 2019, 2020
Nature Human Behavior 2019, 2020

Research Grants

Principal Investigator, Transfer Learning, National Natural Science Foundation for Excellent Young Scholars, 2021-2023
Principal Investigator, Industrial Big Data Transfer Learning: Theories and Algorithms, Beijing Nova Program, 2021-2023
Principal Investigator, Adaptive Perception and Intelligent Scheduling for Manufacturing Tasks, National Major Program for New Generation AI, 2020-2023
Principal Investigator, Robust Transfer Learning for OOD Data in Open Dynamic Environment, MOE Strategic Research Project on Artificial Intelligence Algorithms (Project PI: Prof. Yang Yu @ Nanjing University), 2020-2021
Principal Investigator, Recommendation Algorithm based on Multi-Task Learning, Alibaba Innovative Research (AIR), 2020-2021
Principal Investigator, Recommendation Algorithm based on Deep Learning, Alibaba Innovative Research (AIR), 2018-2019
Principal Investigator, Intelligent Severe Weather Nowcasting System, China Meteorological Administration, 2019-2020
Principal Investigator, Spatiotemporal Deep Learning for Severe Weather Nowcasting, National Key R&D Plan, 2018-2022
Principal Investigator, Spatiotemporal Deep Learning: Algorithms and Applications, Natural Science Foundation of China, 2018-2021
Principal Investigator, Safe Transfer Learning for Big Data Analytics, Natural Science Foundation of China, 2016-2018
Principal Investigator, Scalable Transfer Learning: Theory and Algorithms, China Postdoc Science Foundation, 2015-2016

Teaching

Instructor: Deep Learning, 2018- (清华大学年度教学优秀奖)
Instructor: Machine Learning, 2019-
Instructor: Introduction to Artificial Intelligence, 2021-
Instructor: Introduction to Data Science, 2018-2020
Co-Instructor: Introduction to Big Data Technology, 2017-
Co-Instructor: Foundations of Big Data Systems (A), 2016-
Co-Instructor: Foundations of Big Data Systems (B), 2015-

Students

PhDs

Yue Cao (2014-2019): Co-supervised with Prof. Jianmin Wang. 清华大学特等奖学金, 清华大学优秀博士学位论文, 清华大学优秀博士毕业生 -> Researcher, MSRA
Yunbo Wang (2015-2020): Co-supervised with Prof. Philip S. Yu. 麻省理工学院访问学者, 清华大学优秀博士学位论文, 中国计算机学会优秀博士学位论文奖 -> Assistant Professor, Shanghai Jiao Tong University
Zhongyi Pei (2015-2020): Co-supervised with Prof. Philip S. Yu -> THU Postdoc
Ximei Wang (2017-2022): Co-supervised with Prof. Jianmin Wang ->
Xinyang Chen (2017-2022): ->
Yang Shu (2018-2023): ->
Zhiyu Yao (2019-2024): ->
Yuchen Zhang (2020-2024): Co-supervised with Prof. Jianmin Wang ->
Kaichao You (2020-2025): ->

Masters

Han Zhu (2014-2017): Co-supervised with Prof. Jianmin Wang. 清华大学优秀硕士学位论文, 清华大学优秀硕士毕业生 -> Alimama
Feng Li (2014-2017): Co-supervised with Prof. Jianmin Wang -> Alimama
Qiaoan Chen (2014-2017): Co-supervised with Prof. Jianmin Wang -> WeChat
Qingfu Wen (2015-2018): Co-supervised with Prof. Jianmin Wang -> Tencent
Jue Wang (2015-2018): Co-supervised with Prof. Jianmin Wang -> Government
Ziru Xu (2016-2019): Co-supervised with Prof. Jianmin Wang. 清华大学优秀硕士学位论文, 清华大学优秀硕士毕业生 -> Alimama
Jianjin Zhang (2016-2019): -> Microsoft
Zhifeng Gao (2016-2019): -> Alimama
Yuchen Zhang (2017-2020): Co-supervised with Prof. Jianmin Wang. 清华大学优秀硕士学位论文 -> THU PhD
Jintao Liu (2017-2020): Co-supervised with Prof. Jianmin Wang -> ByteDance
Liang Li (2017-2020): Co-supervised with Prof. Jianmin Wang -> Microsoft
Sinan Wang (2017-2020): 清华大学优秀硕士学位论文, 清华大学优秀硕士毕业生 -> Alibaba
Ying Jin (2018-2021): Co-supervised with Prof. Jianmin Wang -> PhD @ MMLab, CUHK
Xingqiang Du (2018-2021): Co-supervised with Prof. Jianmin Wang -> Kuaishou
Bo Fu (2018-2021): Co-supervised with Prof. Jianmin Wang. 清华大学优秀硕士学位论文, 清华大学优秀硕士毕业生 -> WeChat
Bin Liu (2018-2021): 北京市优秀毕业生 -> Alimama
Chao Huang (2019-2022): ->
Zhi Kou (2019-2022): ->
Jincheng Zhong (2019-2022): ->
Jinghan Gao (2019-2022): ->
Junguang Jiang (2020-2023): ->
Haixu Wu (2020-2023): ->
Jiehui Xu (2020-2023): ->

Undergraduates

Zhangjie Cao (2014-2018): 清华大学优良毕业生 -> Stanford PhD
Shichen Liu (2014-2018): 清华大学优良毕业生 -> USC PhD
Bin Liu (2014-2018): 清华大学优良毕业生 -> THU Master
Lijia Ma (2015-2019): -> UW PhD
Hongyu Zhu (2015-2019): -> CMU Master
Zhiyu Yao (2015-2019): -> THU PhD
Chao Huang (2015-2019): -> THU Master
Zhi Kou (2015-2019): -> THU Master
Kaichao You (2016-2020): 清华大学特等奖学金, 清华大学优秀毕业生 -> THU PhD
Haixu Wu (2016-2020): 清华大学优良毕业生 -> THU Master
Junguang Jiang (2016-2020): -> THU Master
Weirui Ye (2016-2020): -> THU PhD
Ziping Sun (2016-2020): -> THU Master
Tianle Liu (2016-2020): 清华大学优良毕业生 -> Harvard PhD
Hong Liu (2017-2021): 清华大学特等奖学金 -> Stanford PhD
Yong Liu (2017-2021): 清华大学优秀毕业生, 北京市普通高等学校优秀本科生毕业论文 -> THU PhD
Yifei Ji (2017-2021): 北京市优秀毕业生 -> THU Master
Shijie Wang (2017-2021): -> Brown PhD
Haoyu Niu (2017-2021): ->CMU Master
Jialong Wu (2018-2022): ->
Lanxiang Xing (2018-2022): ->
Baixu Chen (2018-2022): ->
Haoyu Ma (2018-2022): ->
Chenyu Wang (2018-2022): ->