Mingsheng Long
Associate Professor, PhD Supervisor
Machine Learning Group
School of Software
Tsinghua University
longmingsheng@gmail.com
mingsheng@tsinghua.edu.cn
Room 11-413, East Main Building, Tsinghua University, Beijing 100084, China
Biography
My research spans the theories, algorithms, and models of machine learning, with persistent dedication to enable adaptive learning machines in the ever-changing world. My current research is boosting advances in transfer, adaptation, and data-efficient learning, foundation deep models, and learning with spatiotemporal and physical knowledge.
Our Machine Learning Group (THUML) is interested in empowering machine learning for representation, perception, prediction, and decision making with good trade-off between accuracy, efficiency, generalizability, and transferability. We focus on the science and engineering problems of Artificial Intelligence (AI), including analysis and understanding of IoT, Spacetime, Sustainability data.My Quote
“Everything should be made as simple as possible, but no simpler.” --Albert Einstein
Research Interests
- Machine Learning: Transfer Learning, Deep Learning, Learning with Knowledge
Education
-
PhD in Computer Science, 2008-2014
Department of Computer Science and Technology, Tsinghua University, Beijing, China
Advisor: Prof. Jianmin Wang -
Bachelor in Electrical Engineering, 2004-2008
Department of Electrical Engineering, Tsinghua University, Beijing, China
Research Experience
-
Postdoc in Machine Learning, 2014-2015
Department of Computer Science, UC Berkeley
Advisor: Prof. Michael I. Jordan -
Postdoc in Machine Learning, 2014-2016
School of Software, Tsinghua University, Beijing, China
Advisor: Prof. Jianmin Wang
Recent News
- 2022.05: One papers got accepted by ICML, congrats! (more)
- 2022.03: Two papers got accepted by TPAMI, congrats! (more)
- 2022.01: Three papers got accepted by ICLR, congrats! (more)
- 2021.09: Two papers got accepted by NeurIPS, congrats! (more)
- 2021.09: One paper got accepted by TPAMI, congrats! (more)
- 2021.05: Four papers got accepted by ICML, congrats! (more)
- 2021.03: Five papers got accepted by CVPR, congrats! (more)
- 2020.09: Four papers got accepted by NeurIPS, congrats! (more)
Selected Publications [Full List] [DBLP] [Google Scholar]
(✉ Corresponding Author)
Thesis
-
Mingsheng Long. Transfer Learning: Problems and Methods. 1-127, 2014 [PDF] (In Chinese)
Journal Articles
-
PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning
Yunbo Wang, Haixu Wu, Jianjin Zhang, Zhifeng Gao, Jianmin Wang, Philip S. Yu, Mingsheng Long ✉
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 99(Preprint), 2022 [arXiv] -
From Big to Small: Adaptive Learning to Partial-Set Domains
Zhangjie Cao, Kaichao You, Ziyang Zhang, Jianmin Wang, Mingsheng Long ✉
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 99(Preprint), 2022 [arXiv] -
VideoDG: Generalizing Temporal Relations in Videos to Novel Domains
Zhiyu Yao, Yunbo Wang, Jianmin Wang, Philip S. Yu, Mingsheng Long ✉
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 99(Preprint), 2021 [Link] [Code] -
Transferable Representation Learning with Deep Adaptation Networks
Mingsheng Long, Yue Cao, Zhangjie Cao, Jianmin Wang, Michael I. Jordan
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 41(12):3071-3085, 2019 [Link] [Code]
Conference Proceedings
-
Flowformer: Linearizing Transformers with Conservation Flows
Haixu Wu, Jialong Wu, Jiehui Xu, Jianmin Wang, Mingsheng Long ✉
International Conference on Machine Learning (ICML), 2022 [arXiv] (Accepted) -
Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy
Jiehui Xu, Haixu Wu, Jianmin Wang, Mingsheng Long ✉
International Conference on Learning Representations (ICLR), 2022 [arXiv] -
X-model: Improving Data Efficiency in Deep Learning with A Minimax Model
Ximei Wang, Xinyang Chen, Jianmin Wang, Mingsheng Long ✉
International Conference on Learning Representations (ICLR), 2022 [arXiv] -
Decoupled Adaptation for Cross-Domain Object Detection
Junguang Jiang, Baixu Chen, Jianmin Wang, Mingsheng Long ✉
International Conference on Learning Representations (ICLR), 2022 [arXiv] -
Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting
Haixu Wu, Jiehui Xu, Jianmin Wang, Mingsheng Long ✉
Neural Information Processing Systems (NeurIPS), 2021 [PDF] [Appendix] [arXiv] [Code] -
Cycle Self-Training for Domain Adaptation
Hong Liu, Jianmin Wang, Mingsheng Long ✉
Neural Information Processing Systems (NeurIPS), 2021 [PDF] [Appendix] [arXiv] [Code] -
LogME: Practical Assessment of Pre-trained Models for Transfer Learning
Kaichao You, Yong Liu, Jianmin Wang, Mingsheng Long ✉
International Conference on Machine Learning (ICML), 2021 [PDF] [Appendix] [Code] -
Zoo-Tuning: Adaptive Transfer from A Zoo of Models
Yang Shu, Zhi Kou, Zhangjie Cao, Jianmin Wang, Mingsheng Long ✉
International Conference on Machine Learning (ICML), 2021 [PDF] [Appendix] [Code] -
Self-Tuning for Data-Efficient Deep Learning
Ximei Wang, Jinghan Gao, Jianmin Wang, Mingsheng Long ✉
International Conference on Machine Learning (ICML), 2021 [PDF] [Appendix] [Code] -
Representation Subspace Distance for Domain Adaptation Regression
Xinyang Chen, Sinan Wang, Jianmin Wang, Mingsheng Long ✉
International Conference on Machine Learning (ICML), 2021 [PDF] [Appendix] [Code] -
Transferable Calibration with Lower Bias and Variance in Domain Adaptation
Ximei Wang, Mingsheng Long ✉, Jianmin Wang, Michael I. Jordan
Neural Information Processing Systems (NeurIPS), 2020 [PDF] [Appendix] [Code] -
Co-Tuning for Transfer Learning
Kaichao You, Zhi Kou, Mingsheng Long ✉, Jianmin Wang
Neural Information Processing Systems (NeurIPS), 2020 [PDF] [Code] -
Stochastic Normalization
Zhi Kou, Kaichao You, Mingsheng Long ✉, Jianmin Wang
Neural Information Processing Systems (NeurIPS), 2020 [PDF] [Code] -
Learning to Adapt to Evolving Domains
Hong Liu, Mingsheng Long ✉, Jianmin Wang, Yu Wang
Neural Information Processing Systems (NeurIPS), 2020 [PDF] [Code] -
Unsupervised Transfer Learning for Spatiotemporal Predictive Networks
Zhiyu Yao, Yunbo Wang, Mingsheng Long ✉, Jianmin Wang
International Conference on Machine Learning (ICML), 2020 [PDF] [Code] -
Catastrophic Forgetting Meets Negative Transfer: Batch Spectral Shrinkage for Safe Transfer Learning
Xinyang Chen, Sinan Wang, Bo Fu, Mingsheng Long ✉, Jianmin Wang
Neural Information Processing Systems (NeurIPS), 2019 [PDF] [Code] -
Transferable Normalization: Towards Improving Transferability of Deep Neural Networks
Ximei Wang, Ying Jin, Mingsheng Long ✉, Jianmin Wang, Michael I. Jordan
Neural Information Processing Systems (NeurIPS), 2019 [PDF] [Code] -
Bridging Theory and Algorithm for Domain Adaptation
Yuchen Zhang, Tianle Liu, Mingsheng Long ✉, Michael I. Jordan
International Conference on Machine Learning (ICML), 2019 [PDF] [Appendix] [Code] (Long Oral) -
Towards Accurate Model Selection in Deep Unsupervised Domain Adaptation
Kaichao You, Ximei Wang, Mingsheng Long ✉, Michael I. Jordan
International Conference on Machine Learning (ICML), 2019 [PDF] [Code] -
Transferable Adversarial Training: A General Approach to Adapting Deep Classifiers
Hong Liu, Mingsheng Long ✉, Jianmin Wang, Michael I. Jordan
International Conference on Machine Learning (ICML), 2019 [PDF] [Code] (Long Oral) -
Transferability vs. Discriminability: Batch Spectral Penalization for Adversarial Domain Adaptation
Xinyang Chen, Sinan Wang, Mingsheng Long ✉, Jianmin Wang
International Conference on Machine Learning (ICML), 2019 [PDF] [Code] -
Conditional Adversarial Domain Adaptation
Mingsheng Long, Zhangjie Cao, Jianmin Wang, Michael I. Jordan
Neural Information Processing Systems (NeurIPS), 2018 [PDF] [Poster] [Code]
PaperDigest Most Influential Papers -
Generalized Zero-Shot Learning with Deep Calibration Network
Shichen Liu, Mingsheng Long ✉, Jianmin Wang, Michael I. Jordan
Neural Information Processing Systems (NeurIPS), 2018 [PDF] [Code] -
PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learning
Yunbo Wang, Zhifeng Gao, Mingsheng Long ✉, Jianmin Wang, Philip S. Yu
International Conference on Machine Learning (ICML), 2018 [PDF] [Code] -
PredRNN: Recurrent Neural Networks for Predictive Learning using Spatiotemporal LSTMs
Yunbo Wang, Mingsheng Long ✉, Jianmin Wang, Zhifeng Gao, Philip S. Yu
Neural Information Processing Systems (NeurIPS), 2017 [PDF] [Code] -
Learning Multiple Tasks with Multilinear Relationship Networks
Mingsheng Long, Zhangjie Cao, Jianmin Wang, Philip S. Yu
Neural Information Processing Systems (NeurIPS), 2017 [PDF] [Poster] [Code] -
Deep Transfer Learning with Joint Adaptation Networks
Mingsheng Long, Han Zhu, Jianmin Wang, Michael I. Jordan
International Conference on Machine Learning (ICML), 2017 [PDF] [Slides] [Poster] [Code] (Long Oral)
PaperDigest Most Influential Papers -
Unsupervised Domain Adaptation with Residual Transfer Networks
Mingsheng Long, Han Zhu, Jianmin Wang, Michael I. Jordan
Neural Information Processing Systems (NeurIPS), 2016 [PDF] [Poster] [Code] -
Learning Transferable Features with Deep Adaptation Networks
Mingsheng Long, Yue Cao, Jianmin Wang, Michael I. Jordan
International Conference on Machine Learning (ICML), 2015 [PDF] [Slides] [Poster] [Code] (Long Oral)
PaperDigest Most Influential Papers
Invited Talks
- Transfer Learning: Theories, Algorithms, and Open Library, FTL-IJCAI 2021 [PDF]
- Transfer Learning: Theories and Algorithms, Chinese Conference on Data Mining, CCDM 2020 [PDF]
- Transfer Learning: Theories and Algorithms, Conference on Pattern Recognition and Computer Vision, PRCV 2019 [PDF]
- Transfer Learning: Theories and Algorithms, CCF Conference on Artificial Intelligence, CCFAI 2019 [PDF]
- Transfer Learning: From Algorithms to Theories and Back, Vision And Learning SEminar, VALSE 2019 [PDF]
- Predictive Learning for Video Prediction, Chinese Conference on Data Mining, CCDM 2018 [PDF]
- Deep Hashing for Multimedia Retrieval, International Conference on Data Science, 2017 [PDF]
Software
- Transfer-Learning-Library (TLlib): An open-source and easy-to-use library for transfer learning [Code @ GitHub] [Documentation]
- Predictive-Learning-Library (Pred-Learn): An open-source and easy-to-use package for predictive learning [Code @ GitHub]
- DeepHash: An open-source package for deep learning to hash [Code @ GitHub]
Awards
- Test of Time Award, FTL-IJCAI, 2021
- AI 2000 Most Influential Scholar Award in Machine Learning (rank 1 in domestic universities), 2021
- Excellent Young Scholar, National Natural Science Foundation, China, 2020
- Rising Star in Science and Technology, Beijing, 2020
- Popular Dissertation Top 20, CNKI, China, 2019
- First Prize, Technical Invention Award, Ministry of Education (MOE), China, 2018
- Distinguished Dissertation Award, China Association for Artificial Intelligence (CAAI), 2016
- Distinguished Dissertation Nominee, China Computer Federation (CCF), 2014
- Outstanding PhD Graduate, Tsinghua University, 2014
- Distinguished Dissertation Award, Tsinghua University, 2014
- Best Paper Nominee, SIAM SDM, 2012
Professional Services
PC Chair
- Transfer and Multi-Task Learning Workshop, NeurIPS 2015
- Transferring and Adapting Source Knowledge in Computer Vision Workshop and Challenge (TASK-CV), ICCV 2017
Area Chair
- International Conference on Machine Learning (ICML) 2021, 2022
- Neural Information Processing Systems (NeurIPS) 2020, 2021
- International Conference on Learning Representations (ICLR) 2020, 2021, 2022
- AAAI Conference on Artificial Intelligence (AAAI) 2022
- International Joint Conference on Artificial Intelligence (IJCAI) 2021
Associate Editor
- IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 2021
- Transactions on Machine Learning Research (TMLR) 2022
Guest Editor
- IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 2020, 2021
(Senior) PC Member | Reviewer
- International Conference on Machine Learning (ICML) 2014, 2017, 2018, 2019, 2020
- Neural Information Processing Systems (NeurIPS) 2016, 2017, 2018, 2019
- IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2018, 2019, 2020, 2021, 2022
- IEEE International Conference on Computer Vision (ICCV) 2019, 2021
- European Conference on Computer Vision (ECCV) 2018, 2020
- ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) 2018, 2019, 2020
- AAAI Conference on Artificial Intelligence (AAAI) 2018, 2019, 2020, 2021
- International Joint Conference on Artificial Intelligence (IJCAI) 2017, 2018, 2019, 2020
- Artificial Intelligence Journal (AIJ) 2016, 2018
- Journal of Machine Learning Research (JMLR) 2018, 2020, 2021
- International Journal of Computer Vision (IJCV) 2019, 2020
- IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 2016, 2017, 2018, 2019, 2020, 2021
- IEEE Transactions on Knowledge and Data Engineering (TKDE) 2014, 2015, 2016, 2017, 2018, 2019, 2020
- IEEE Transactions on Neural Networks and Learning Systems (TNNLS) 2016, 2017, 2018, 2019
- IEEE Transactions on Image Processing (TIP) 2014, 2015, 2016, 2017, 2018, 2019
- ACM Computing Surveys 2019, 2020
- Nature Communications 2019, 2020
- Nature Human Behavior 2019, 2020
Research Grants
- Principal Investigator, Transfer Learning, National Natural Science Foundation for Excellent Young Scholars, 2021-2023
- Principal Investigator, Industrial Big Data Transfer Learning: Theories and Algorithms, Beijing Nova Program, 2021-2023
- Principal Investigator, Adaptive Perception and Intelligent Scheduling for Manufacturing Tasks, National Major Program for New Generation AI, 2020-2023
- Principal Investigator, Robust Transfer Learning for OOD Data in Open Dynamic Environment, MOE Strategic Research Project on Artificial Intelligence Algorithms (Project PI: Prof. Yang Yu @ Nanjing University), 2020-2021
- Principal Investigator, Recommendation Algorithm based on Multi-Task Learning, Alibaba Innovative Research (AIR), 2020-2021
- Principal Investigator, Recommendation Algorithm based on Deep Learning, Alibaba Innovative Research (AIR), 2018-2019
- Principal Investigator, Intelligent Severe Weather Nowcasting System, China Meteorological Administration, 2019-2020
- Principal Investigator, Spatiotemporal Deep Learning for Severe Weather Nowcasting, National Key R&D Plan, 2018-2022
- Principal Investigator, Spatiotemporal Deep Learning: Algorithms and Applications, Natural Science Foundation of China, 2018-2021
- Principal Investigator, Safe Transfer Learning for Big Data Analytics, Natural Science Foundation of China, 2016-2018
- Principal Investigator, Scalable Transfer Learning: Theory and Algorithms, China Postdoc Science Foundation, 2015-2016
Teaching
- Instructor: Deep Learning, 2018- (清华大学年度教学优秀奖)
- Instructor: Machine Learning, 2019- (Top 25%)
- Instructor: Introduction to Artificial Intelligence, 2021- (Top 25%)
- Instructor: Introduction to Data Science, 2018-2020
- Co-Instructor: Introduction to Big Data Technology, 2017-2020
- Co-Instructor: Foundations of Big Data Systems (A), 2016-2020
- Co-Instructor: Foundations of Big Data Systems (B), 2015-2020
Students
PhDs
- Yue Cao (2014-2019): Co-supervised with Prof. Jianmin Wang. 清华大学特等奖学金, 清华大学优秀博士学位论文, 清华大学优秀博士毕业生 -> Researcher, MSRA
- Yunbo Wang (2015-2020): Co-supervised with Prof. Philip S. Yu. 麻省理工学院访问学者, 清华大学优秀博士学位论文, 中国计算机学会优秀博士学位论文奖 -> Assistant Professor, Shanghai Jiao Tong University
- Zhongyi Pei (2015-2020): Co-supervised with Prof. Philip S. Yu -> THU Postdoc
- Ximei Wang (2017-2022): Co-supervised with Prof. Jianmin Wang ->
- Xinyang Chen (2017-2022): ->
- Yang Shu (2018-2023): ->
- Zhiyu Yao (2019-2024): ->
- Yuchen Zhang (2020-2024): Co-supervised with Prof. Jianmin Wang ->
- Kaichao You (2020-2025): ->
- Yong Liu (2021-2026): ->
Masters
- Han Zhu (2014-2017): Co-supervised with Prof. Jianmin Wang. 清华大学优秀硕士学位论文, 清华大学优秀硕士毕业生 -> Alimama
- Feng Li (2014-2017): Co-supervised with Prof. Jianmin Wang -> Alimama
- Qiaoan Chen (2014-2017): Co-supervised with Prof. Jianmin Wang -> WeChat
- Qingfu Wen (2015-2018): Co-supervised with Prof. Jianmin Wang -> Tencent
- Jue Wang (2015-2018): Co-supervised with Prof. Jianmin Wang -> Government
- Ziru Xu (2016-2019): Co-supervised with Prof. Jianmin Wang. 清华大学优秀硕士学位论文, 清华大学优秀硕士毕业生 -> Alimama
- Jianjin Zhang (2016-2019): -> Microsoft
- Zhifeng Gao (2016-2019): -> Alimama
- Yuchen Zhang (2017-2020): Co-supervised with Prof. Jianmin Wang. 清华大学优秀硕士学位论文 -> THU PhD
- Jintao Liu (2017-2020): Co-supervised with Prof. Jianmin Wang -> ByteDance
- Liang Li (2017-2020): Co-supervised with Prof. Jianmin Wang -> Microsoft
- Sinan Wang (2017-2020): 清华大学优秀硕士学位论文, 清华大学优秀硕士毕业生 -> Alibaba
- Ying Jin (2018-2021): Co-supervised with Prof. Jianmin Wang -> PhD @ MMLab, CUHK
- Xingqiang Du (2018-2021): Co-supervised with Prof. Jianmin Wang -> Kuaishou
- Bo Fu (2018-2021): Co-supervised with Prof. Jianmin Wang. 清华大学优秀硕士学位论文, 清华大学优秀硕士毕业生 -> WeChat
- Bin Liu (2018-2021): 北京市优秀毕业生 -> Alimama
- Chao Huang (2019-2022): ->
- Zhi Kou (2019-2022): ->
- Jincheng Zhong (2019-2022): ->
- Jinghan Gao (2019-2022): ->
- Junguang Jiang (2020-2023): ->
- Haixu Wu (2020-2023): ->
- Jiehui Xu (2020-2023): ->
- Tengge Hu (2021-2024):
- Kaiyuan Chen (2021-2024):
Undergraduates
- Zhangjie Cao (2014-2018): 清华大学优良毕业生 -> Stanford PhD
- Shichen Liu (2014-2018): 清华大学优良毕业生 -> USC PhD
- Bin Liu (2014-2018): 清华大学优良毕业生 -> THU Master
- Lijia Ma (2015-2019): -> UW PhD
- Hongyu Zhu (2015-2019): -> CMU Master
- Zhiyu Yao (2015-2019): -> THU PhD
- Chao Huang (2015-2019): -> THU Master
- Zhi Kou (2015-2019): -> THU Master
- Kaichao You (2016-2020): 清华大学特等奖学金, 清华大学优秀毕业生 -> THU PhD
- Haixu Wu (2016-2020): 清华大学优良毕业生 -> THU Master
- Junguang Jiang (2016-2020): -> THU Master
- Weirui Ye (2016-2020): -> THU PhD
- Ziping Sun (2016-2020): -> THU Master
- Tianle Liu (2016-2020): 清华大学优良毕业生 -> Harvard PhD
- Hong Liu (2017-2021): 清华大学特等奖学金 -> Stanford PhD
- Yong Liu (2017-2021): 清华大学优秀毕业生, 北京市普通高等学校优秀本科生毕业论文 -> THU PhD
- Yifei Ji (2017-2021): 北京市优秀毕业生 -> THU Master
- Shijie Wang (2017-2021): -> Brown PhD
- Haoyu Niu (2017-2021): -> CMU Master
- Jialong Wu (2018-2022): ->
- Lanxiang Xing (2018-2022): ->
- Baixu Chen (2018-2022): ->
- Haoyu Ma (2018-2022): ->
- Qi Li (2018-2022): ->
- Chenyu Wang (2018-2022): ->
- Han Zhu (2014-2017): Co-supervised with Prof. Jianmin Wang. 清华大学优秀硕士学位论文, 清华大学优秀硕士毕业生 -> Alimama
- Feng Li (2014-2017): Co-supervised with Prof. Jianmin Wang -> Alimama
- Qiaoan Chen (2014-2017): Co-supervised with Prof. Jianmin Wang -> WeChat
- Qingfu Wen (2015-2018): Co-supervised with Prof. Jianmin Wang -> Tencent
- Jue Wang (2015-2018): Co-supervised with Prof. Jianmin Wang -> Government
- Ziru Xu (2016-2019): Co-supervised with Prof. Jianmin Wang. 清华大学优秀硕士学位论文, 清华大学优秀硕士毕业生 -> Alimama
- Jianjin Zhang (2016-2019): -> Microsoft
- Zhifeng Gao (2016-2019): -> Alimama
- Yuchen Zhang (2017-2020): Co-supervised with Prof. Jianmin Wang. 清华大学优秀硕士学位论文 -> THU PhD
- Jintao Liu (2017-2020): Co-supervised with Prof. Jianmin Wang -> ByteDance
- Liang Li (2017-2020): Co-supervised with Prof. Jianmin Wang -> Microsoft
- Sinan Wang (2017-2020): 清华大学优秀硕士学位论文, 清华大学优秀硕士毕业生 -> Alibaba
- Ying Jin (2018-2021): Co-supervised with Prof. Jianmin Wang -> PhD @ MMLab, CUHK
- Xingqiang Du (2018-2021): Co-supervised with Prof. Jianmin Wang -> Kuaishou
- Bo Fu (2018-2021): Co-supervised with Prof. Jianmin Wang. 清华大学优秀硕士学位论文, 清华大学优秀硕士毕业生 -> WeChat
- Bin Liu (2018-2021): 北京市优秀毕业生 -> Alimama
- Chao Huang (2019-2022): ->
- Zhi Kou (2019-2022): ->
- Jincheng Zhong (2019-2022): ->
- Jinghan Gao (2019-2022): ->
- Junguang Jiang (2020-2023): ->
- Haixu Wu (2020-2023): ->
- Jiehui Xu (2020-2023): ->
- Tengge Hu (2021-2024):
- Kaiyuan Chen (2021-2024):