Welcome to Mingrui Liu's Homepage

About Me

I am an assistant professor at Department of Computer Science, George Mason University since Fall 2021. Before that I was a postdoc at Rafik B. Hariri Institute at Boston University from 2020-2021, hosted by Francesco Orabona. I received my Ph.D. at Department of Computer Science, The University of Iowa in August 2020, under the advise of Tianbao Yang. Before that I studied at Institute of Natural Sciences and School of Mathematical Sciences at Shanghai Jiao Tong University. I have also spent time working at industrial research labs, such as IBM research AI and Alibaba DAMO Academy. Here is my Google Scholar Citations.

I am looking for self-motivated PhD students (fully-funded) with strong mathematical ablities and (or) programming skills to solve challenging machine learning problems elegantly with mathematical analysis and empirical studies. The main technical skills we need include mathematical optimization, statistical learning theory, algorithms, and deep learning. If you are interested, please drop me an email with your CV and transcript, and apply our PhD program here. Undergrad and graduate student visitors are also welcome. This link provides an overview of our fast-growing GMU CS department.

Research

My research addresses the fundamental problem of efficiency in machine learning, with a focus on the interplay of optimization and learning. I design algorithms with provable performance guarantees that tackle challenges arising in modern learning paradigms, spanning computational, statistical, and scalability dimensions. My research goal is to design provably efficient algorithms for modern machine learning problems with strong empirical performance. My research topics include:

Optimization for Machine Learning: I design and analyze optimization algorithms with rigorous efficiency guarantees for modern learning problems, including the training of language models such as Transformers [NeurIPS'22], hierarchical optimization (e.g., learning problems with minimax and bilevel formulation) [ICLR'24 Spotlight, ICML'24, JMLR'21], optimization methods with adaptivity guarantees and its fundamental limits [ICLR'25, ICLR'20], and global convergence of neural network optimization [NeurIPS'23].

Algorithmic Generalization and Feature Learning: I study how optimization and learning dynamics shape both generalization performance and the emergence of features in modern models. My work characterizes the sample and computational complexity of problems such as pairwise learning [NeurIPS'21], and develops feature learning theory for neural networks [ICML'24].

Distributed and Scalable Learning: I design scalable algorithms for distributed learning that remain efficient under real-world constraints, such as limited communication [NeurIPS'22 Spotlight], decentralization [NeurIPS'20], and node unavailability [NeurIPS'24]. Recently, I focus on moving beyond black-box analyses to develop principled, fine-grained theoretical frameworks for distributed learning problems [ICML'25, ICLR'25].

Adaptive Learning: I work on designing algorithms that learn efficiently by selecting informative data and adapting to data distribution shifts. My work includes coreset selection for data-efficient training [NeurIPS'23], continual learning without catastrophic forgetting [NeurIPS'20 Spotlight].

Recent Selected Publications [Full List]

(New! ) Constant Stepsize Local GD for Logistic Regression: Acceleration by Instability
Michael Crawshaw^#, Blake Woodworth, Mingrui Liu.
Proceedings of 42th International Conference on Machine Learning, 2025. (ICML 2025)
(New! ) Local Steps Speed Up Local GD for Heterogeneous Distributed Logistic Regression
Michael Crawshaw^#, Blake Woodworth, Mingrui Liu.
In 13th International Conference on Learning Representations, 2025. (ICLR 2025)
(New! ) Complexity Lower Bounds of Adaptive Gradient Algorithms for Non-convex Stochastic Optimization under Relaxed Smoothness
Michael Crawshaw^#, Mingrui Liu.
In 13th International Conference on Learning Representations, 2025. (ICLR 2025)
Federated Learning under Periodic Client Participation and Heterogeneous Data: A New Communication-Efficient Algorithm and Analysis
Michael Crawshaw^#, Mingrui Liu.
In Advances in Neural Information Processing Systems 37, 2024. (NeurIPS 2024)
An Accelerated Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness
Xiaochuan Gong^#, Jie Hao^#, Mingrui Liu.
In Advances in Neural Information Processing Systems 37, 2024. (NeurIPS 2024)
Provable Benefits of Local Steps in Heterogeneous Federated Learning for Neural Networks: A Feature Learning Perspective
Yajie Bao^#, Michael Crawshaw^#, Mingrui Liu.
Proceedings of 41th International Conference on Machine Learning, 2024. (ICML 2024)
A Nearly Optimal Single Loop Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness
Xiaochuan Gong^#, Jie Hao^#, Mingrui Liu.
Proceedings of 41th International Conference on Machine Learning, 2024. (ICML 2024)
Bilevel Optimization under Unbounded Smoothness: A New Algorithm and Convergence Analysis
Jie Hao^#, Xiaochuan Gong^#, Mingrui Liu.
In 12th International Conference on Learning Representations, 2024. (ICLR 2024) (Spotlight, 5% acceptance rate)
Federated Learning with Client Subsampling, Data Heterogeneity, and Unbounded Smoothness: A New Algorithm and Lower Bounds
Michael Crawshaw^#, Yajie Bao^#, Mingrui Liu.
In Advances in Neural Information Processing Systems 36, 2023. (NeurIPS 2023)
Global Convergence Analysis of Local SGD for Two-layer Neural Network without Overparameterization
Yajie Bao^#, Amarda Shehu, Mingrui Liu.
In Advances in Neural Information Processing Systems 36, 2023. (NeurIPS 2023)
Bilevel Coreset Selection in Continual Learning: A New Formulation and Algorithm
Jie Hao^#, Kaiyi Ji, Mingrui Liu.
In Advances in Neural Information Processing Systems 36, 2023. (NeurIPS 2023)
EPISODE: Episodic Gradient Clipping with Periodic Resampled Corrections for Federated Learning with Heterogeneous Data
Michael Crawshaw^#, Yajie Bao^#, Mingrui Liu.
In 11th International Conference on Learning Representations, 2023. (ICLR 2023)
A Communication-Efficient Distributed Gradient Clipping Algorithm for Training Deep Neural Networks
Mingrui Liu, Zhenxun Zhuang, Yunwen Lei, Chunyang Liao.
In Advances in Neural Information Processing Systems 35, 2022. (NeurIPS 2022) (Spotlight, 5% acceptance rate)
Robustness to Unbounded Smoothness of Generalized SignSGD
Michael Crawshaw^#*, Mingrui Liu^*, Francesco Orabona^*, Wei Zhang^*, Zhenxun Zhuang^*.
In Advances in Neural Information Processing Systems 35, 2022. (NeurIPS 2022)
Will Bilevel Optimizers Benefit from Loops
Kaiyi Ji, Mingrui Liu, Yingbin Liang, Lei Ying.
In Advances in Neural Information Processing Systems 35, 2022. (NeurIPS 2022) (Spotlight, 5% acceptance rate)
Fast Composite Optimization and Statistical Recovery in Federated Learning
Yajie Bao^#, Michael Crawshaw^#, Shan Luo, Mingrui Liu.
Proceedings of 39th International Conference on Machine Learning, 2022. (ICML 2022)
Understanding AdamW through Proximal Methods and Scale-Freeness
Zhenxun Zhuang, Mingrui Liu, Ashok Cutkosky, Francesco Orabona.
Transactions on Machine Learning Research, 2022. (TMLR 2022)
On the Initialization for Convex-Concave Min-max Problems
Mingrui Liu, Francesco Orabona.
Algorithmic Learning Theory, 2022. (ALT 2022)
On the Last Iterate Convergence of Momentum Methods
Xiaoyu Li, Mingrui Liu, Francesco Orabona.
Algorithmic Learning Theory, 2022. (ALT 2022)
Generalization Guarantee of SGD for Pairwise Learning
Yunwen Lei, Mingrui Liu, Yiming Ying.
Advances in Neural Information Processing Systems 34, 2021. (NeurIPS 2021)

Last update: 08-01-2025

Mingrui Liu

Ph.D.

EMAIL

About Me

Research

Recent News

Recent Selected Publications [Full List]