Liangyu Wang

Liangyu Wang

LLM, MLsys

Publications

ZO2: Scalable Zeroth-Order Fine-Tuning for Extremely Large Language Models with Limited GPU Memory

Published in NeurIPS workshop, 2024; arXiv preprint, 2025

This paper presents a novel framework for efficient zeroth-order fine-tuning of extremely large language models with limited GPU memory.

Recommended citation: Liangyu Wang, Jie Ren, Hang Xu, Junxiao Wang, Huanyi Xie, David E. Keyes, and Di Wang. (2025). "ZO2: Scalable Zeroth-Order Fine-Tuning for Extremely Large Language Models with Limited GPU Memory." arXiv preprint arXiv:2503.12668
Download Paper

FlashDP: Memory-Efficient and High-Throughput DP-SGD Training for Large Language Models

Published in NeurIPS workshop 2024, 2024

This paper presents a memory-efficient and high-throughput approach for training large language models with differential privacy guarantees.

Recommended citation: Liangyu Wang, Junxiao Wang, Jie Ren, Zihang Xiang, David E. Keyes, and Di Wang. (2024). "FlashDP: Memory-Efficient and High-Throughput DP-SGD Training for Large Language Models." NeurIPS workshop 2024.
Download Paper