Shih-Yang (Sean) Liu

I'm a Research Scientist at NVIDIA Research, where I work on efficient deep learning and reinforcement learning for LLM reasoning.

I received my PhD from HKUST, where I was a member of the Vision and System Design Lab (VSDL), advised by Prof. Tim Kwang-Ting CHENG. I was also mentored by Zechun Liu, and we worked closely on model quantization.

Email  /  Google Scholar  /  Twitter(X)  /  Github

Research

I'm interested in efficient deep learning: model compression, parameter-efficient finetuning, and more recently reinforcement learning for efficient LLM reasoning. Most of my research is about getting more out of a model per unit of compute, whether at training or inference time. Representative papers are highlighted.

We show that applying GRPO-style normalization to a summed multi-reward signal collapses distinct reward combinations into identical advantages, blinding the policy to per-reward progress. GDPO instead normalizes each reward independently within the group before aggregation, a drop-in replacement for GRPO that improves per-reward convergence and lifts AIME accuracy by up to 6.3%.

We presented DoRA, a new parameter-efficient fine-tuning approach, which consistently outperforms LoRA in fine-tuning LLM without incurring additional inference costs. These improvements are particularly notable for smaller ranks with 37.2% improvement over LoRA for rank 8 and 22.4% improvement for rank 4.

We show that the accuracy drop under length penalties comes from inadequate RL optimization rather than the penalty design itself, identifying advantage-estimation bias, entropy collapse, and sparse reward as the culprits. DLER combines batch-wise reward normalization, higher clipping, dynamic sampling, and a simple truncation penalty to cut chain-of-thought length by 50-70% with no loss in accuracy.

EoRA augments a compressed LLM with task-specific low-rank matrices derived in the eigenspace of the compression error, recovering accuracy in minutes without any fine-tuning. It improves 3-bit LLaMA3-8B by 10.84% on ARC-Challenge, 6.74% on MathQA, and 11.45% on GSM8K over prior training-free low-rank methods.

In this study, we address weight oscillation in quantization-aware training and its negative impact on model performance. We propose three techniques: statistical weight quantization (StatsQ), confidence-guided annealing (CGA), and query-key reparameterization (QKR). These techniques improve quantization robustness and accuracy in the ViT model. The proposed 2-bit DeiT-T/DeiT-S algorithms outperform the previous state-of-the-art by 9.8% and 7.7%, respectively.

We introduced LLM-FP4, a novel post-training quantization framework which for the first time is capable of quantizing both the activation and weight of LLM to 4 bits without substantial loss in accuracy, outperforming previous methods by up to 13.1%.

Thanks to Barron's website template.