Research Interest

• Efficient Language Models: [NeurIPS 2025] [ICLR'25 Spotlight] [NeurIPS'24]
• Efficient Computer Vision: [ICCV 2025] [ECCV'24 Oral] [ICML'23][CVPR'23] [ICCV'21] [ICML'20]
• Model Compression: [NeurIPS'22] [ICML'22] [ICLR'22] [NeurIPS'21] [ICML'21] [ICLR'21 Spotlight] [NeurIPS'20]
• Algorithm/Hardware Co-design: [ISCA'23] [ICCAD'23] [MICRO'21] [ICML'21]

News

• [Nov. 2025] Our Nemotron-Flash-1B/3B/3B-Instruct are publicly available on HuggingFace!
• [Sep. 2025] Two works Nemotron-Flash and Nemotron-CLIMB are accepted by NeurIPS 2025.
• [May 2025] I have joined NVIDIA Research as a Research Scientist!
• [Apr. 2025] I have completed my PhD defense, titled "Hierarchical and Hardware-Aware Optimization for Enhancing AI Model Efficiency: From Bits to Modules to Models"!
• [Feb. 2025] Our work Hymba is selected as ICLR 2025 Spotlight!
• [Jan. 2025] Two works Hymba and LongMamba are accepted by ICLR 2025.
• [Nov. 2024] Our developed small language model Hymba, which is the best sub-2B langauge model in 2024, is available on HuggingFace [Hymba-1.5B-Base][Hymba-1.5B-Instruct]!
• [Sep. 2024] Two works AmoemaLLM and Rad-NeRF are accepted by NeurIPS 2024.
• [Jul. 2024] Our work Omni-Recon is accepted by ECCV 2024 as an Oral Paper!
• [May  2024] I will join NVIDIA Research as a Research Intern, working on efficient language models.
• [Mar. 2024] We will present a tutorial titled "Dive Deep into Real-Time Neural Rendering on NVIDIA GPUs" at NVIDIA GTC 2024!
• [Aug. 2023] I am selected to receive the IBM PhD Fellowship Award!
• [Jul. 2023] Our Gen-NeRF demo won 2nd place in the 33rd ACM SIGDA/IEEE CEDA University Demonstration at DAC 2023!
• [Jul. 2023] Our work GPT4AIGChip is accepted by ICCAD 2023.
• [Jul. 2023] I will co-chair DAC 2023 Ph.D. Forum and give an invited talk on AutoDL at DAC 2023. See you in San Francisco!
• [May  2023] I am selected as Machine Learning and Systems Rising Stars 2023.
• [Apr. 2023] Two works NeRFool and Master-ASR are accepted by ICML 2023.
• [Mar. 2023] Our work Gen-NeRF is accepted by ISCA 2023.
• [Feb. 2023] Two works Auto-CARD and Hint-Aug are accepted by CVPR 2023.
• [Feb. 2023] Two works Robust Transferred Tickets and NetBooster are accepted by DAC 2023.
• [Sep. 2022] Our work S3-Router for efficient automated speech recognition (in collaboration with IBM) is accepted by NeurIPS 2022.
• [May  2022] Our work DepthShrinker (in collaboration with Meta) is accepted by ICML 2022.
• [Jan. 2022] Our work Patch-Fool is accepted by ICLR 2022.

Selected Publications (see full publication list here; * indicates equal contribution)

Yonggan Fu*, Xin Dong*, Shizhe Diao, Matthijs Van keirsbilck, Hanrong Ye, Wonmin Byeon, Yashaswi Karnati, Lucas Liebenwein, Hannah Zhang, Nikolaus Binder, Maksim Khadkevich, Alexander Keller, Jan Kautz, Yingyan (Celine) Lin, Pavlo Molchanov
Nemotron-Flash: Towards Latency-Optimal Hybrid Small Language Models
NeurIPS 2025 [Paper] [Nemotron-Flash-1B][Nemotron-Flash-3B][Nemotron-Flash-3B-Instruct]
Integration into TensorRT-LLM

---

Shizhe Diao, Yu Yang, Yonggan Fu, Xin Dong, Dan Su, Markus Kliegl, Zijia Chen, Peter Belcak, Yoshi Suhara, Hongxu Yin, Mostofa Patwary, Yingyan (Celine) Lin, Jan Kautz, Pavlo Molchanov
CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-training
NeurIPS 2025 Datasets & Benchmarks Track [Paper] [Nemotron-ClimbMix][Nemotron-ClimbLab]
Spotlight Paper

---

Zhenbang Du*, Yonggan Fu*, Lifu Wang*, Jiayi Qian, Xiao Luo, Yingyan (Celine) Lin
Fewer Denoising Steps or Cheaper Per-Step Inference: Towards Compute-Optimal Diffusion Model Deployment
ICCV 2025 [Paper] [Code]

---

Yonggan Fu*, Xin Dong*, Shizhe Diao, Wonmin Byeon, Zijia Chen, Ameya Sunil Mahabaleshwarkar, Shih-Yang Liu, Matthijs Van Keirsbilck, Min-Hung Chen, Yoshi Suhara, Yingyan (Celine) Lin, Jan Kautz, Pavlo Molchanov
Hymba: A Hybrid-head Architecture for Small Language Models
ICLR 2025 [Paper] [Code] [Hymba-1.5B-Base][Hymba-1.5B-Instruct]
Spotlight Paper

---

Yonggan Fu, Zhongzhi Yu, Jiayi Qian, Junwei Li, Yongan Zhang, Dachuan Shi, Xiangchi Yuan, Roman Yakunin, Yingyan (Celine) Lin
AmoebaLLM: Constructing Any-Shape Large Language Models for Efficient and Instant Deployment
NeurIPS 2024 [Paper] [Code] [Video] [Slide]

---

Yonggan Fu, Huaizhi Qu, Zhifan Ye, Chaojian Li, Kevin Zhao, Yingyan (Celine) Lin
Omni-Recon: Harnessing Image-based Rendering for General-Purpose Neural Radiance Fields
ECCV 2024 [Paper] [Code] [Video] [Slide]
Oral Paper

---

Yonggan Fu*, Yongan Zhang*, Zhongzhi Yu*, Sixu Li, Zhifan Ye, Chaojian Li, Cheng Wan, Yingyan (Celine) Lin
GPT4AIGChip: Towards Next-Generation AI Accelerator Design Automation via Large Language Models
ICCAD 2023 [Paper] [Slide], Covered by [The Next Platform] [Anastasi In Tech]

---

Yonggan Fu, Ye Yuan, Souvik Kundu, Shang Wu, Shunyao Zhang, Yingyan (Celine) Lin
NeRFool: Uncovering the Vulnerability of Generalizable Neural Radiance Fields against Adversarial Perturbations
ICML 2023 [Paper] [Code] [Video] [Slide]
In collaboration with Intel

---

Yonggan Fu, Yuecheng Li, Chenghui Li, Jason Saragih, Peizhao Zhang, Xiaoliang Dai, Yingyan (Celine) Lin
Auto-CARD: Efficient and Robust Codec Avatar Driving for Real-time Mobile Telepresence
CVPR 2023 [Paper] [Video] [Slide]
In collaboration with Meta

---

Yonggan Fu*, Zhifan Ye*, Jiayi Yuan, Shunyao Zhang, Sixu Li, Haoran You, Yingyan (Celine) Lin
Gen-NeRF: Efficient and Generalizable Neural Radiance Fields via Algorithm-Hardware Co-Design
ISCA 2023 [Paper] [Video] [Slide]
2nd Place in the 33rd ACM SIGDA/IEEE CEDA University Demonstration at DAC 2023

---

Yonggan Fu, Yang Zhang, Kaizhi Qian, Zhifan Ye, Zhongzhi Yu, Cheng-I Lai, Yingyan (Celine) Lin
Losses Can Be Blessings: Routing Self-Supervised Speech Representations Towards Efficient Multilingual and Multitask Speech Processing
NeurIPS 2022 [Paper] [Code] [Video] [Slide]
In collaboration with MIT-IBM Watson AI Lab

---

Yonggan Fu, Haichuan Yang, Jiayi Yuan, Meng Li, Cheng Wan, Raghuraman Krishnamoorthi, Vikas Chandra, Yingyan (Celine) Lin
DepthShrinker: A New Compression Paradigm Towards Boosting Real-Hardware Efficiency of Compact Neural Networks
ICML 2022 [Paper] [Code] [Video], Covered by [The Next Platform]
In collaboration with Meta

---

Yonggan Fu*, Shunyao Zhang*, Shang Wu*, Cheng Wan, Yingyan (Celine) Lin
Patch-Fool: Are Vision Transformers Always Robust Against Adversarial Perturbations?
ICLR 2022 [Paper] [Code] [Video]

---

Yonggan Fu, Qixuan Yu, Yang Zhang, Shang Wu, Xu Ouyang, David Cox, Yingyan (Celine) Lin
Drawing Robust Scratch Tickets: Subnetworks with Inborn Robustness Are Found within Randomly Initialized Networks
NeurIPS 2021 [Paper] [Code] [Video]
In collaboration with MIT-IBM Watson AI Lab

---

Yonggan Fu, Yang Zhao, Qixuan Yu, Chaojian Li, Yingyan (Celine) Lin
2-in-1 Accelerator: Enabling Random Precision Switch for Winning Both Adversarial Robustness and Efficiency
MICRO 2021 [Paper] [Video]

---

Yonggan Fu, Yang Zhang, Yue Wang, Zhihan Lu, Vivek Boominathan, Ashok Veeraraghavan, Yingyan (Celine) Lin
SACoD: Sensor Algorithm Co-Design Towards Efficient CNN-powered Intelligent PhlatCam
ICCV 2021 [Paper] [Code] [Video]
In collaboration with MIT-IBM Watson AI Lab

---

Yonggan Fu, Qixuan Yu, Meng Li, Vikas Chandra, Yingyan (Celine) Lin
Double-Win Quant: Aggressively Winning Robustness of Quantized Deep Neural Networks via Random Precision Training and Inference
ICML 2021 [Paper] [Code] [Video]
In collaboration with Meta

---

Yonggan Fu, Yongan Zhang, Yang Zhang, David Cox, Yingyan (Celine) Lin
Auto-NBA: Efficient and Effective Search Over the Joint Space of Networks, Bitwidths, and Accelerators
ICML 2021 [Paper] [Code] [Video]
In collaboration with MIT-IBM Watson AI Lab

---

Yonggan Fu, Han Guo, Meng Li, Xin Yang, Yining Ding, Vikas Chandra, Yingyan (Celine) Lin
CPT: Efficient Deep Neural Network Training via Cyclic Precision
ICLR 2021 [Paper] [Code] [Video]
Spotlight paper (top 6%), in collaboration with Meta

---

Yonggan Fu, Haoran You, Yang Zhao, Yue Wang, Chaojian Li, Kailash Gopalakrishnan, Zhangyang Wang, Yingyan (Celine) Lin
FracTrain: Fractionally Squeezing Bit Savings Both Temporally and Spatially for Efficient DNN Training
NeurIPS 2020 [Paper] [Code] [Video]

---

Yonggan Fu, Wuyang Chen, Haotao Wang, Haoran Li, Yingyan (Celine) Lin, Zhangyang Wang
AutoGan-Distiller: Searching to Compress Generative Adversarial Networks
ICML 2020 [Paper] [Code] [Video]

---

Education

• Ph.D. Student advised by Dr. Yingyan (Celine) Lin, Jan. 2023 - May 2025
  School of Computer Science, Georgia Institute of Technology
• Ph.D. Student advised by Dr. Yingyan (Celine) Lin, Aug. 2019 - Dec. 2022
  Department of Electrical and Computer Engineering, Rice University
• Bachelor’s Degree in Applied Physics & Computer Science, Aug. 2015 - Jun. 2019
  School of the Gifted Young, University of Science and Technology of China

Internship Experience

• Research Intern, Learning and Perception Research Group, NVIDIA Research, May 2024 - Apr. 2025
  Project: Hybrid Language Model Architecture Design
  Mentor: Dr. Pavlo Molchanov and Dr. Xin Dong
• Research Intern, Reality Lab, Meta Inc., May 2022 - Dec. 2022
  Project: Efficient and Robust Codec Avatar Driving for Real-time Mobile Telepresence
  Mentor: Dr. Yuecheng Li
• Research Intern, On-device AI Group, Meta Inc., Sep. 2021 - Dec. 2021
  Project: DepthShrinker - A New Compression Paradigm Towards Boosting Real-Hardware Efficiency of Compact Neural Networks
  Mentor: Dr. Meng Li and Dr. Haichuan Yang
• Research Intern, On-device AI Group, Meta Inc., May 2021 - Aug. 2021
  Project: On-device Image Super Resolution via Dynamic Convolution
  Mentor: Dr. Meng Li and Dr. Haichuan Yang
• Research Assistant, USTC-AC Lab, University of Science and Technology of China, Sep. 2018 - May 2019
  Project: Facial Landmark Localization and Empathy Recognition
  Advisor: Dr. Shangfei Wang
• Research Assistant, University of California, Santa Barbara, Jun. 2018 - Sep. 2018
  Project: Compiler and Architecture Co-optimization for Quantum Computing
  Advisor: Dr. Yufei Ding

Selected Awards

• [Aug. 2023] IBM PhD Fellowship Award 2023
• [Jul. 2023] 2nd place of University Demo Best Demonstration at DAC 2023
• [May  2023] Machine Learning and Systems Rising Stars 2023
• [Dec. 2022] Rice Engineering Alumni Graduate Student Travel Grant
• [Oct. 2022] NeurIPS 2022 Scholar Award
• [Jul. 2022] 1st Place of University Demo Best Demonstration at DAC 2022
• [Jun. 2022] ICML 2022 Travel Grant
• [Dec. 2021] DAC Young Student Fellow
• [Jul. 2019] Outstanding Undergraduate Award of University of Science and Technology of China
• [Jan. 2019] 2nd Place in OMG-Empathy Prediction Challenge held by ICFG 2019
• [Oct. 2018] Gold Scholarship of University of Science and Technology of China
• [Oct. 2017] Gold Scholarship of University of Science and Technology of China

Community Services

• Session chair of DAC 2025
• Co-chair of 2023 DAC Ph.D. Forum
• CVPR 2022 Outstanding Reviewer
• Conference Review: Regularly serve as a reviewer for NeurIPS, ICLR, ICML, CVPR, ICCV, ECCV, and AAAI since 2020
• Journal Review: TODAES, JMLR, TNNLS, TCAD, TCAS-II, TCSVT, JSA

Invited Talks

• [Nov. 2025] Invited Talk at Efficient ML Lecture @ Georgia Tech
• [Aug. 2023] Invited Talk at Institute for Computer Science, Artificial Intelligence and Technology (INSAIT) @ Bulgaria
• [Jul. 2023] Invited Talk at AutoDL Tutorial @ DAC 2023
• [Aug. 2022] Invited Talk at HARMONY-RTML Project Workshop @ Rice University
• [Apr. 2022] Invited Talk at 5k Language Project Seminar @ MIT-IBM Watson AI Lab
• [Mar. 2022] Invited Talk at NICS Lab @ Tsinghua University
• [Feb. 2022] Invited Talk at Artificial Intelligence Time @ Tsinghua Universty
• [Oct. 2020] Invited Talk at Ken Kennedy Institute Data Science Conference 2020