Hi, I am currently an AI researcher at Huawei Noah’s Ark Lab. I got my Ph.D. degree at UCLA Math Department, advised by Professor Guido Montufar. My research mainly focuses on Large Language Models, Mechanistic Interpretability and Deep Learning Theory. Here is my resume.
Education
- Ph.D., Department of Mathematics, University of California, Los Angeles, Dec 2022
- B.S., School of Mathematical Sciences, Peking University, Jul 2017
Publication
Towards understanding how transformer perform multi-step reasoning with matching operation
Zhiwei Wang, Yunji Wang, Zhongwang Zhang, Zhangchen Zhou, Hui Jin, Tianyang Hu, Jiacheng Sun, Zhenguo Li, Yaoyu Zhang, Zhi-Qin John Xu, 2025.
Submitted to The Forty-second International Conference on Machine Learning (ICML 2025).
We propose a buffer mechanism and found evidence that supports such mechanism being employed by language models during the reasoning process. We propose a method to enhance the model’s reasoning capability, significantly improving data utilization efficiency in logical reasoning datasets.
Download [pdf].
Exact Conversion of In-Context Learning to Model Weights in Linearized-Attention Transformers
Brian K Chen, Tianyang Hu, Hui Jin, Hwee Kuan Lee, Kenji Kawaguchi, 2025.
Published in The Forty-First International Conference on Machine Learning (ICML 2024).
We find a way to convert the prompts into the model weights by introducing an extra bias term into the attention module.
Download [pdf].
How Do LLMs Acquire New Knowledge? A Knowledge Circuits Perspective on Continual Pre-Training
Yixin Ou, Yunzhi Yao, Ningyu Zhang, Hui Jin, Jiacheng Sun, Shumin Deng, Zhenguo Li, Huajun Chen, 2025.
Submitted to The 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025) .
We analyze how LLMs learn new knowledge through the lens of knowledge circuit evolution, identifying computational subgraphs that facilitate knowledge storage and processing.
Download [pdf].
Characterizing the Spectrum of the NTK via a Power Series Expansion
Michael Murray, Hui Jin, Benjamin Bowman, Guido Montufar, 2022.
Published in The Eleventh International Conference on Learning Representations (ICLR 2023).
Download [pdf].
Learning curves for Gaussian process regression with power-law priors and targets
Hui Jin, Pradeep Kr Banerjee, Guido Montúfar, 2021.
Published in The Tenth International Conference on Learning Representations (ICLR 2022).
Workshop version presented at Workshop on Bayesian Deep Learning NeurIPS, 2021.
Implicit bias of gradient descent for mean squared error regression with wide neural networks
Hui Jin, Guido Montúfar, 2020.
Journal of Machine Learning Research JMLR 24(137):1-97, 2023. Repo GitHub.
Noisy Subgraph Isomorphisms on Multiplex Networks
Hui Jin, Xie He, Yanghui Wang, Hao Li, Andrea L Bertozzi, 2019.
Published in 2019 IEEE International Conference on Big Data (Big Data).
Download [pdf].
Teaching
2021 Fall: PIC 16A Python with Applications I
Undergraduate course, UCLA, 2021
Core Python language constructs, applications, text processing, data visualization, interaction with spreadsheets and machine learning.
2020 Fall: PIC 10B Intermediate Programming
Undergraduate course, UCLA, 2020
Abstract data types and their implementation using C++ class mechanism; dynamic data structures, including linked lists, stacks, queues, trees, and hash tables; applications; object-oriented programming and software reuse; recursion; algorithms for sorting and searching.