Projects

Paper
Muon learns balanced solutions in matrix factorization without slow saddle-to-saddle dynamics arxiv.org/pdf/2606.30509 →
Mark Rhee*, Jamie Simon, Dhruva Karkada — Summer 2026

We study how the Muon optimizer's parameter trajectories differ from gradient descent in matrix factorization. Muon avoids slow saddle-to-saddle dynamics, remains stable beyond the critical learning rate, and conserves a distinct matrix quantity that drives weights toward balanced solutions. Completed during a summer research internship funded by Imbue.

Teaching
Learning Mechanics DeCal Course website →
Fall 2026

I'm designing and teaching a technical course that gives an introduction to learning mechanics. The course is built around the perspective paper There Will Be a Scientific Theory of Deep Learning (Simon et al., 2026). I wrote detailed, pedagogically-minded lecture notes introducing neural networks, deep linear networks, lazy learning and kernel regression, the lazy/rich dichotomy, and various approaches to studying feature learning.

Blog
Deep linear networks are a surprisingly useful toy model of weight-space dynamics learningmechanics.pub →
Spring 2026

What/why/how do deep neural networks learn? I co-authored this blog post for learningmechanics.pub to explain how a toy model (deep linear networks) can elucidate and make concrete intuition about deep learning.