Abstract: While score-based diffusion models have achieved remarkable success in high-dimensional generative modeling, some basic theoretical questions have not been precisely resolved. In this talk, we address minimax optimality of density estimation, functional estimation, and hypothesis testing. First, we show diffusion models achieve the optimal density estimation rate over Holder balls. This result is a consequence of our sharp characterization of minimax score estimation across all noising levels. A key contribution is our lower bound argument which involves a slight…

Abstract: Moravec’s Paradox observes that AI systems have struggled far more with learning physical action than symbolic reasoning. Yet just recently, there has been a tremendous increase in the capability of AI-driven robotic systems, reminiscent  of the early acceleration in language modeling capabilities a few years prior.  Using the lens of control-theoretic stability, this talk will demonstrate an exponential separation between natural regimes for learning in the physical world and in discrete/symbolic settings, thereby providing a mathematical basis for Moravec’s…

Abstract: Spectral gradient methods, such as the recently proposed Muon optimizer, are a promising alternative to standard gradient descent for training deep neural networks and transformers. Yet, it remains unclear in which regimes these spectral methods are expected to perform better. In this talk, I will present a simple condition that predicts when a spectral update yields a larger decrease in the loss than a standard gradient step. Informally, this criterion holds when, on the one hand, the gradient of the…

TBD

TBD

TBD