Error bounds for particle gradient descent, and extensions of the log-Sobolev and Talagrand inequalitiesby Rocco…
A Theoretical Analysis of Self-Supervised Learning for Vision Transformersby Yu Huang, Zixin Wen, Yuejie Chi,…
Neural Redshift: Random Networks are not Random Functionsby Damien Teney, Armand Nicolicioiu, Valentin Hartmann, Ehsan…
Stochastic gradient descent for streaming linear and rectified linear systems with adversarial corruptionsby Halyun Jeong,…
Parallel Momentum Methods Under Biased Gradient Estimationsby Ali Beikmohammadi, Sarit Khirirat, Sindri MagnússonFirst submitted to…
Shifted Interpolation for Differential Privacyby Jinho Bok, Weijie Su, Jason M. AltschulerFirst submitted to arxiv…
Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Modelsby Frederik Kunstner, Robin…
Implicit Optimization Bias of Next-Token Prediction in Linear Modelsby Christos ThrampoulidisFirst submitted to arxiv on:…
Large Language Models As Evolution Strategiesby Robert Tjarko Lange, Yingtao Tian, Yujin TangFirst submitted to…
Large Stepsize Gradient Descent for Logistic Loss: Non-Monotonicity of the Loss Improves Optimization Efficiencyby Jingfeng…