Distributed Stochastic Gradient Descent with Staleness: A Stochastic Delay Differential Equation Based Frameworkby Siyuan Yu,…
Why Warmup the Learning Rate? Underlying Mechanisms and Improvementsby Dayal Singh Kalra, Maissam BarkeshliFirst submitted…
Contextual Dynamic Pricing: Algorithms, Optimality, and Local Differential Privacy Constraintsby Zifeng Zhao, Feiyu Jiang, Yi…
Online Learning and Information Exponents: On The Importance of Batch size, and Time/Complexity Tradeoffsby Luca…
What Improves the Generalization of Graph Transformers? A Theoretical Dive into the Self-attention and Positional…
Understanding Stochastic Natural Gradient Variational Inferenceby Kaiwen Wu, Jacob R. GardnerFirst submitted to arxiv on:…
Demystifying SGD with Doubly Stochastic Gradientsby Kyurae Kim, Joohwan Ko, Yi-An Ma, Jacob R. GardnerFirst…
Efficient Sign-Based Optimization: Accelerating Convergence via Variance Reductionby Wei Jiang, Sifan Yang, Wenhao Yang, Lijun…
Stochastic Resetting Mitigates Latent Gradient Bias of SGD from Label Noiseby Youngkyoung Bae, Yeongwoo Song,…
Sharpness-Aware Minimization Enhances Feature Quality via Balanced Learningby Jacob Mitchell Springer, Vaishnavh Nagarajan, Aditi RaghunathanFirst…