Non-convergence of Adam and other adaptive stochastic gradient descent optimization methods for non-vanishing learning ratesby…
Stochastic Gradient Descent for Two-layer Neural Networksby Dinghao Cao, Zheng-Chu Guo, Lei ShiFirst submitted to…
On the Complexity of Learning Sparse Functions with Statistical and Gradient Queriesby Nirmit Joshi, Theodor…
Privacy of the last iterate in cyclically-sampled DP-SGD on nonconvex composite lossesby Weiwei Kong, Mónica…
Approximating Two-Layer ReLU Networks for Hidden State Analysis in Differential Privacyby Antti KoskelaFirst submitted to…
Langevin Dynamics: A Unified Perspective on Optimization via Lyapunov Potentialsby August Y. Chen, Ayush Sekhari,…
Bias of Stochastic Gradient Descent or the Architecture: Disentangling the Effects of Overparameterization of Neural…
Revisiting Nearest Neighbor for Tabular Data: A Deep Tabular Baseline Two Decades Laterby Han-Jia Ye,…
Accelerating Distributed Optimization: A Primal-Dual Perspective on Local Stepsby Junchi Yang, Murat Yildirim, Qiu FengFirst…
Stochastic Differential Equations models for Least-Squares Stochastic Gradient Descentby Adrien Schertzer, Loucas Pillaud-VivienFirst submitted to…