Summary of Stacking As Accelerated Gradient Descent, by Naman Agarwal and Pranjal Awasthi and Satyen Kale and Eric Zhao
Stacking as Accelerated Gradient Descentby Naman Agarwal, Pranjal Awasthi, Satyen Kale, Eric ZhaoFirst submitted to…
Summary of Directional Smoothness and Gradient Methods: Convergence and Adaptivity, by Aaron Mishkin et al.
Directional Smoothness and Gradient Methods: Convergence and Adaptivityby Aaron Mishkin, Ahmed Khaled, Yuanhao Wang, Aaron…
Summary of On the Origins Of Linear Representations in Large Language Models, by Yibo Jiang et al.
On the Origins of Linear Representations in Large Language Modelsby Yibo Jiang, Goutham Rajendran, Pradeep…
Summary of Inverse-free Fast Natural Gradient Descent Method For Deep Learning, by Xinwei Ou et al.
Inverse-Free Fast Natural Gradient Descent Method for Deep Learningby Xinwei Ou, Ce Zhu, Xiaolin Huang,…
Summary of Level Set Teleportation: An Optimization Perspective, by Aaron Mishkin et al.
Level Set Teleportation: An Optimization Perspectiveby Aaron Mishkin, Alberto Bietti, Robert M. GowerFirst submitted to…
Summary of How Well Can Transformers Emulate In-context Newton’s Method?, by Angeliki Giannou et al.
How Well Can Transformers Emulate In-context Newton’s Method?by Angeliki Giannou, Liu Yang, Tianhao Wang, Dimitris…
Summary of Sofim: Stochastic Optimization Using Regularized Fisher Information Matrix, by Mrinmay Sen et al.
SOFIM: Stochastic Optimization Using Regularized Fisher Information Matrixby Mrinmay Sen, A. K. Qin, Gayathri C,…
Summary of Noise Misleads Rotation Invariant Algorithms on Sparse Targets, by Manfred K. Warmuth (1) et al.
Noise misleads rotation invariant algorithms on sparse targetsby Manfred K. Warmuth, Wojciech Kotłowski, Matt Jones,…
Summary of Enhancing Llm Safety Via Constrained Direct Preference Optimization, by Zixuan Liu et al.
Enhancing LLM Safety via Constrained Direct Preference Optimizationby Zixuan Liu, Xiaolin Sun, Zizhan ZhengFirst submitted…
Summary of From Zero to Hero: How Local Curvature at Artless Initial Conditions Leads Away From Bad Minima, by Tony Bonnaire et al.
From Zero to Hero: How local curvature at artless initial conditions leads away from bad…