Summary of Efficient Adaptive Optimization Via Subset-norm and Subspace-momentum: Fast, Memory-reduced Training with Convergence Guarantees, by Thien Hang Nguyen et al.
Efficient Adaptive Optimization via Subset-Norm and Subspace-Momentum: Fast, Memory-Reduced Training with Convergence Guaranteesby Thien Hang…