Large Deviations and Improved Mean-squared Error Rates of Nonlinear SGD: Heavy-tailed Noise and Power of…
SDP4Bit: Toward 4-bit Communication Quantization in Sharded Data Parallelism for LLM Trainingby Jinda Jia, Cong…
QuAILoRA: Quantization-Aware Initialization for LoRAby Neal Lawton, Aishwarya Padmakumar, Judith Gaspers, Jack FitzGerald, Anoop Kumar,…
EvoPress: Towards Optimal Dynamic Model Compression via Evolutionary Searchby Oliver Sieberling, Denis Kuznedelev, Eldar Kurtic,…
Understanding the difficulty of low-precision post-training quantization of large language modelsby Zifei Xu, Sayeh Sharify,…
Nonlinear Stochastic Gradient Descent and Heavy-tailed Noise: A Unified Framework and High-probability Guaranteesby Aleksandar Armacki,…
DPLM-2: A Multimodal Diffusion Protein Language Modelby Xinyou Wang, Zaixiang Zheng, Fei Ye, Dongyu Xue,…
Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMsby Tianyu Guo, Druv Pai, Yu Bai,…
A Unified View of Delta Parameter Editing in Post-Trained Large-Scale Modelsby Qiaoyu Tang, Le Yu,…
Progressive Mixed-Precision Decoding for Efficient LLM Inferenceby Hao Mark Chen, Fuwen Tan, Alexandros Kouris, Royson…