Summary of Spinquant: Llm Quantization with Learned Rotations, by Zechun Liu et al.
SpinQuant: LLM quantization with learned rotationsby Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv…
Summary of Mindstar: Enhancing Math Reasoning in Pre-trained Llms at Inference Time, by Jikun Kang et al.
MindStar: Enhancing Math Reasoning in Pre-trained LLMs at Inference Timeby Jikun Kang, Xin Zhe Li,…
Summary of Spp: Sparsity-preserved Parameter-efficient Fine-tuning For Large Language Models, by Xudong Lu et al.
SPP: Sparsity-Preserved Parameter-Efficient Fine-Tuning for Large Language Modelsby Xudong Lu, Aojun Zhou, Yuhui Xu, Renrui…
Summary of Basis Selection: Low-rank Decomposition Of Pretrained Large Language Models For Target Applications, by Yang Li et al.
Basis Selection: Low-Rank Decomposition of Pretrained Large Language Models for Target Applicationsby Yang Li, Changsheng…
Summary of Microadam: Accurate Adaptive Optimization with Low Space Overhead and Provable Convergence, by Ionut-vlad Modoranu et al.
MicroAdam: Accurate Adaptive Optimization with Low Space Overhead and Provable Convergenceby Ionut-Vlad Modoranu, Mher Safaryan,…
Summary of Bisup: Bidirectional Quantization Error Suppression For Large Language Models, by Minghui Zou et al.
BiSup: Bidirectional Quantization Error Suppression for Large Language Modelsby Minghui Zou, Ronghui Guo, Sai Zhang,…
Summary of Pv-tuning: Beyond Straight-through Estimation For Extreme Llm Compression, by Vladimir Malinovskii et al.
PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compressionby Vladimir Malinovskii, Denis Mazur, Ivan Ilin, Denis…
Summary of Not All Language Model Features Are One-dimensionally Linear, by Joshua Engels et al.
Not All Language Model Features Are One-Dimensionally Linearby Joshua Engels, Eric J. Michaud, Isaac Liao,…
Summary of Simpo: Simple Preference Optimization with a Reference-free Reward, by Yu Meng et al.
SimPO: Simple Preference Optimization with a Reference-Free Rewardby Yu Meng, Mengzhou Xia, Danqi ChenFirst submitted…
Summary of Evaluating Large Language Models For Public Health Classification and Extraction Tasks, by Joshua Harris et al.
Evaluating Large Language Models for Public Health Classification and Extraction Tasksby Joshua Harris, Timothy Laurence,…