RepQuant: Towards Accurate Post-Training Quantization of Large Transformer Models via Scale Reparameterizationby Zhikai Li, Xuewen…
Accurate LoRA-Finetuning Quantization of LLMs via Information Retentionby Haotong Qin, Xudong Ma, Xingyu Zheng, Xiaoyang…
ApiQ: Finetuning of 2-Bit Quantized Large Language Modelby Baohao Liao, Christian Herold, Shahram Khadivi, Christof…
L4Q: Parameter Efficient Quantization-Aware Fine-Tuning on Large Language Modelsby Hyesung Jeon, Yulhwa Kim, Jae-joon KimFirst…
QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooksby Albert Tseng, Jerry Chee,…
BiLLM: Pushing the Limit of Post-Training Quantization for LLMsby Wei Huang, Yangdong Liu, Haotong Qin,…
Optimal and Near-Optimal Adaptive Vector Quantizationby Ran Ben-Basat, Yaniv Ben-Itzhak, Michael Mitzenmacher, Shay VargaftikFirst submitted…
KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cacheby Zirui Liu, Jiayi Yuan, Hongye Jin,…
Stability Analysis of Various Symbolic Rule Extraction Methods from Recurrent Neural Networkby Neisarg Dave, Daniel…
Leveraging Continuously Differentiable Activation Functions for Learning in Quantized Noisy Environmentsby Vivswan Shah, Nathan YoungbloodFirst…