Summary of Zeroth-order Fine-tuning Of Llms with Extreme Sparsity, by Wentao Guo et al.
Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsityby Wentao Guo, Jikai Long, Yimeng Zeng, Zirui Liu,…
Summary of Sltrain: a Sparse Plus Low-rank Approach For Parameter and Memory Efficient Pretraining, by Andi Han et al.
SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretrainingby Andi Han, Jiaxiang…
Summary of Magr: Weight Magnitude Reduction For Enhancing Post-training Quantization, by Aozhong Zhang et al.
MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantizationby Aozhong Zhang, Naigang Wang, Yanxia Deng, Xin…
Summary of Outliers and Calibration Sets Have Diminishing Effect on Quantization Of Modern Llms, by Davide Paglieri et al.
Outliers and Calibration Sets have Diminishing Effect on Quantization of Modern LLMsby Davide Paglieri, Saurabh…
Summary of Effective Interplay Between Sparsity and Quantization: From Theory to Practice, by Simla Burcu Harma et al.
Effective Interplay between Sparsity and Quantization: From Theory to Practiceby Simla Burcu Harma, Ayan Chakraborty,…
Summary of Lcq: Low-rank Codebook Based Quantization For Large Language Models, by Wen-pu Cai et al.
LCQ: Low-Rank Codebook based Quantization for Large Language Modelsby Wen-Pu Cai, Ming-Yang Li, Wu-Jun LiFirst…
Summary of Understanding and Minimising Outlier Features in Neural Network Training, by Bobby He et al.
Understanding and Minimising Outlier Features in Neural Network Trainingby Bobby He, Lorenzo Noci, Daniele Paliotta,…
Summary of Exploiting Llm Quantization, by Kazuki Egashira et al.
Exploiting LLM Quantizationby Kazuki Egashira, Mark Vero, Robin Staab, Jingxuan He, Martin VechevFirst submitted to…
Summary of 4-bit Shampoo For Memory-efficient Network Training, by Sike Wang et al.
4-bit Shampoo for Memory-Efficient Network Trainingby Sike Wang, Pan Zhou, Jia Li, Hua HuangFirst submitted…
Summary of I-llm: Efficient Integer-only Inference For Fully-quantized Low-bit Large Language Models, by Xing Hu et al.
I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Modelsby Xing Hu, Yuan Cheng, Dawei…