QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuningby Hossein Rajabzadeh, Mojtaba Valipour,…
Recovering the Pre-Fine-Tuning Weights of Generative Modelsby Eliahu Horwitz, Jonathan Kahana, Yedid HoshenFirst submitted to…
SusFL: Energy-Aware Federated Learning-based Monitoring for Sustainable Smart Farmsby Dian Chen, Paul Yang, Ing-Ray Chen,…
LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the Wildby Ziyu Zhao, Leilei…
Can LLMs Learn New Concepts Incrementally without Forgetting?by Junhao Zheng, Shengjie Qiu, Qianli MaFirst submitted…
LoRA-drop: Efficient LoRA Parameter Pruning based on Output Evaluationby Hongyun Zhou, Xiangyu Lu, Wang Xu,…
Accurate LoRA-Finetuning Quantization of LLMs via Information Retentionby Haotong Qin, Xudong Ma, Xingyu Zheng, Xiaoyang…
ApiQ: Finetuning of 2-Bit Quantized Large Language Modelby Baohao Liao, Christian Herold, Shahram Khadivi, Christof…
L4Q: Parameter Efficient Quantization-Aware Fine-Tuning on Large Language Modelsby Hyesung Jeon, Yulhwa Kim, Jae-joon KimFirst…
Flora: Low-Rank Adapters Are Secretly Gradient Compressorsby Yongchang Hao, Yanshuai Cao, Lili MouFirst submitted to…