Randomized Asymmetric Chain of LoRA: The First Meaningful Theoretical Framework for Low-Rank Adaptationby Grigory Malinovsky,…
SLIM: Let LLM Learn More and Forget Less with Soft LoRA and Identity Mixtureby Jiayi…
One Initialization to Rule them All: Fine-tuning via Explained Variance Adaptationby Fabian Paischer, Lukas Hauzenberger,…
Learning on LoRAs: GL-Equivariant Processing of Low-Rank Weight Spaces for Large Finetuned Modelsby Theo Putterman,…
NEAT: Nonlinear Parameter-efficient Adaptation of Pre-trained Modelsby Yibo Zhong, Haoxiang Jiang, Lincan Li, Ryumei Nakada,…
Fira: Can We Achieve Full-rank Training of LLMs Under Low-rank Constraint?by Xi Chen, Kaituo Feng,…
Selective Aggregation for Low-Rank Adaptation in Federated Learningby Pengxin Guo, Shuang Zeng, Yanran Wang, Huijie…
DLP-LoRA: Efficient Task-Specific LoRA Fusion with a Dynamic, Lightweight Plugin for Large Language Modelsby Yuxuan…
MoS: Unleashing Parameter Efficiency of Low-Rank Adaptation with Mixture of Shardsby Sheng Wang, Liheng Chen,…
Fisher Information-based Efficient Curriculum Federated Learning with Large Language Modelsby Ji Liu, Jiaxiang Ren, Ruoming…