Summary of Copra: a Progressive Lora Training Strategy, by Zhan Zhuang et al.
CopRA: A Progressive LoRA Training Strategyby Zhan Zhuang, Xiequn Wang, Yulong Zhang, Wei Li, Yu…
Summary of Lora Vs Full Fine-tuning: An Illusion Of Equivalence, by Reece Shuttleworth et al.
LoRA vs Full Fine-tuning: An Illusion of Equivalenceby Reece Shuttleworth, Jacob Andreas, Antonio Torralba, Pratyusha…
Summary of Kd-lora: a Hybrid Approach to Efficient Fine-tuning with Lora and Knowledge Distillation, by Rambod Azimi et al.
KD-LoRA: A Hybrid Approach to Efficient Fine-Tuning with LoRA and Knowledge Distillationby Rambod Azimi, Rishav…
Summary of Closed-form Merging Of Parameter-efficient Modules For Federated Continual Learning, by Riccardo Salami et al.
Closed-form merging of parameter-efficient modules for Federated Continual Learningby Riccardo Salami, Pietro Buzzega, Matteo Mosconi,…
Summary of Mira: a Method Of Federated Multi-task Learning For Large Language Models, by Ahmed Elbakary et al.
MIRA: A Method of Federated MultI-Task Learning for LaRge LAnguage Modelsby Ahmed Elbakary, Chaouki Ben…
Summary of Mor: Mixture Of Ranks For Low-rank Adaptation Tuning, by Chuanyu Tang et al.
MoR: Mixture of Ranks for Low-Rank Adaptation Tuningby Chuanyu Tang, Yilong Chen, Zhenyu Zhang, Junyuan…
Summary of Lora Soups: Merging Loras For Practical Skill Composition Tasks, by Akshara Prabhakar et al.
LoRA Soups: Merging LoRAs for Practical Skill Composition Tasksby Akshara Prabhakar, Yuanzhi Li, Karthik Narasimhan,…
Summary of Loko: Low-rank Kalman Optimizer For Online Fine-tuning Of Large Models, by Hossein Abdi et al.
LoKO: Low-Rank Kalman Optimizer for Online Fine-Tuning of Large Modelsby Hossein Abdi, Mingfei Sun, Andi…
Summary of Lolcats: on Low-rank Linearizing Of Large Language Models, by Michael Zhang et al.
LoLCATs: On Low-Rank Linearizing of Large Language Modelsby Michael Zhang, Simran Arora, Rahul Chalamala, Alan…
Summary of Fed-pilot: Optimizing Lora Assignment For Efficient Federated Foundation Model Fine-tuning, by Zikai Zhang et al.
Fed-piLot: Optimizing LoRA Assignment for Efficient Federated Foundation Model Fine-Tuningby Zikai Zhang, Jiahao Xu, Ping…