An Empirical Analysis of Forgetting in Pre-trained Models with Incremental Low-Rank Updatesby Albin Soutif–Cormerais, Simone…
On Fairness of Low-Rank Adaptation of Large Modelsby Zhoujie Ding, Ken Ziyu Liu, Pura Peetathawatchai,…
Bridging The Gap between Low-rank and Orthogonal Adaptation via Householder Reflection Adaptationby Shen Yuan, Haotian…
Understanding Linear Probing then Fine-tuning Language Models from NTK Perspectiveby Akiyoshi Tomihari, Issei SatoFirst submitted…
Mixture of Latent Experts Using Tensor Productsby Zhan Su, Fengran Mo, Prayag Tiwari, Benyou Wang,…
LoRA-Ensemble: Efficient Uncertainty Modelling for Self-attention Networksby Michelle Halbheer, Dominik J. Mühlematter, Alexander Becker, Dominik…
MoRA: High-Rank Updating for Parameter-Efficient Fine-Tuningby Ting Jiang, Shaohan Huang, Shengyue Luo, Zihan Zhang, Haizhen…
Low-Rank Adaptation of Time Series Foundational Models for Out-of-Domain Modality Forecastingby Divij Gupta, Anubhav Bhatti,…
LoRA Learns Less and Forgets Lessby Dan Biderman, Jacob Portes, Jose Javier Gonzalez Ortiz, Mansheej…
Harmonizing Generalization and Personalization in Federated Prompt Learningby Tianyu Cui, Hongxia Li, Jingya Wang, Ye…