RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantizationby Xijie Huang, Zechun Liu, Shih-Yang Liu,…
SBoRA: Low-Rank Adaptation with Regional Weight Updatesby Lai-Man Po, Yuyang Liu, Haoxuan Wu, Tianqi Zhang,…
LoRA-GA: Low-Rank Adaptation with Gradient Approximationby Shaowen Wang, Linxi Yu, Jian LiFirst submitted to arxiv…
Retrieval-Augmented Mixture of LoRA Experts for Uploadable Machine Learningby Ziyu Zhao, Leilei Gan, Guoyin Wang,…
FLoCoRA: Federated learning compression with low-rank adaptationby Lucas Grativol Ribeiro, Mathieu Leonardon, Guillaume Muller, Virginie…
Sparse High Rank Adaptersby Kartikeya Bhardwaj, Nilesh Prasad Pandey, Sweta Priyadarshi, Viswanath Ganapathy, Shreya Kadambi,…
VIRL: Volume-Informed Representation Learning towards Few-shot Manufacturability Estimationby Yu-hsuan Chen, Jonathan Cagan, Levent Burak karaFirst…
Mixture-of-Subspaces in Low-Rank Adaptationby Taiqiang Wu, Jiahao Wang, Zhe Zhao, Ngai WongFirst submitted to arxiv…
BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Modelsby Yibin Wang, Haizhou Shi, Ligong…
The Impact of Initialization on LoRA Finetuning Dynamicsby Soufiane Hayou, Nikhil Ghosh, Bin YuFirst submitted…