Summary of Low-rank Quantization-aware Training For Llms, by Yelysei Bondarenko et al.
Low-Rank Quantization-Aware Training for LLMsby Yelysei Bondarenko, Riccardo Del Chiaro, Markus NagelFirst submitted to arxiv…
Summary of Online Dpo: Online Direct Preference Optimization with Fast-slow Chasing, by Biqing Qi et al.
Online DPO: Online Direct Preference Optimization with Fast-Slow Chasingby Biqing Qi, Pengfei Li, Fangyuan Li,…
Summary of Federated Lora with Sparse Communication, by Kevin Kuo et al.
Federated LoRA with Sparse Communicationby Kevin Kuo, Arian Raje, Kousik Rajesh, Virginia SmithFirst submitted to…
Summary of Compressible Dynamics in Deep Overparameterized Low-rank Learning & Adaptation, by Can Yaras et al.
Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptationby Can Yaras, Peng Wang, Laura Balzano,…
Summary of Computational Limits Of Low-rank Adaptation (lora) For Transformer-based Models, by Jerry Yao-chieh Hu et al.
Computational Limits of Low-Rank Adaptation (LoRA) for Transformer-Based Modelsby Jerry Yao-Chieh Hu, Maojiang Su, En-Jui…
Summary of Choice Of Peft Technique in Continual Learning: Prompt Tuning Is Not All You Need, by Martin Wistuba and Prabhu Teja Sivaprasad and Lukas Balles and Giovanni Zappella
Choice of PEFT Technique in Continual Learning: Prompt Tuning is Not All You Needby Martin…
Summary of Quanta: Efficient High-rank Fine-tuning Of Llms with Quantum-informed Tensor Adaptation, by Zhuo Chen et al.
QuanTA: Efficient High-Rank Fine-Tuning of LLMs with Quantum-Informed Tensor Adaptationby Zhuo Chen, Rumen Dangovski, Charlotte…
Summary of Ether: Efficient Finetuning Of Large-scale Models with Hyperplane Reflections, by Massimo Bini et al.
ETHER: Efficient Finetuning of Large-Scale Models with Hyperplane Reflectionsby Massimo Bini, Karsten Roth, Zeynep Akata,…
Summary of Svft: Parameter-efficient Fine-tuning with Singular Vectors, by Vijay Lingam et al.
SVFT: Parameter-Efficient Fine-Tuning with Singular Vectorsby Vijay Lingam, Atula Tejaswi, Aditya Vavre, Aneesh Shetty, Gautham…
Summary of Owlore: Outlier-weighed Layerwise Sampled Low-rank Projection For Memory-efficient Llm Fine-tuning, by Pengxiang Li et al.
OwLore: Outlier-weighed Layerwise Sampled Low-Rank Projection for Memory-Efficient LLM Fine-tuningby Pengxiang Li, Lu Yin, Xiaowei…