Summary of Adapters Strike Back, by Jan-martin O. Steitz and Stefan Roth
Adapters Strike Backby Jan-Martin O. Steitz, Stefan RothFirst submitted to arxiv on: 10 Jun 2024CategoriesMain:…
Summary of Low-rank Quantization-aware Training For Llms, by Yelysei Bondarenko et al.
Low-Rank Quantization-Aware Training for LLMsby Yelysei Bondarenko, Riccardo Del Chiaro, Markus NagelFirst submitted to arxiv…
Summary of Federated Lora with Sparse Communication, by Kevin Kuo et al.
Federated LoRA with Sparse Communicationby Kevin Kuo, Arian Raje, Kousik Rajesh, Virginia SmithFirst submitted to…
Summary of Compressible Dynamics in Deep Overparameterized Low-rank Learning & Adaptation, by Can Yaras et al.
Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptationby Can Yaras, Peng Wang, Laura Balzano,…
Summary of Repurposing Language Models Into Embedding Models: Finding the Compute-optimal Recipe, by Alicja Ziarko et al.
Repurposing Language Models into Embedding Models: Finding the Compute-Optimal Recipeby Alicja Ziarko, Albert Q. Jiang,…
Summary of Computational Limits Of Low-rank Adaptation (lora) For Transformer-based Models, by Jerry Yao-chieh Hu et al.
Computational Limits of Low-Rank Adaptation (LoRA) for Transformer-Based Modelsby Jerry Yao-Chieh Hu, Maojiang Su, En-Jui…
Summary of A Study Of Optimizations For Fine-tuning Large Language Models, by Arjun Singh et al.
A Study of Optimizations for Fine-tuning Large Language Modelsby Arjun Singh, Nikhil Pandey, Anup Shirgaonkar,…
Summary of Quanta: Efficient High-rank Fine-tuning Of Llms with Quantum-informed Tensor Adaptation, by Zhuo Chen et al.
QuanTA: Efficient High-Rank Fine-Tuning of LLMs with Quantum-Informed Tensor Adaptationby Zhuo Chen, Rumen Dangovski, Charlotte…
Summary of Mamba State-space Models Are Lyapunov-stable Learners, by John T. Halloran et al.
Mamba State-Space Models Are Lyapunov-Stable Learnersby John T. Halloran, Manbir Gulati, Paul F. RoysdonFirst submitted…
Summary of Owlore: Outlier-weighed Layerwise Sampled Low-rank Projection For Memory-efficient Llm Fine-tuning, by Pengxiang Li et al.
OwLore: Outlier-weighed Layerwise Sampled Low-Rank Projection for Memory-Efficient LLM Fine-tuningby Pengxiang Li, Lu Yin, Xiaowei…