Summary of Kd-lora: a Hybrid Approach to Efficient Fine-tuning with Lora and Knowledge Distillation, by Rambod Azimi et al.
KD-LoRA: A Hybrid Approach to Efficient Fine-Tuning with LoRA and Knowledge Distillationby Rambod Azimi, Rishav…
Summary of Lora Done Rite: Robust Invariant Transformation Equilibration For Lora Optimization, by Jui-nan Yen et al.
LoRA Done RITE: Robust Invariant Transformation Equilibration for LoRA Optimizationby Jui-Nan Yen, Si Si, Zhao…
Summary of Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise Lora, by Sangmin Bae et al.
Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRAby Sangmin Bae, Adam Fisch, Hrayr Harutyunyan,…
Summary of Less Is More: Extreme Gradient Boost Rank-1 Adaption For Efficient Finetuning Of Llms, by Yifei Zhang et al.
Less is More: Extreme Gradient Boost Rank-1 Adaption for Efficient Finetuning of LLMsby Yifei Zhang,…
Summary of On the Crucial Role Of Initialization For Matrix Factorization, by Bingcong Li et al.
On the Crucial Role of Initialization for Matrix Factorizationby Bingcong Li, Liang Zhang, Aryan Mokhtari,…
Summary of Advancing Super-resolution in Neural Radiance Fields Via Variational Diffusion Strategies, by Shrey Vishen et al.
Advancing Super-Resolution in Neural Radiance Fields via Variational Diffusion Strategiesby Shrey Vishen, Jatin Sarabu, Saurav…
Summary of Closed-form Merging Of Parameter-efficient Modules For Federated Continual Learning, by Riccardo Salami et al.
Closed-form merging of parameter-efficient modules for Federated Continual Learningby Riccardo Salami, Pietro Buzzega, Matteo Mosconi,…
Summary of More: Multi-modal Contrastive Pre-training with Transformers on X-rays, Ecgs, and Diagnostic Report, by Samrajya Thapa et al.
MoRE: Multi-Modal Contrastive Pre-training with Transformers on X-Rays, ECGs, and Diagnostic Reportby Samrajya Thapa, Koushik…
Summary of Natural Galore: Accelerating Galore For Memory-efficient Llm Training and Fine-tuning, by Arijit Das
Natural GaLore: Accelerating GaLore for memory-efficient LLM Training and Fine-tuningby Arijit DasFirst submitted to arxiv…
Summary of Beyond 2:4: Exploring V:n:m Sparsity For Efficient Transformer Inference on Gpus, by Kang Zhao et al.
Beyond 2:4: exploring V:N:M sparsity for efficient transformer inference on GPUsby Kang Zhao, Tao Yuan,…