Summary of Natural Galore: Accelerating Galore For Memory-efficient Llm Training and Fine-tuning, by Arijit Das
Natural GaLore: Accelerating GaLore for memory-efficient LLM Training and Fine-tuningby Arijit DasFirst submitted to arxiv…
Summary of Treebon: Enhancing Inference-time Alignment with Speculative Tree-search and Best-of-n Sampling, by Jiahao Qiu et al.
TreeBoN: Enhancing Inference-Time Alignment with Speculative Tree-Search and Best-of-N Samplingby Jiahao Qiu, Yifu Lu, Yifan…
Summary of Ldadam: Adaptive Optimization From Low-dimensional Gradient Statistics, by Thomas Robert and Mher Safaryan and Ionut-vlad Modoranu and Dan Alistarh
LDAdam: Adaptive Optimization from Low-Dimensional Gradient Statisticsby Thomas Robert, Mher Safaryan, Ionut-Vlad Modoranu, Dan AlistarhFirst…
Summary of Towards Optimal Adapter Placement For Efficient Transfer Learning, by Aleksandra I. Nowak et al.
Towards Optimal Adapter Placement for Efficient Transfer Learningby Aleksandra I. Nowak, Otniel-Bogdan Mercea, Anurag Arnab,…
Summary of Understanding and Alleviating Memory Consumption in Rlhf For Llms, by Jin Zhou et al.
Understanding and Alleviating Memory Consumption in RLHF for LLMsby Jin Zhou, Hanmei Yang, Steven, Tang,…
Summary of Rac: Efficient Llm Factuality Correction with Retrieval Augmentation, by Changmao Li and Jeffrey Flanigan
RAC: Efficient LLM Factuality Correction with Retrieval Augmentationby Changmao Li, Jeffrey FlaniganFirst submitted to arxiv…
Summary of Stacking Small Language Models For Generalizability, by Laurence Liang
Stacking Small Language Models for Generalizabilityby Laurence LiangFirst submitted to arxiv on: 21 Oct 2024CategoriesMain:…
Summary of Mitigating Forgetting in Llm Supervised Fine-tuning and Preference Learning, by Heshan Fernando et al.
Mitigating Forgetting in LLM Supervised Fine-Tuning and Preference Learningby Heshan Fernando, Han Shen, Parikshit Ram,…
Summary of Mira: a Method Of Federated Multi-task Learning For Large Language Models, by Ahmed Elbakary et al.
MIRA: A Method of Federated MultI-Task Learning for LaRge LAnguage Modelsby Ahmed Elbakary, Chaouki Ben…
Summary of Compact: Compressed Activations For Memory-efficient Llm Training, by Yara Shamshoum et al.
CompAct: Compressed Activations for Memory-Efficient LLM Trainingby Yara Shamshoum, Nitzan Hodos, Yuval Sieradzki, Assaf SchusterFirst…