Summary of Mira: a Method Of Federated Multi-task Learning For Large Language Models, by Ahmed Elbakary et al.
MIRA: A Method of Federated MultI-Task Learning for LaRge LAnguage Modelsby Ahmed Elbakary, Chaouki Ben…
Summary of Collaboratively Adding New Knowledge to An Llm, by Rhui Dih Lee and Laura Wynter
Collaboratively adding new knowledge to an LLMby Rhui Dih Lee, Laura WynterFirst submitted to arxiv…
Summary of Implicit Regularization Of Sharpness-aware Minimization For Scale-invariant Problems, by Bingcong Li et al.
Implicit Regularization of Sharpness-Aware Minimization for Scale-Invariant Problemsby Bingcong Li, Liang Zhang, Niao HeFirst submitted…
Summary of Quailora: Quantization-aware Initialization For Lora, by Neal Lawton et al.
QuAILoRA: Quantization-Aware Initialization for LoRAby Neal Lawton, Aishwarya Padmakumar, Judith Gaspers, Jack FitzGerald, Anoop Kumar,…
Summary of Fitv2: Scalable and Improved Flexible Vision Transformer For Diffusion Model, by Zidong Wang and Zeyu Lu and Di Huang and Cai Zhou and Wanli Ouyang and and Lei Bai
FiTv2: Scalable and Improved Flexible Vision Transformer for Diffusion Modelby ZiDong Wang, Zeyu Lu, Di…
Summary of Mor: Mixture Of Ranks For Low-rank Adaptation Tuning, by Chuanyu Tang et al.
MoR: Mixture of Ranks for Low-Rank Adaptation Tuningby Chuanyu Tang, Yilong Chen, Zhenyu Zhang, Junyuan…
Summary of Lora Soups: Merging Loras For Practical Skill Composition Tasks, by Akshara Prabhakar et al.
LoRA Soups: Merging LoRAs for Practical Skill Composition Tasksby Akshara Prabhakar, Yuanzhi Li, Karthik Narasimhan,…
Summary of In-context Kv-cache Eviction For Llms Via Attention-gate, by Zihao Zeng et al.
In-context KV-Cache Eviction for LLMs via Attention-Gateby Zihao Zeng, Bokai Lin, Tianqi Hou, Hao Zhang,…
Summary of Loko: Low-rank Kalman Optimizer For Online Fine-tuning Of Large Models, by Hossein Abdi et al.
LoKO: Low-Rank Kalman Optimizer for Online Fine-Tuning of Large Modelsby Hossein Abdi, Mingfei Sun, Andi…
Summary of At-moe: Adaptive Task-planning Mixture Of Experts Via Lora Approach, by Xurui Li et al.
AT-MoE: Adaptive Task-planning Mixture of Experts via LoRA Approachby Xurui Li, Juanjuan YaoFirst submitted to…