Summary of Curriculum Learning with Quality-driven Data Selection, by Biao Wu et al.
Curriculum Learning with Quality-Driven Data Selectionby Biao Wu, Fang Meng, Ling ChenFirst submitted to arxiv…
Summary of Training-free Exponential Context Extension Via Cascading Kv Cache, by Jeffrey Willette et al.
Training-Free Exponential Context Extension via Cascading KV Cacheby Jeffrey Willette, Heejun Lee, Youngwan Lee, Myeongjae…
Summary of Bypass Back-propagation: Optimization-based Structural Pruning For Large Language Models Via Policy Gradient, by Yuan Gao et al.
Bypass Back-propagation: Optimization-based Structural Pruning for Large Language Models via Policy Gradientby Yuan Gao, Zujing…
Summary of Markov Constraint As Large Language Model Surrogate, by Alexandre Bonlarron et al.
Markov Constraint as Large Language Model Surrogateby Alexandre Bonlarron, Jean-Charles RéginFirst submitted to arxiv on:…
Summary of Alps: Improved Optimization For Highly Sparse One-shot Pruning For Large Language Models, by Xiang Meng et al.
ALPS: Improved Optimization for Highly Sparse One-Shot Pruning for Large Language Modelsby Xiang Meng, Kayhan…
Summary of When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models, by Haoran You et al.
When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Modelsby…
Summary of Switchlora: Switched Low-rank Adaptation Can Learn Full-rank Information, by Kaiye Zhou et al.
SwitchLoRA: Switched Low-Rank Adaptation Can Learn Full-Rank Informationby Kaiye Zhou, Shucheng Wang, Jun XuFirst submitted…
Summary of Parallelizing Linear Transformers with the Delta Rule Over Sequence Length, by Songlin Yang et al.
Parallelizing Linear Transformers with the Delta Rule over Sequence Lengthby Songlin Yang, Bailin Wang, Yu…
Summary of Shiftaddllm: Accelerating Pretrained Llms Via Post-training Multiplication-less Reparameterization, by Haoran You et al.
ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterizationby Haoran You, Yipin Guo, Yichao Fu, Wei…
Summary of Block Transformer: Global-to-local Language Modeling For Fast Inference, by Namgyu Ho et al.
Block Transformer: Global-to-Local Language Modeling for Fast Inferenceby Namgyu Ho, Sangmin Bae, Taehyeon Kim, Hyunjik…