Summary of Mini-sequence Transformer: Optimizing Intermediate Memory For Long Sequences Training, by Cheng Luo et al.
Mini-Sequence Transformer: Optimizing Intermediate Memory for Long Sequences Trainingby Cheng Luo, Jiawei Zhao, Zhuoming Chen,…
Summary of When Can Transformers Count to N?, by Gilad Yehudai et al.
When Can Transformers Count to n?by Gilad Yehudai, Haim Kaplan, Asma Ghandeharioun, Mor Geva, Amir…
Summary of Characterizing Prompt Compression Methods For Long Context Inference, by Siddharth Jha et al.
Characterizing Prompt Compression Methods for Long Context Inferenceby Siddharth Jha, Lutfi Eren Erdogan, Sehoon Kim,…
Summary of Multimodal Task Vectors Enable Many-shot Multimodal In-context Learning, by Brandon Huang et al.
Multimodal Task Vectors Enable Many-Shot Multimodal In-Context Learningby Brandon Huang, Chancharik Mitra, Assaf Arbelle, Leonid…
Summary of Moa: Mixture Of Sparse Attention For Automatic Large Language Model Compression, by Tianyu Fu et al.
MoA: Mixture of Sparse Attention for Automatic Large Language Model Compressionby Tianyu Fu, Haofeng Huang,…
Summary of A Training-free Sub-quadratic Cost Transformer Model Serving Framework with Hierarchically Pruned Attention, by Heejun Lee et al.
A Training-free Sub-quadratic Cost Transformer Model Serving Framework With Hierarchically Pruned Attentionby Heejun Lee, Geon…
Summary of Samba: Simple Hybrid State Space Models For Efficient Unlimited Context Language Modeling, by Liliang Ren et al.
Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modelingby Liliang Ren, Yang…
Summary of A Study Of Optimizations For Fine-tuning Large Language Models, by Arjun Singh et al.
A Study of Optimizations for Fine-tuning Large Language Modelsby Arjun Singh, Nikhil Pandey, Anup Shirgaonkar,…
Summary of Dataset Decomposition: Faster Llm Training with Variable Sequence Length Curriculum, by Hadi Pouransari et al.
Dataset Decomposition: Faster LLM Training with Variable Sequence Length Curriculumby Hadi Pouransari, Chun-Liang Li, Jen-Hao…
Summary of Asymptotic Theory Of In-context Learning by Linear Attention, By Yue M. Lu et al.
Asymptotic theory of in-context learning by linear attentionby Yue M. Lu, Mary I. Letey, Jacob…