Summary of Round and Round We Go! What Makes Rotary Positional Encodings Useful?, by Federico Barbero et al.
Round and Round We Go! What makes Rotary Positional Encodings useful?by Federico Barbero, Alex Vitvitskyi,…
Summary of Time Transfer: on Optimal Learning Rate and Batch Size in the Infinite Data Limit, by Oleg Filatov et al.
Time Transfer: On Optimal Learning Rate and Batch Size In The Infinite Data Limitby Oleg…
Summary of Accelerating Diffusion Transformers with Token-wise Feature Caching, by Chang Zou et al.
Accelerating Diffusion Transformers with Token-wise Feature Cachingby Chang Zou, Xuyang Liu, Ting Liu, Siteng Huang,…
Summary of Sftmix: Elevating Language Model Instruction Tuning with Mixup Recipe, by Yuxin Xiao et al.
SFTMix: Elevating Language Model Instruction Tuning with Mixup Recipeby Yuxin Xiao, Shujian Zhang, Wenxuan Zhou,…
Summary of Differential Transformer, by Tianzhu Ye et al.
Differential Transformerby Tianzhu Ye, Li Dong, Yuqing Xia, Yutao Sun, Yi Zhu, Gao Huang, Furu…
Summary of Prefixquant: Eliminating Outliers by Prefixed Tokens For Large Language Models Quantization, By Mengzhao Chen et al.
PrefixQuant: Eliminating Outliers by Prefixed Tokens for Large Language Models Quantizationby Mengzhao Chen, Yi Liu,…
Summary of Dept: Decoupled Embeddings For Pre-training Language Models, by Alex Iacob et al.
DEPT: Decoupled Embeddings for Pre-training Language Modelsby Alex Iacob, Lorenzo Sani, Meghdad Kurmanji, William F.…
Summary of Tidaldecode: Fast and Accurate Llm Decoding with Position Persistent Sparse Attention, by Lijie Yang et al.
TidalDecode: Fast and Accurate LLM Decoding with Position Persistent Sparse Attentionby Lijie Yang, Zhihao Zhang,…
Summary of Sparsepo: Controlling Preference Alignment Of Llms Via Sparse Token Masks, by Fenia Christopoulou et al.
SparsePO: Controlling Preference Alignment of LLMs via Sparse Token Masksby Fenia Christopoulou, Ronald Cardenas, Gerasimos…
Summary of Timer-xl: Long-context Transformers For Unified Time Series Forecasting, by Yong Liu et al.
Timer-XL: Long-Context Transformers for Unified Time Series Forecastingby Yong Liu, Guo Qin, Xiangdong Huang, Jianmin…