Summary of Chain and Causal Attention For Efficient Entity Tracking, by Erwan Fagnou et al.
Chain and Causal Attention for Efficient Entity Trackingby Erwan Fagnou, Paul Caillon, Blaise Delattre, Alexandre…
Summary of Levattention: Time, Space, and Streaming Efficient Algorithm For Heavy Attentions, by Ravindran Kannan et al.
LevAttention: Time, Space, and Streaming Efficient Algorithm for Heavy Attentionsby Ravindran Kannan, Chiranjib Bhattacharyya, Praneeth…
Summary of Transformers Learn Variable-order Markov Chains In-context, by Ruida Zhou et al.
Transformers learn variable-order Markov chains in-contextby Ruida Zhou, Chao Tian, Suhas DiggaviFirst submitted to arxiv…
Summary of Differential Transformer, by Tianzhu Ye et al.
Differential Transformerby Tianzhu Ye, Li Dong, Yuqing Xia, Yutao Sun, Yi Zhu, Gao Huang, Furu…
Summary of Dept: Decoupled Embeddings For Pre-training Language Models, by Alex Iacob et al.
DEPT: Decoupled Embeddings for Pre-training Language Modelsby Alex Iacob, Lorenzo Sani, Meghdad Kurmanji, William F.…
Summary of Tidaldecode: Fast and Accurate Llm Decoding with Position Persistent Sparse Attention, by Lijie Yang et al.
TidalDecode: Fast and Accurate LLM Decoding with Position Persistent Sparse Attentionby Lijie Yang, Zhihao Zhang,…
Summary of Timer-xl: Long-context Transformers For Unified Time Series Forecasting, by Yong Liu et al.
Timer-XL: Long-Context Transformers for Unified Time Series Forecastingby Yong Liu, Guo Qin, Xiangdong Huang, Jianmin…
Summary of Timecnn: Refining Cross-variable Interaction on Time Point For Time Series Forecasting, by Ao Hu et al.
TimeCNN: Refining Cross-Variable Interaction on Time Point for Time Series Forecastingby Ao Hu, Dongkai Wang,…
Summary of Mastering Chinese Chess Ai (xiangqi) Without Search, by Yu Chen et al.
Mastering Chinese Chess AI (Xiangqi) Without Searchby Yu Chen, Juntong Lin, Zhichao ShuFirst submitted to…
Summary of On the Optimization and Generalization Of Two-layer Transformers with Sign Gradient Descent, by Bingrui Li et al.
On the Optimization and Generalization of Two-layer Transformers with Sign Gradient Descentby Bingrui Li, Wei…