The Importance of Being Scalable: Improving the Speed and Accuracy of Neural Network Interatomic Potentials…
Approximate attention with MLP: a pruning strategy for attention-based model in multivariate time series forecastingby…
Stick-breaking Attentionby Shawn Tan, Yikang Shen, Songlin Yang, Aaron Courville, Rameswar PandaFirst submitted to arxiv…
Focus Where It Matters: Graph Selective State Focused Attention Networksby Shikhar Vashistha, Neetesh KumarFirst submitted…
In-context Learning for Mixture of Linear Regressions: Existence, Generalization and Training Dynamicsby Yanhao Jin, Krishnakumar…
In-context KV-Cache Eviction for LLMs via Attention-Gateby Zihao Zeng, Bokai Lin, Tianqi Hou, Hao Zhang,…
State-space models can learn in-context by gradient descentby Neeraj Mohan Sushma, Yudou Tian, Harshvardhan Mestha,…
Rethinking Graph Transformer Architecture Design for Node Classificationby Jiajun Zhou, Xuanze Chen, Chenxuan Xie, Yu…
Quadratic Gating Functions in Mixture of Experts: A Statistical Insightby Pedram Akbarian, Huy Nguyen, Xing…
What Does It Mean to Be a Transformer? Insights from a Theoretical Hessian Analysisby Weronika…