Gridded Transformer Neural Processes for Large Unstructured Spatio-Temporal Databy Matthew Ashman, Cristiana Diaconu, Eric Langezaal,…
Cluster-wise Graph Transformer with Dual-granularity Kernelized Attentionby Siyuan Huang, Yunchong Song, Jiayue Zhou, Zhouhan LinFirst…
Does Spatial Cognition Emerge in Frontier Models?by Santhosh Kumar Ramakrishnan, Erik Wijmans, Philipp Kraehenbuehl, Vladlen…
Round and Round We Go! What makes Rotary Positional Encodings useful?by Federico Barbero, Alex Vitvitskyi,…
Extracting Finite State Machines from Transformersby Rik Adriaensen, Jaron MaeneFirst submitted to arxiv on: 8…
Enhancing Temporal Modeling of Video LLMs via Time Gatingby Zi-Yuan Hu, Yiwu Zhong, Shijia Huang,…
Does RoBERTa Perform Better than BERT in Continual Learning: An Attention Sink Perspectiveby Xueying Bai,…
From Sparse Dependence to Sparse Attention: Unveiling How Chain-of-Thought Enhances Transformer Sample Efficiencyby Kaiyue Wen,…
LevAttention: Time, Space, and Streaming Efficient Algorithm for Heavy Attentionsby Ravindran Kannan, Chiranjib Bhattacharyya, Praneeth…
Transformers learn variable-order Markov chains in-contextby Ruida Zhou, Chao Tian, Suhas DiggaviFirst submitted to arxiv…