Irregularity-Informed Time Series Analysis: Adaptive Modelling of Spatial and Temporal Dynamicsby Liangwei Nathan Zheng, Zhengyang…
MoH: Multi-Head Attention as Mixture-of-Head Attentionby Peng Jin, Bo Zhu, Li Yuan, Shuicheng YanFirst submitted…
A Robust Multisource Remote Sensing Image Matching Method Utilizing Attention and Feature Enhancement Against Noise…
On the Training Convergence of Transformers for In-Context Classification of Gaussian Mixturesby Wei Shen, Ruida…
Meta-DT: Offline Meta-RL as Conditional Sequence Modeling with World Model Disentanglementby Zhi Wang, Li Zhang,…
On Rank-Dependent Generalisation Error Bounds for Transformersby Lan V. TruongFirst submitted to arxiv on: 15…
DODT: Enhanced Online Decision Transformer Learning through Dreamer’s Actor-Critic Trajectory Forecastingby Eric Hanchen Jiang, Zhi…
Survey and Evaluation of Converging Architecture in LLMs based on Footsteps of Operationsby Seongho Kim,…
Beyond Linear Approximations: A Novel Pruning Approach for Attention Matrixby Yingyu Liang, Jiangxuan Long, Zhenmei…
Bypassing the Exponential Dependency: Looped Transformers Efficiently Learn In-context by Multi-step Gradient Descentby Bo Chen,…