Summary of Local to Global: Learning Dynamics and Effect Of Initialization For Transformers, by Ashok Vardhan Makkuva et al.
Local to Global: Learning Dynamics and Effect of Initialization for Transformersby Ashok Vardhan Makkuva, Marco…
Summary of Exact Conversion Of In-context Learning to Model Weights in Linearized-attention Transformers, by Brian K Chen et al.
Exact Conversion of In-Context Learning to Model Weights in Linearized-Attention Transformersby Brian K Chen, Tianyang…
Summary of Multi-layer Learnable Attention Mask For Multimodal Tasks, by Wayner Barrios and Souyoung Jin
Multi-layer Learnable Attention Mask for Multimodal Tasksby Wayner Barrios, SouYoung JinFirst submitted to arxiv on:…
Summary of Short-term Inland Vessel Trajectory Prediction with Encoder-decoder Models, by Kathrin Donandt et al.
Short-term Inland Vessel Trajectory Prediction with Encoder-Decoder Modelsby Kathrin Donandt, Karim Böttger, Dirk SöffkerFirst submitted…
Summary of Improved Context-sensitive Transformer Model For Inland Vessel Trajectory Prediction, by Kathrin Donandt et al.
Improved context-sensitive transformer model for inland vessel trajectory predictionby Kathrin Donandt, Karim Böttger, Dirk SöffkerFirst…
Summary of Block Transformer: Global-to-local Language Modeling For Fast Inference, by Namgyu Ho et al.
Block Transformer: Global-to-Local Language Modeling for Fast Inferenceby Namgyu Ho, Sangmin Bae, Taehyeon Kim, Hyunjik…
Summary of Contextual Counting: a Mechanistic Study Of Transformers on a Quantitative Task, by Siavash Golkar et al.
Contextual Counting: A Mechanistic Study of Transformers on a Quantitative Taskby Siavash Golkar, Alberto Bietti,…
Summary of A Temporal Kolmogorov-arnold Transformer For Time Series Forecasting, by Remi Genet and Hugo Inzirillo
A Temporal Kolmogorov-Arnold Transformer for Time Series Forecastingby Remi Genet, Hugo InzirilloFirst submitted to arxiv…
Summary of Towards Efficient Mixture Of Experts: a Holistic Study Of Compression Techniques, by Shwai He et al.
Towards Efficient Mixture of Experts: A Holistic Study of Compression Techniquesby Shwai He, Daize Dong,…
Summary of Learning to Grok: Emergence Of In-context Learning and Skill Composition in Modular Arithmetic Tasks, by Tianyu He et al.
Learning to grok: Emergence of in-context learning and skill composition in modular arithmetic tasksby Tianyu…