Bridging the Training-Inference Gap in LLMs by Leveraging Self-Generated Tokensby Zhepeng Cen, Yao Liu, Siliang…
A Large Language Model-Driven Reward Design Framework via Dynamic Feedback for Reinforcement Learningby Shengjie Sun,…
Decomposing The Dark Matter of Sparse Autoencodersby Joshua Engels, Logan Riggs, Max TegmarkFirst submitted to…
Rethinking Token Reduction for State Space Modelsby Zheng Zhan, Yushu Wu, Zhenglun Kong, Changdi Yang,…
Optimizing Attention with Mirror Descent: Generalized Max-Margin Token Selectionby Addison Kristanto Julistiono, Davoud Ataee Tarzanagh,…
Revisiting SLO and Goodput Metrics in LLM Servingby Zhibin Wang, Shipeng Li, Yuhang Zhou, Xue…
In-context learning and Occam’s razorby Eric Elmoznino, Tom Marty, Tejas Kasetty, Leo Gagnon, Sarthak Mittal,…
Fluid: Scaling Autoregressive Text-to-image Generative Models with Continuous Tokensby Lijie Fan, Tianhong Li, Siyang Qin,…
Enhancing Generalization in Sparse Mixture of Experts Models: The Case for Increased Expert Activation in…
Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMsby Tianyu Guo, Druv Pai, Yu Bai,…