Summary of Quadratic Gating Functions in Mixture Of Experts: a Statistical Insight, by Pedram Akbarian et al.
Quadratic Gating Functions in Mixture of Experts: A Statistical Insightby Pedram Akbarian, Huy Nguyen, Xing…
Summary of Mimetic Initialization Helps State Space Models Learn to Recall, by Asher Trockman et al.
Mimetic Initialization Helps State Space Models Learn to Recallby Asher Trockman, Hrayr Harutyunyan, J. Zico…
Summary of 3ds: Decomposed Difficulty Data Selection’s Case Study on Llm Medical Domain Adaptation, by Hongxin Ding et al.
3DS: Decomposed Difficulty Data Selection’s Case Study on LLM Medical Domain Adaptationby Hongxin Ding, Yue…
Summary of Towards Better Multi-head Attention Via Channel-wise Sample Permutation, by Shen Yuan et al.
Towards Better Multi-head Attention via Channel-wise Sample Permutationby Shen Yuan, Hongteng XuFirst submitted to arxiv…
Summary of A Few-shot Label Unlearning in Vertical Federated Learning, by Hanlin Gu et al.
A few-shot Label Unlearning in Vertical Federated Learningby Hanlin Gu, Hong Xi Tae, Chee Seng…
Summary of What Does It Mean to Be a Transformer? Insights From a Theoretical Hessian Analysis, by Weronika Ormaniec et al.
What Does It Mean to Be a Transformer? Insights from a Theoretical Hessian Analysisby Weronika…
Summary of When Attention Sink Emerges in Language Models: An Empirical View, by Xiangming Gu et al.
When Attention Sink Emerges in Language Models: An Empirical Viewby Xiangming Gu, Tianyu Pang, Chao…
Summary of Artificial Intelligence-based Triaging Of Cutaneous Melanocytic Lesions, by Ruben T. Lucassen et al.
Artificial Intelligence-Based Triaging of Cutaneous Melanocytic Lesionsby Ruben T. Lucassen, Nikolas Stathonikos, Gerben E. Breimer,…
Summary of Lolcats: on Low-rank Linearizing Of Large Language Models, by Michael Zhang et al.
LoLCATs: On Low-Rank Linearizing of Large Language Modelsby Michael Zhang, Simran Arora, Rahul Chalamala, Alan…
Summary of Learning Linear Attention in Polynomial Time, by Morris Yau et al.
Learning Linear Attention in Polynomial Timeby Morris Yau, Ekin Akyürek, Jiayuan Mao, Joshua B. Tenenbaum,…