Summary of Power-softmax: Towards Secure Llm Inference Over Encrypted Data, by Itamar Zimerman et al.
Power-Softmax: Towards Secure LLM Inference over Encrypted Databy Itamar Zimerman, Allon Adir, Ehud Aharoni, Matan…
Summary of Training on Fake Labels: Mitigating Label Leakage in Split Learning Via Secure Dimension Transformation, by Yukun Jiang et al.
Training on Fake Labels: Mitigating Label Leakage in Split Learning via Secure Dimension Transformationby Yukun…
Summary of Mind the Gap: a Spectral Analysis Of Rank Collapse and Signal Propagation in Attention Layers, by Alireza Naderi et al.
Mind the Gap: a Spectral Analysis of Rank Collapse and Signal Propagation in Attention Layersby…
Summary of Benign Overfitting in Single-head Attention, by Roey Magen et al.
Benign Overfitting in Single-Head Attentionby Roey Magen, Shuning Shang, Zhiwei Xu, Spencer Frei, Wei Hu,…
Summary of Upcycling Large Language Models Into Mixture Of Experts, by Ethan He et al.
Upcycling Large Language Models into Mixture of Expertsby Ethan He, Abhinav Khattar, Ryan Prenger, Vijay…
Summary of Differential Transformer, by Tianzhu Ye et al.
Differential Transformerby Tianzhu Ye, Li Dong, Yuqing Xia, Yutao Sun, Yi Zhu, Gao Huang, Furu…
Summary of Exaq: Exponent Aware Quantization For Llms Acceleration, by Moran Shkolnik et al.
EXAQ: Exponent Aware Quantization For LLMs Accelerationby Moran Shkolnik, Maxim Fishman, Brian Chmiel, Hilla Ben-Yaacov,…
Summary of On Expert Estimation in Hierarchical Mixture Of Experts: Beyond Softmax Gating Functions, by Huy Nguyen and Xing Han and Carl Harris and Suchi Saria and Nhat Ho
On Expert Estimation in Hierarchical Mixture of Experts: Beyond Softmax Gating Functionsby Huy Nguyen, Xing…
Summary of Unveil Benign Overfitting For Transformer in Vision: Training Dynamics, Convergence, and Generalization, by Jiarui Jiang et al.
Unveil Benign Overfitting for Transformer in Vision: Training Dynamics, Convergence, and Generalizationby Jiarui Jiang, Wei…
Summary of Cottention: Linear Transformers with Cosine Attention, by Gabriel Mongaras and Trevor Dohm and Eric C. Larson
Cottention: Linear Transformers With Cosine Attentionby Gabriel Mongaras, Trevor Dohm, Eric C. LarsonFirst submitted to…