Summary of Towards Efficient Mixture Of Experts: a Holistic Study Of Compression Techniques, by Shwai He et al.
Towards Efficient Mixture of Experts: A Holistic Study of Compression Techniquesby Shwai He, Daize Dong,…
Summary of Parrot: Multilingual Visual Instruction Tuning, by Hai-long Sun et al.
Parrot: Multilingual Visual Instruction Tuningby Hai-Long Sun, Da-Wei Zhou, Yang Li, Shiyin Lu, Chao Yi,…
Summary of Reservoir History Matching Of the Norne Field with Generative Exotic Priors and a Coupled Mixture Of Experts — Physics Informed Neural Operator Forward Model, by Clement Etienam et al.
Reservoir History Matching of the Norne field with generative exotic priors and a coupled Mixture…
Summary of Monde: Mixture Of Near-data Experts For Large-scale Sparse Models, by Taehyun Kim et al.
MoNDE: Mixture of Near-Data Experts for Large-Scale Sparse Modelsby Taehyun Kim, Kwanseok Choi, Youngmock Cho,…
Summary of Enhancing Fast Feed Forward Networks with Load Balancing and a Master Leaf Node, by Andreas Charalampopoulos et al.
Enhancing Fast Feed Forward Networks with Load Balancing and a Master Leaf Nodeby Andreas Charalampopoulos,…
Summary of A Provably Effective Method For Pruning Experts in Fine-tuned Sparse Mixture-of-experts, by Mohammed Nowaz Rabbani Chowdhury et al.
A Provably Effective Method for Pruning Experts in Fine-tuned Sparse Mixture-of-Expertsby Mohammed Nowaz Rabbani Chowdhury,…
Summary of Wasserstein Distances, Neuronal Entanglement, and Sparsity, by Shashata Sawmya et al.
Wasserstein Distances, Neuronal Entanglement, and Sparsityby Shashata Sawmya, Linghao Kong, Ilia Markov, Dan Alistarh, Nir…
Summary of Revisiting Moe and Dense Speed-accuracy Comparisons For Llm Training, by Xianzhi Du et al.
Revisiting MoE and Dense Speed-Accuracy Comparisons for LLM Trainingby Xianzhi Du, Tom Gunter, Xiang Kong,…
Summary of Unchosen Experts Can Contribute Too: Unleashing Moe Models’ Power by Self-contrast, By Chufan Shi et al.
Unchosen Experts Can Contribute Too: Unleashing MoE Models’ Power by Self-Contrastby Chufan Shi, Cheng Yang,…
Summary of Graph Sparsification Via Mixture Of Graphs, by Guibin Zhang et al.
Graph Sparsification via Mixture of Graphsby Guibin Zhang, Xiangguo Sun, Yanwei Yue, Chonghe Jiang, Kun…