Co-Supervised Learning: Improving Weak-to-Strong Generalization with Hierarchical Mixture of Expertsby Yuejiang Liu, Alexandre AlahiFirst submitted…
Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Modelsby…
Scaling physics-informed hard constraints with mixture-of-expertsby Nithin Chalapathi, Yiheng Du, Aditi KrishnapriyanFirst submitted to arxiv…
Towards an empirical understanding of MoE design choicesby Dongyang Fan, Bettina Messmer, Martin JaggiFirst submitted…
Multilinear Mixture of Experts: Scalable Expert Specialization through Factorizationby James Oldfield, Markos Georgopoulos, Grigorios G.…
HyperMoE: Towards Better Mixture of Experts via Transferring Among Expertsby Hao Zhao, Zihan Qiu, Huijia…
AMEND: A Mixture of Experts Framework for Long-tailed Trajectory Predictionby Ray Coden Mercurius, Ehsan Ahmadi,…
Fiddler: CPU-GPU Orchestration for Fast Inference of Mixture-of-Experts Modelsby Keisuke Kamahori, Tian Tang, Yile Gu,…
Multimodal Clinical Trial Outcome Prediction with Large Language Modelsby Wenhao Zheng, Liaoyaqi Wang, Dongshen Peng,…
On Parameter Estimation in Deviated Gaussian Mixture of Expertsby Huy Nguyen, Khai Nguyen, Nhat HoFirst…