Summary of Da-moe: Towards Dynamic Expert Allocation For Mixture-of-experts Models, by Maryam Akhavan Aghdam et al.
DA-MoE: Towards Dynamic Expert Allocation for Mixture-of-Experts Modelsby Maryam Akhavan Aghdam, Hongpeng Jin, Yanzhao WuFirst…
Summary of Vila-u: a Unified Foundation Model Integrating Visual Understanding and Generation, by Yecheng Wu et al.
VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationby Yecheng Wu, Zhuoyang Zhang, Junyu…
Summary of Residual Stream Analysis with Multi-layer Saes, by Tim Lawson et al.
Residual Stream Analysis with Multi-Layer SAEsby Tim Lawson, Lucy Farnik, Conor Houghton, Laurence AitchisonFirst submitted…
Summary of Preserving Empirical Probabilities in Bert For Small-sample Clinical Entity Recognition, by Abdul Rehman et al.
Preserving Empirical Probabilities in BERT for Small-sample Clinical Entity Recognitionby Abdul Rehman, Jian Jun Zhang,…
Summary of Masked Diffusion Models Are Secretly Time-agnostic Masked Models and Exploit Inaccurate Categorical Sampling, by Kaiwen Zheng et al.
Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Samplingby Kaiwen Zheng,…
Summary of Deconfounded Causality-aware Parameter-efficient Fine-tuning For Problem-solving Improvement Of Llms, by Ruoyu Wang et al.
Deconfounded Causality-aware Parameter-Efficient Fine-Tuning for Problem-Solving Improvement of LLMsby Ruoyu Wang, Xiaoxuan Li, Lina YaoFirst…
Summary of Olmoe: Open Mixture-of-experts Language Models, by Niklas Muennighoff et al.
OLMoE: Open Mixture-of-Experts Language Modelsby Niklas Muennighoff, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Jacob Morrison,…
Summary of Revisiting Smoe Language Models by Evaluating Inefficiencies with Task Specific Expert Pruning, By Soumajyoti Sarkar et al.
Revisiting SMoE Language Models by Evaluating Inefficiencies with Task Specific Expert Pruningby Soumajyoti Sarkar, Leonard…
Summary of Prompt Compression with Context-aware Sentence Encoding For Fast and Improved Llm Inference, by Barys Liskavets et al.
Prompt Compression with Context-Aware Sentence Encoding for Fast and Improved LLM Inferenceby Barys Liskavets, Maxim…
Summary of Self-supervised Vision Transformers For Writer Retrieval, by Tim Raven et al.
Self-Supervised Vision Transformers for Writer Retrievalby Tim Raven, Arthur Matei, Gernot A. FinkFirst submitted to…