Summary of How Lightweight Can a Vision Transformer Be, by Jen Hong Tan
How Lightweight Can A Vision Transformer Beby Jen Hong TanFirst submitted to arxiv on: 25…
Summary of Norface: Improving Facial Expression Analysis by Identity Normalization, By Hanwei Liu et al.
Norface: Improving Facial Expression Analysis by Identity Normalizationby Hanwei Liu, Rudong An, Zhimeng Zhang, Bowen…
Summary of Qwen2 Technical Report, by An Yang et al.
Qwen2 Technical Reportby An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou,…
Summary of Investigating the Potential Of Sparse Mixtures-of-experts For Multi-domain Neural Machine Translation, by Nadezhda Chirkova et al.
Investigating the potential of Sparse Mixtures-of-Experts for multi-domain neural machine translationby Nadezhda Chirkova, Vassilina Nikoulina,…
Summary of Simsmoe: Solving Representational Collapse Via Similarity Measure, by Giang Do et al.
SimSMoE: Solving Representational Collapse via Similarity Measureby Giang Do, Hung Le, Truyen TranFirst submitted to…
Summary of Adamoe: Token-adaptive Routing with Null Experts For Mixture-of-experts Language Models, by Zihao Zeng et al.
AdaMoE: Token-Adaptive Routing with Null Experts for Mixture-of-Experts Language Modelsby Zihao Zeng, Yibo Miao, Hongcheng…
Summary of Skywork-moe: a Deep Dive Into Training Techniques For Mixture-of-experts Language Models, by Tianwen Wei et al.
Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Modelsby Tianwen Wei, Bo Zhu,…
Summary of Yuan 2.0-m32: Mixture Of Experts with Attention Router, by Shaohua Wu et al.
Yuan 2.0-M32: Mixture of Experts with Attention Routerby Shaohua Wu, Jiangang Luo, Xi Chen, Lingjun…
Summary of Lora-switch: Boosting the Efficiency Of Dynamic Llm Adapters Via System-algorithm Co-design, by Rui Kong et al.
LoRA-Switch: Boosting the Efficiency of Dynamic LLM Adapters via System-Algorithm Co-designby Rui Kong, Qiyang Li,…
Summary of Meteora: Multiple-tasks Embedded Lora For Large Language Models, by Jingwei Xu and Junyu Lai and Yunpeng Huang
MeteoRA: Multiple-tasks Embedded LoRA for Large Language Modelsby Jingwei Xu, Junyu Lai, Yunpeng HuangFirst submitted…