O1 Replication Journey – Part 2: Surpassing O1-preview through Simple Distillation, Big Progress or Bitter…
Adversarial Prompt Distillation for Vision-Language Modelsby Lin Luo, Xin Wang, Bojia Zi, Shihao Zhao, Xingjun…
Improving Mathematical Reasoning Capabilities of Small Language Models via Feedback-Driven Distillationby Xunyu Zhu, Jian Li,…
Learning with Less: Knowledge Distillation from Large Language Models via Unlabeled Databy Juanhui Li, Sreyashi…
Over-parameterized Student Model via Tensor Decomposition Boosted Knowledge Distillationby Yu-Liang Zhan, Zhong-Yi Lu, Hao Sun,…
Multimodal Commonsense Knowledge Distillation for Visual Question Answeringby Shuo Yang, Siwen Luo, Soyeon Caren HanFirst…
Larger models yield better results? Streamlined severity classification of ADHD-related concerns using BERT-based knowledge distillationby…
Towards Effective Data-Free Knowledge Distillation via Diverse Diffusion Augmentationby Muquan Li, Dongyang Zhang, Tao He,…
Pre-training Distillation for Large Language Models: A Design Space Explorationby Hao Peng, Xin Lv, Yushi…
LLaVA-Ultra: Large Chinese Language and Vision Assistant for Ultrasoundby Xuechen Guo, Wenhao Chai, Shi-Yan Li,…