Montessori-Instruct: Generate Influential Training Data Tailored for Student Learningby Xiaochuan Li, Zichun Yu, Chenyan XiongFirst…
MoE-Pruner: Pruning Mixture-of-Experts Large Language Model using the Hints from Its Routerby Yanyue Xie, Zhi…
What is Left After Distillation? How Knowledge Transfer Impacts Fairness and Biasby Aida Mohammadshahi, Yani…
Efficient and Robust Knowledge Distillation from A Stronger Teacher Based on Correlation Matchingby Wenqi Niu,…
Progressive distillation induces an implicit curriculumby Abhishek Panigrahi, Bingbin Liu, Sadhika Malladi, Andrej Risteski, Surbhi…
Auto-GDA: Automatic Domain Adaptation for Efficient Grounding Verification in Retrieval-Augmented Generationby Tobias Leemann, Periklis Petridis,…
CPFD: Confidence-aware Privileged Feature Distillation for Short Video Classificationby Jinghao Shi, Xiang Shen, Kaili Zhao,…
Dataset Distillation via Knowledge Distillation: Towards Efficient Self-Supervised Pre-Training of Deep Networksby Siddharth Joshi, Jiayi…
HarmAug: Effective Data Augmentation for Knowledge Distillation of Safety Guard Modelsby Seanie Lee, Haebin Seong,…
Fair4Free: Generating High-fidelity Fair Synthetic Samples using Data Free Distillationby Md Fahim Sikder, Daniel de…