Summary of Retro: Reusing Teacher Projection Head For Efficient Embedding Distillation on Lightweight Models Via Self-supervised Learning, by Khanh-binh Nguyen and Chae Jung Park
Retro: Reusing teacher projection head for efficient embedding distillation on Lightweight Models via Self-supervised Learningby…