Summary of Freeze the Backbones: a Parameter-efficient Contrastive Approach to Robust Medical Vision-language Pre-training, by Jiuming Qin et al.
Freeze the backbones: A Parameter-Efficient Contrastive Approach to Robust Medical Vision-Language Pre-training
by Jiuming Qin, Che Liu, Sibo Cheng, Yike Guo, Rossella Arcucci
First submitted to arxiv on: 2 Jan 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed Adaptor framework addresses the limitations of existing Vision-Language Self-Supervised Learning (VL-SSL) approaches by preserving prior knowledge in pre-trained encoders while reducing computation requirements. By freezing pre-trained image and text encoders and employing a lightweight Adaptor module for cross-modal learning, the framework achieves competitive performance on medical image classification and segmentation tasks across three datasets, with trainable parameters reduced by over 90% compared to current pre-training approaches. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The Adaptor framework is a new way of learning from radiographic images and textual reports. It helps doctors diagnose patients better. The idea is to keep using big models that were already trained on lots of data, and then add a small extra part that makes the model learn from medical images. This makes it faster and more efficient. The results show that this new approach works well and can even be used with very little training data. |
Keywords
* Artificial intelligence * Image classification * Self supervised
