Summary of Cross-modal Prototype Based Multimodal Federated Learning Under Severely Missing Modality, by Huy Q. Le et al.
Cross-Modal Prototype based Multimodal Federated Learning under Severely Missing Modality
by Huy Q. Le, Chu Myaet Thwal, Yu Qiao, Ye Lin Tun, Minh N. H. Nguyen, Eui-Nam Huh, Choong Seon Hong
First submitted to arxiv on: 25 Jan 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes Multimodal Federated Cross Prototype Learning (MFCPL), a novel approach for multimodal federated learning (MFL) when dealing with severely missing modalities. The authors highlight the challenges of data heterogeneity and missing modalities, which can lead to misalignment during local training and impact the performance of global models. To address this issue, MFCPL leverages complete prototypes to provide diverse modality knowledge at both modality-shared and modality-specific levels. This approach also introduces cross-modal alignment for regularization of modality-specific features, enhancing overall performance in scenarios with severely missing modalities. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about a new way to help different devices share information without sharing their own data. Right now, these devices can’t work together because they have different kinds of data and sometimes some data is missing. This makes it hard for them to learn from each other. The authors came up with a solution called Multimodal Federated Cross Prototype Learning (MFCPL). It uses the complete information from one device to help the others learn, even when some data is missing. |
Keywords
* Artificial intelligence * Alignment * Federated learning * Regularization
