Summary of Balanced Multi-modal Federated Learning Via Cross-modal Infiltration, by Yunfeng Fan et al.
Balanced Multi-modal Federated Learning via Cross-Modal Infiltration
by Yunfeng Fan, Wenchao Xu, Haozhao Wang, Jiaqi Zhu, Song Guo
First submitted to arxiv on: 31 Dec 2023
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The Federated Learning (FL) paradigm enables collaborative neural network training without exposing clients’ raw data. Current FL solutions primarily focus on uni-modal data, while multimodal data remains largely unexplored. This paper proposes a novel Cross-Modal Infiltration Federated Learning (FedCMI) framework to alleviate modality imbalance and knowledge heterogeneity in distributed conditions. The framework incorporates a two-projector module to integrate dominant modality knowledge while promoting local feature exploitation of weak modalities. Additionally, a class-wise temperature adaptation scheme ensures fair performance across different classes. Experimental results on popular datasets confirm the effectiveness of FedCMI for fully exploiting multimodal information. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Federated learning helps computers learn together without sharing private data. This is important because it keeps our personal info safe. Right now, most computer learning systems focus on one type of data, like pictures or words. But what if we want to use different types of data together? That’s a big problem in multimodal learning (MFL). The main issue is when some types of data are stronger than others, and it’s hard to combine them correctly. This paper proposes a new way to solve this problem called Cross-Modal Infiltration Federated Learning (FedCMI). It uses special tools to bring together the strong and weak data types and make sure they work well together. The authors tested their method on some popular datasets and found that it works really well! |
Keywords
* Artificial intelligence * Federated learning * Neural network * Temperature
