Summary of Multifix: An Xai-friendly Feature Inducing Approach to Building Models From Multimodal Data, by Mafalda Malafaia et al.
MultiFIX: An XAI-friendly feature inducing approach to building models from multimodal data
by Mafalda Malafaia, Thalea Schlender, Peter A. N. Bosman, Tanja Alderliesten
First submitted to arxiv on: 19 Feb 2024
Categories
- Main: Artificial Intelligence (cs.AI)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes a new multimodal data fusion pipeline called MultiFIX that can extract and combine relevant features from different data modalities, such as images and tabular data, to make predictions. The pipeline uses end-to-end deep learning architecture to train a predictive model and extract representative features of each modality. Explainable AI techniques are used to explain each part of the model, including attention maps for image inputs and symbolic expressions for tabular inputs. The fusion of the extracted features is also replaced by a symbolic expression learned with GP-GOMEA. The authors demonstrate the strengths and limitations of MultiFIX on synthetic problems and apply it to a publicly available dataset for detecting malignant skin lesions. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper introduces a new way to combine different types of data, like images and numbers, to make predictions. This is important because decisions in the health domain often rely on different kinds of information. The authors create a special pipeline called MultiFIX that can extract and combine features from each type of data. They also explain how each part of the pipeline works using special techniques. The results show that MultiFIX can be useful for certain tasks, but not all. The paper applies this new approach to detecting skin lesions. |
Keywords
» Artificial intelligence » Attention » Deep learning
