Summary of Holmes: Holonym-meronym Based Semantic Inspection For Convolutional Image Classifiers, by Francesco Dibitonto et al.
HOLMES: HOLonym-MEronym based Semantic inspection for Convolutional Image Classifiers
by Francesco Dibitonto, Fabio Garcea, André Panisson, Alan Perotti, Lia Morra
First submitted to arxiv on: 13 Mar 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel technique called HOLMES (HOLonym-MEronym based Semantic inspection) is proposed to decompose image classification model outputs into component-level explanations. By leveraging ontologies, web scraping, and transfer learning, HOLMES constructs meronym-based detectors for a given holonym (class). The method produces heatmaps at the meronym level and probes the holonym CNN with occluded images to highlight the importance of each part on the classification output. Compared to state-of-the-art saliency methods, HOLMES provides information about both where and what the holonym CNN is looking at without relying on densely annotated datasets or forcing concepts to be associated with single computational units. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary HOLMES is a new way to understand how image classification models work. It takes a model’s output and breaks it down into smaller parts, showing us what features are most important for the prediction. This helps us see why the model made its decision. HOLMES uses special tools like ontologies (collections of knowledge) and web scraping (gathering data from websites) to do this. The results show that HOLMES is a good way to understand image classification models, and it can even be used to make them better. |
Keywords
* Artificial intelligence * Classification * Cnn * Image classification * Transfer learning
