Summary of Towards Deconfounded Image-text Matching with Causal Inference, by Wenhui Li et al.
Towards Deconfounded Image-Text Matching with Causal Inference
by Wenhui Li, Xinqi Su, Dan Song, Lanjun Wang, Kun Zhang, An-An Liu
First submitted to arxiv on: 22 Aug 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes a novel approach to image-text matching, addressing limitations in existing methods that overlook biases in datasets and learn spurious correlations. The authors utilize Structural Causal Models (SCMs) to illustrate how intra- and inter-modal confounders impact the task, and then introduce the Deconfounded Causal Inference Network (DCIN) to eliminate these biases. DCIN decomposes confounders into visual and textual features, eliminating spurious correlations and enabling the model to learn causality rather than biased associations. The method is evaluated on Flickr30K and MSCOCO datasets, demonstrating its superiority. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper tries to fix a problem with how computers match images with words. Right now, they just copy what’s in big datasets, which can be bad because those datasets might not be fair or balanced. This means the computer ends up learning things that aren’t really important, and it gets worse when it has to work on new tasks. The authors want to change this by using a special kind of model that looks for the real causes behind what’s in images and words, rather than just copying what’s already there. They test their idea with two big datasets, and it does better than other methods. |
Keywords
» Artificial intelligence » Inference
