Summary of Mixture Of Rationale: Multi-modal Reasoning Mixture For Visual Question Answering, by Tao Li et al.
Mixture of Rationale: Multi-Modal Reasoning Mixture for Visual Question Answering
by Tao Li, Linjun Shou, Xuejun Liu
First submitted to arxiv on: 3 Jun 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed Mixture of Rationales (MoR) method is a novel multi-modal reasoning approach for zero-shot visual question answering (VQA). Unlike existing methods that rely on a single rationale within the Chain of Thoughts (CoT) framework, MoR mixes multiple rationales to capture the complexity of VQA. This approach uses a frozen Vision-and-Language Pre-trained Models (VLPM) model to dynamically generate and fuse multi-modal thoughts. The authors evaluate MoR on two challenging VQA datasets, NLVR2 and OKVQA, with OFA and VL-T5 as representative backbones. MoR achieves significant accuracy improvements of 12.43% on NLVR2 and 2.45% on OKVQA-S. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary MoR is a new way for computers to answer questions about pictures without being trained beforehand. It’s like having a conversation with someone, but instead of words, it uses images and text together. This method is special because it takes multiple ideas from different sources and combines them to get the best answer. It was tested on two difficult picture-question tasks and did better than other methods in finding the right answers. |
Keywords
» Artificial intelligence » Multi modal » Question answering » T5 » Zero shot
