Summary of Noisyeqa: Benchmarking Embodied Question Answering Against Noisy Queries, by Tao Wu et al.
NoisyEQA: Benchmarking Embodied Question Answering Against Noisy Queries
by Tao Wu, Chuhao Zhou, Yen Heng Wong, Lin Gu, Jianfei Yang
First submitted to arxiv on: 14 Dec 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Robotics (cs.RO)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper introduces a new benchmark for Embodied Question Answering (EQA) called NoisyEQA, designed to evaluate an agent’s ability to recognize and correct noisy questions. This is achieved by introducing four types of noise common in real-world applications: Latent Hallucination Noise, Memory Noise, Perception Noise, and Semantic Noise. The authors also propose a ‘Self-Correction’ prompting mechanism and a new evaluation metric to measure both noise detection capability and answer quality. Current EQA agents struggle to detect noise in questions, leading to responses that frequently contain erroneous information. The Self-Correction Prompting mechanism can effectively improve the accuracy of agent answers. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper is about making question-answering machines better at dealing with noisy or confusing questions. Right now, these machines often give wrong answers when they encounter questions that are hard to understand. To help them get better, the authors created a new way to test how well they can recognize and fix problems in questions. They also came up with a new way to ask the questions that helps the machines learn from their mistakes. |
Keywords
» Artificial intelligence » Hallucination » Prompting » Question answering
