Summary of Idat: a Multi-modal Dataset and Toolkit For Building and Evaluating Interactive Task-solving Agents, by Shrestha Mohanty et al.
IDAT: A Multi-Modal Dataset and Toolkit for Building and Evaluating Interactive Task-Solving Agents
by Shrestha Mohanty, Negar Arabzadeh, Andrea Tupini, Yuxuan Sun, Alexey Skrynnik, Artem Zholus, Marc-Alexandre Côté, Julia Kiseleva
First submitted to arxiv on: 12 Jul 2024
Categories
- Main: Artificial Intelligence (cs.AI)
- Secondary: Computation and Language (cs.CL); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper addresses the challenge of developing AI agents that can understand and execute natural language instructions through the IGLU competition at NeurIPS. Despite advancements, there is a need for effective evaluation platforms and datasets. The authors introduce a scalable data collection tool, resulting in a Multi-Modal dataset with 9,000 utterances and 1,000 clarification questions. They also present an interactive evaluation platform for qualitative analysis of agent performance through multi-turn communication with human annotators. These assets, referred to as IDAT (IGLU Dataset And Toolkit), aim to advance the development of intelligent AI agents. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper is about making computers understand and follow instructions in natural language. This is a big problem because we need computers to work with humans easily. The authors create a tool that can collect lots of examples of people giving commands to an AI system, which helps make the AI better. They also make a way for people to test if the AI is working well by having humans talk to it and see how it responds. |
Keywords
* Artificial intelligence * Multi modal
