Summary of Hapfi: History-aware Planning Based on Fused Information, by Sujin Jeon et al.
HAPFI: History-Aware Planning based on Fused Information
by Sujin Jeon, Suyeon Shin, Byoung-Tak Zhang
First submitted to arxiv on: 23 Jul 2024
Categories
- Main: Artificial Intelligence (cs.AI)
- Secondary: Robotics (cs.RO)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper introduces a task called Embodied Instruction Following (EIF), which involves planning a sequence of sub-goals based on high-level natural language instructions. The authors argue that an agent must consider its past experiences when making decisions in each step. They propose History-Aware Planning based on Fused Information (HAPFI) to leverage historical data from diverse modalities, including RGB observations, bounding boxes, and high-level instructions. HAPFI integrates these modalities using a Mutually Attentive Fusion method. The authors compare their approach with others that neglect historical data and demonstrate its superiority in action planning capability. They also provide qualitative evidence highlighting the importance of leveraging historical multi-modal data. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Embodied Instruction Following is a way for machines to follow instructions given in natural language. Imagine you’re trying to make a salad, and someone tells you what to do step by step: “Rinse a slice of lettuce…” The authors of this paper think that machines should also consider their past experiences when following these kinds of instructions. They propose a new way for machines to use information from the past to make better decisions in each step. This approach combines different types of data, like what’s happening around them and what they’re supposed to do. The results show that this method is better than others at planning the right actions. It also helps machines recover better if they make a mistake. |
Keywords
* Artificial intelligence * Multi modal
