Summary of Continual Vision-and-language Navigation, by Seongjun Jeong et al.
Continual Vision-and-Language Navigation
by Seongjun Jeong, Gi-Cheon Kang, Seongho Choi, Joochan Kim, Byoung-Tak Zhang
First submitted to arxiv on: 22 Mar 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed Continual Vision-and-Language Navigation (CVLN) paradigm tackles a critical limitation in current VLN agent development, which assumes a unrealistic train-once-deploy-once strategy. Instead, CVLN enables agents to continually learn and adapt to changing environments by training and evaluating incrementally across multiple scene domains. This approach considers diverse forms of natural language instructions, including initial-instruction based and dialogue-based navigation. To facilitate this learning process, two baseline methods are introduced: Perplexity Replay (PerpR) and Episodic Self-Replay (ESR). Experimental results demonstrate that PerpR and ESR outperform comparison methods by effectively utilizing replay memory. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary CVLN agents can navigate to a destination using natural language instructions and visual cues. But, they only learn once and then forget how to adapt to new environments. This is unrealistic because real-world agents need to continually learn and adapt. To fix this, we propose a CVLN paradigm that lets agents learn and adapt incrementally across different environments. We also introduce two baseline methods: PerpR and ESR, which help the agent learn from its past experiences. |
Keywords
» Artificial intelligence » Perplexity
