Summary of Zero-shot Vision-and-language Navigation with Collision Mitigation in Continuous Environment, by Seongjun Jeong et al.
Zero-Shot Vision-and-Language Navigation with Collision Mitigation in Continuous Environment
by Seongjun Jeong, Gi-Cheon Kang, Joochan Kim, Byoung-Tak Zhang
First submitted to arxiv on: 7 Oct 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary We present Vision-and-Language Navigation with Collision Mitigation (VLN-CM), a zero-shot approach that navigates through environments while avoiding collisions. VLN-CM comprises four modules, predicting the direction and distance of the next movement at each step. Large foundation models are utilized for each module. The Attention Spot Predictor (ASP) uses ChatGPT to split navigation instructions into attention spots, which are objects or scenes. The View Selector (VS) selects panorama images based on CLIP similarity, choosing the angle as the direction to move. The Progress Monitor (PM) decides which attention spot to focus on next using a rule-based approach. If consecutive decreases in similarity occur, the PM determines the agent has passed the current spot and moves on. For distance selection, we employed the Open Map Predictor (OMP), which uses panorama depth information to predict an occupancy mask and selects a collision-free distance. Our method outperformed baseline methods, with OMP effectively mitigating collisions. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Imagine a robot that can navigate through environments without bumping into things! This paper proposes a new way for the robot to understand what it needs to do and where it should go next. It’s called Vision-and-Language Navigation with Collision Mitigation, or VLN-CM for short. The robot uses big models to help it decide which direction to move in and how far to go before stopping. This helps prevent collisions and lets the robot get where it needs to go safely. |
Keywords
» Artificial intelligence » Attention » Mask » Zero shot
