Summary of Doscenes: An Autonomous Driving Dataset with Natural Language Instruction For Human Interaction and Vision-language Navigation, by Parthib Roy et al.
doScenes: An Autonomous Driving Dataset with Natural Language Instruction for Human Interaction and Vision-Language Navigation
by Parthib Roy, Srinivasa Perisetla, Shashank Shriram, Harsha Krishnaswamy, Aryan Keskar, Ross Greer
First submitted to arxiv on: 8 Dec 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper introduces doScenes, a novel dataset designed to facilitate research on human-vehicle instruction interactions. The dataset is focused on short-term directives that directly influence vehicle motion and bridges the gap between instruction and driving response. The dataset annotates multimodal sensor data with natural language instructions and referentiality tags, enabling context-aware and adaptive planning. Unlike existing datasets, doScenes emphasizes actionable directives tied to static and dynamic scene objects, addressing limitations in prior research. This work lays the foundation for developing learning strategies that seamlessly integrate human instructions into autonomous systems. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper creates a special dataset called doScenes to help robots understand what humans want them to do. The dataset has natural language instructions like “turn left” or “go straight” and matches those with sensor data from cameras, lidar, and other sensors. This allows the robot to respond in a more context-aware way. The dataset focuses on short-term directives that affect the robot’s movement, which is different from previous datasets. This work will help make autonomous vehicles safer and more effective by allowing them to better understand human instructions. |
