Summary of Enhancing Vision-language Models with Scene Graphs For Traffic Accident Understanding, by Aaron Lohner et al.
Enhancing Vision-Language Models with Scene Graphs for Traffic Accident Understanding
by Aaron Lohner, Francesco Compagno, Jonathan Francis, Alessandro Oltramari
First submitted to arxiv on: 8 Jul 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Robotics (cs.RO)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes a novel approach for classifying traffic accidents in autonomous driving and road monitoring systems. The method represents traffic scenes as graphs, where objects are nodes and distances/directions between them are edges (scene graphs). By fusing scene graph inputs with visual and textual representations, the authors achieve better results. They introduce a multi-stage pipeline that preprocesses videos, encodes them into scene graphs, and aligns this representation with vision and language modalities before classification. The method is tested on 4 classes of traffic accidents using the Detection of Traffic Anomaly (DoTA) benchmark, achieving a balanced accuracy score of 57.77%, an increase of nearly 5 percentage points compared to not using scene graph information. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research helps create better autonomous driving systems and road monitoring tools by identifying different types of traffic accidents. The approach is unique in that it represents each accident as a visual “map” showing the position and movement of objects, like cars. This map is then combined with information from cameras and text data to help predict what type of accident is happening. The method was tested on real-life videos and showed significant improvement over previous methods. |
Keywords
» Artificial intelligence » Classification
