Summary of Egtr: Extracting Graph From Transformer For Scene Graph Generation, by Jinbae Im et al.
EGTR: Extracting Graph from Transformer for Scene Graph Generation
by Jinbae Im, JeongYeon Nam, Nokyung Park, Hyungmin Lee, Seunghyun Park
First submitted to arxiv on: 2 Apr 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed lightweight one-stage Scene Graph Generation (SGG) model leverages the multi-head self-attention layers of DETR’s decoder to extract relation graphs from various object relationships. The model utilizes by-products from self-attention and a shallow relation extraction head for effective relation graph extraction. Additionally, a novel relation smoothing technique adjusts relation labels according to object detection quality, allowing for continuous curriculum training that focuses on object detection initially and shifts to multi-task learning as performance improves. The method also incorporates a connectivity prediction task as an auxiliary task to predict relations between object pairs. Experiments demonstrate the effectiveness and efficiency of the proposed approach on Visual Genome and Open Image V6 datasets. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary A new way to understand scenes is being developed by researchers who are working with computer vision. They want to identify objects in pictures and figure out how they relate to each other. This is called Scene Graph Generation, or SGG for short. The team came up with a simpler approach that uses something called self-attention to find relationships between objects. They also invented a way to adjust their method based on how well they detect the objects in the first place. To make it even better, they added another task that helps predict whether certain objects are related or not. This new method worked really well when tested with two different types of images. |
Keywords
* Artificial intelligence * Decoder * Multi task * Object detection * Self attention
