Summary of Spatio-temporal Context Prompting For Zero-shot Action Detection, by Wei-jhe Huang et al.
Spatio-Temporal Context Prompting for Zero-Shot Action Detection
by Wei-Jhe Huang, Min-Hung Chen, Shang-Hong Lai
First submitted to arxiv on: 28 Aug 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Medium Difficulty summary: This paper focuses on enhancing spatio-temporal action detection by incorporating interaction modeling, which captures the relationship between people and their surrounding context. The authors propose a method that leverages pre-trained image-language models to detect unseen actions. The approach includes a Context Prompting module that utilizes contextual information to prompt labels, and an Interest Token Spotting mechanism that employs pretrained visual knowledge to find each person’s interest context tokens. The experiments show that the proposed method achieves superior results compared to previous approaches on J-HMDB, UCF101-24, and AVA datasets. Additionally, the code and data are publicly available. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Low Difficulty summary: This research paper is about improving a computer task called spatio-temporal action detection. It involves recognizing actions in videos and understanding how people interact with their surroundings. The authors want to make this task better by using pre-trained models that can learn from images and text. They propose new ways to do this, such as using contextual information and identifying the interests of individual people. The results show that their approach is more effective than previous methods on several video datasets. |
Keywords
» Artificial intelligence » Prompt » Prompting » Token
