Summary of Learning Long-term Spatial-temporal Graphs For Active Speaker Detection, by Kyle Min et al.
Learning Long-Term Spatial-Temporal Graphs for Active Speaker Detection
by Kyle Min, Sourya Roy, Subarna Tripathi, Tanaya Guha, Somdeb Majumdar
First submitted to arxiv on: 15 Jul 2022
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper introduces SPELL, a novel framework for active speaker detection (ASD) in videos with multiple speakers. It’s a complex task that requires learning effective audiovisual features and spatial-temporal correlations over long temporal windows. The proposed approach uses spatial-temporal graph learning to reduce ASD to a node classification task. This allows the model to reason over long temporal contexts without relying on computationally expensive fully connected graph neural networks. Experimental results on the AVA-ActiveSpeaker dataset show that SPELL outperforms all previous state-of-the-art approaches while requiring significantly lower memory and computational resources. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper creates a new way to find who’s talking in videos with many people. It’s like trying to figure out who’s speaking when there are lots of voices in a crowded room! The new method, called SPELL, uses special connections between different parts of the video to help it understand who’s saying what. This makes it much better at finding the person talking than other methods that use more complicated ways to analyze the video. And the best part is that it doesn’t need as many computer resources or memory to work. |
Keywords
» Artificial intelligence » Classification
