Summary of Rewind Dataset: Privacy-preserving Speaking Status Segmentation From Multimodal Body Movement Signals in the Wild, by Jose Vargas Quiros et al.
REWIND Dataset: Privacy-preserving Speaking Status Segmentation from Multimodal Body Movement Signals in the Wild
by Jose Vargas Quiros, Chirag Raman, Stephanie Tan, Ekin Gedik, Laura Cabrera-Quiros, Hayley Hung
First submitted to arxiv on: 2 Mar 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Signal Processing (eess.SP)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper tackles the challenge of recognizing human speech in social interactions, a crucial task in understanding social behavior. The traditional approach involves individual voice recordings, but this is often impractical due to cost, logistics, and privacy concerns. Instead, machine learning models trained on video and wearable sensor data can detect speech by analyzing related gestures, providing an unobtrusive and privacy-preserving method. These models should be trained using labels obtained from the audio signal itself. However, existing datasets lack high-quality individual speech recordings, relying instead on human-annotated video-based annotations without validation against audio-ground truth. This paper presents a new multimodal dataset with high-quality individual speech recordings of 33 subjects in a professional networking event. Three baselines are proposed for no-audio speaking status segmentation: video-based, body acceleration (chest-worn accelerometer), and body pose tracks. The binary speaking status signal extracted from the audio is predicted at a time resolution not available in previous datasets. This dataset provides signals and ground truth necessary to evaluate various speaking status detection methods. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about recognizing when people are talking or not while they’re socializing. It’s important because it helps us understand how we interact with each other. Currently, the best way to do this is by using individual voice recordings, but that can be tricky and may involve privacy concerns. Instead, scientists have developed a new way to detect speech by looking at people’s gestures while they talk. To make this work, they need to train special computer models on data from video cameras and wearable devices like smartwatches. But so far, there haven’t been many datasets that include both good audio recordings and annotations of what people are doing. That’s why the authors created a new dataset with 33 people talking at a professional networking event. They also proposed three ways to recognize when someone is speaking or not: by looking at their video, body movement, and acceleration. This research can help us develop better technology for understanding social interactions. |
Keywords
* Artificial intelligence * Machine learning
