Summary of Mujo: Multimodal Joint Feature Space Learning For Human Activity Recognition, by Stefan Gerd Fritsch et al.
MuJo: Multimodal Joint Feature Space Learning for Human Activity Recognition
by Stefan Gerd Fritsch, Cennet Oguz, Vitor Fortes Rey, Lala Ray, Maximilian Kiefer-Emmanouilidis, Paul Lukowicz
First submitted to arxiv on: 6 Jun 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed work introduces a comprehensive Fitness Multimodal Activity Dataset (FiMAD) and a pre-training method called MuJo (Multimodal Joint Feature Space Learning) to enhance Human Activity Recognition (HAR) performance across various modalities. FiMAD is created using YouTube fitness videos and contains parallel video, language, pose, and simulated IMU sensor data. The authors show that classifiers pre-trained on FiMAD can increase the performance on real HAR datasets such as MM-Fit, MyoGym, MotionSense, and MHEALTH. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Human activity recognition is a problem in artificial intelligence that helps us understand what people are doing in different situations. This research wants to make it easier to recognize activities using sensors like those found in mobile phones and smartwatches. To do this, they created a big dataset called FiMAD which has lots of different types of data, such as videos, language, poses, and sensor readings. They also developed a new way to learn from this data, called MuJo. The authors tested their approach on several real-world datasets and showed that it can improve the accuracy of activity recognition while using less training data. |
Keywords
» Artificial intelligence » Activity recognition
