Summary of Cm2-net: Continual Cross-modal Mapping Network For Driver Action Recognition, by Ruoyu Wang et al.
CM2-Net: Continual Cross-Modal Mapping Network for Driver Action Recognition
by Ruoyu Wang, Chen Cai, Wenqian Wang, Jianjun Gao, Dan Lin, Wenyang Liu, Kim-Hui Yap
First submitted to arxiv on: 17 Jun 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes a novel approach to improve driver action recognition by developing a Continual Cross-Modal Mapping Network (CM2-Net) that learns from multiple modalities, including infrared, depth, and RGB. Existing methods require extensive data collection for each modality, but CM2-Net can learn from newly-incoming modalities using prompts from previously learned modalities. This is achieved through Accumulative Cross-modal Mapping Prompting (ACMP), which maps informative features from previous modalities to the feature space of new modalities. The network continually learns and updates its prompts throughout the process, leading to improved recognition performance. Experimental results on the Drive&Act dataset demonstrate the effectiveness of CM2-Net for both uni- and multi-modal driver action recognition. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about making it easier to recognize what drivers are doing inside cars using different types of cameras. Right now, it’s hard to collect a lot of data for each type of camera, but this new approach can learn from any new camera by looking at what the other cameras have learned before. It’s like how we learn new things in school by building on what we already know. This method is better than others because it keeps getting smarter and more accurate as it learns. The results show that this method works well for recognizing driver actions, whether using one or multiple camera types. |
Keywords
» Artificial intelligence » Multi modal » Prompting
