Summary of Velora: a Low-rank Adaptation Approach For Efficient Rgb-event Based Recognition, by Lan Chen et al.
VELoRA: A Low-Rank Adaptation Approach for Efficient RGB-Event based Recognition
by Lan Chen, Haoxiang Yang, Pengpeng Shao, Haoyu Song, Xiao Wang, Zhicheng Zhao, Yaowei Wang, Yonghong Tian
First submitted to arxiv on: 28 Dec 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed method combines RGB and Event cameras to enhance pattern recognition performance by leveraging deep neural networks with fine-tuning strategies. Inspired by large model applications, the introduction of large models can further improve multi-modal task performance. However, full fine-tuning is inefficient, so lightweight methods like LoRA and Adapter are used. The paper proposes a novel parameter-efficient fine-tuning (PEFT) strategy to adapt pre-trained foundation vision models for RGB-Event-based classification. It extracts features from RGB frames and event streams using ViT with modality-specific LoRA tuning, considers frame differences to capture motion cues via a frame difference backbone network, and feeds these features into Transformer layers for efficient multi-modal feature learning. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper proposes a new way to improve pattern recognition by combining two types of cameras. It uses big neural networks that can be adjusted to get better results. This helps with tasks that use both visual and motion information. The method is efficient because it doesn’t need to fully adjust the big models, which would be time-consuming. |
Keywords
» Artificial intelligence » Classification » Fine tuning » Lora » Multi modal » Parameter efficient » Pattern recognition » Transformer » Vit
