Summary of Soar: Self-supervision Optimized Uav Action Recognition with Efficient Object-aware Pretraining, by Ruiqi Xian et al.
SOAR: Self-supervision Optimized UAV Action Recognition with Efficient Object-Aware Pretraining
by Ruiqi Xian, Xiyang Wu, Tianrui Guan, Xijun Wang, Boqing Gong, Dinesh Manocha
First submitted to arxiv on: 26 Sep 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Robotics (cs.RO)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel self-supervised pretraining algorithm called SOAR is introduced for aerial footage captured by Unmanned Aerial Vehicles (UAVs). The algorithm incorporates human object knowledge throughout the pretraining process to enhance efficiency and downstream action recognition performance. This approach differs from prior works that primarily incorporate object information during fine-tuning. Specifically, a novel object-aware masking strategy retains visibility of certain patches related to objects, while an object-aware loss function adjusts reconstruction loss to prevent bias towards less informative background patches. SOAR with a vanilla ViT backbone outperforms best UAV action recognition models on the NEC-Drone and UAV-Human datasets, achieving 9.7% and 21.4% boosts in top-1 accuracy, respectively, while delivering an inference speed of 18.7ms per video (2x to 5x faster than prior methods). SOAR also obtains comparable accuracy to prior self-supervised learning (SSL) methods while requiring 87.5% less pretraining time and 25% less memory usage. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary SOAR is a new way to make computers learn from aerial videos taken by drones. It helps them recognize actions in the video better. The algorithm does this by focusing on specific parts of the video that have people or objects, rather than just the background. This makes it more efficient and accurate. SOAR is faster and uses less memory than other methods that do something similar. It also works well on different datasets. |
Keywords
» Artificial intelligence » Fine tuning » Inference » Loss function » Pretraining » Self supervised » Vit
