Summary of Magic: Mastering Physical Adversarial Generation in Context Through Collaborative Llm Agents, by Yun Xing et al.
MAGIC: Mastering Physical Adversarial Generation in Context through Collaborative LLM Agents
by Yun Xing, Nhat Chung, Jie Zhang, Yue Cao, Ivor Tsang, Yang Liu, Lei Ma, Qing Guo
First submitted to arxiv on: 11 Dec 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes a novel framework called MAGIC (Mastering Physical Adversarial Generation In Context) to generate physical adversarial attacks in driving scenarios. The authors reformulate this challenge as a one-shot patch generation problem, developing a deep generative model that considers scene context to create adversarial patches. This approach enables direct physical deployment in matching environments. The primary challenge lies in achieving two objectives: generating effective misleads of object detection systems while determining contextually appropriate deployments within the scenes. To address this, MAGIC uses multi-modal LLM agents, comprising a GAgent for adv-patch generation, DAgent for deployment strategy determination, and EAgent for iterative refinement. The authors validate their method on both digital (nuImage) and physical levels (manually captured real-world scenes), demonstrating its effectiveness in attacking widely applied object detection systems like YOLO and DETR series. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary MAGIC is a new way to make fake images that can trick self-driving cars. It’s hard to create these fake images because they have to look realistic and be able to trick the car’s cameras. The researchers developed a special computer program that can understand what it sees and create fake images that fit in with the real scene. This program has three parts: one part makes the fake image, another part decides where to put it in the scene, and the third part checks if everything is working correctly. The researchers tested their program on both digital and real-world scenes and found that it was very good at tricking the car’s cameras. |
Keywords
» Artificial intelligence » Generative model » Multi modal » Object detection » One shot » Yolo
