Summary of Interdreamer: Zero-shot Text to 3d Dynamic Human-object Interaction, by Sirui Xu et al.
InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interaction
by Sirui Xu, Ziyin Wang, Yu-Xiong Wang, Liang-Yan Gui
First submitted to arxiv on: 28 Mar 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper presents a novel framework for generating human-object interactions without requiring direct training on text-interaction pair data. The authors decouple interaction semantics and dynamics, leveraging pre-trained large models for high-level control over interaction semantics while introducing a world model to comprehend simple physics and object motion. The resulting framework, InterDreamer, generates realistic and coherent 3D HOI sequences in a zero-shot manner. The authors demonstrate the effectiveness of their approach on the BEHAVE and CHAIRS datasets. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper makes it possible for computers to generate human-like interactions between people and objects without needing special training data. This is achieved by breaking down the interaction into two parts: what’s happening (semantics) and how it’s happening (dynamics). The computer uses its existing knowledge to control the semantics, while a new “world model” helps with the dynamics. The result is a way for computers to generate realistic interactions without needing any specific training data. |
Keywords
* Artificial intelligence * Semantics * Zero shot
