Summary of Meta 3d Assetgen: Text-to-mesh Generation with High-quality Geometry, Texture, and Pbr Materials, by Yawar Siddiqui et al.
Meta 3D AssetGen: Text-to-Mesh Generation with High-Quality Geometry, Texture, and PBR Materials
by Yawar Siddiqui, Tom Monnier, Filippos Kokkinos, Mahendra Kariya, Yanir Kleiman, Emilien Garreau, Oran Gafni, Natalia Neverova, Andrea Vedaldi, Roman Shapovalov, David Novotny
First submitted to arxiv on: 2 Jul 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Graphics (cs.GR)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper presents Meta 3D AssetGen, a significant advancement in text-to-3D generation that produces high-quality meshes with texture and material control. Unlike previous works, AssetGen outputs physically-based rendering (PBR) materials, allowing for realistic relighting. The model generates several views of an object with factored shaded and albedo appearance channels, then reconstructs colors, metalness, and roughness in 3D using a deferred shading loss for efficient supervision. Additionally, it uses a sign-distance function to represent the 3D shape more reliably and introduces a corresponding loss for direct shape supervision. The model is implemented using fused kernels for high memory efficiency. After mesh extraction, a texture refinement transformer operating in UV space improves sharpness and details. AssetGen achieves significant improvements in Chamfer Distance and LPIPS over concurrent works and industry competitors, with human preference of 72%. The project page provides access to generated assets. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper introduces Meta 3D AssetGen, a new way to create realistic 3D objects from text descriptions. This is important because it can help us make more accurate computer models of real-world things like buildings or cars. The model uses a combination of techniques to generate the object’s shape and appearance, including physically-based rendering (PBR) materials that allow for realistic lighting. AssetGen is able to create high-quality 3D objects with texture and material control, which is useful for applications such as computer-aided design, gaming, or virtual reality. |
Keywords
» Artificial intelligence » Transformer
