Summary of Relitlrm: Generative Relightable Radiance For Large Reconstruction Models, by Tianyuan Zhang et al.
RelitLRM: Generative Relightable Radiance for Large Reconstruction Models
by Tianyuan Zhang, Zhengfei Kuang, Haian Jin, Zexiang Xu, Sai Bi, Hao Tan, He Zhang, Yiwei Hu, Milos Hasan, William T. Freeman, Kai Zhang, Fujun Luan
First submitted to arxiv on: 8 Oct 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Graphics (cs.GR); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes RelitLRM, a Large Reconstruction Model that generates high-quality Gaussian splatting representations of 3D objects under novel illuminations from sparse (4-8) posed images. Unlike previous inverse rendering methods requiring dense captures and slow optimization, RelitLRM uses a feed-forward transformer-based model with a novel combination of geometry reconstruction and relightable appearance generation based on diffusion. The model is trained end-to-end on synthetic multi-view renderings of objects under varying known illuminations. This architecture enables the decomposition of geometry and appearance, resolving ambiguities between material and lighting, and capturing multi-modal distributions of shadows and specularity. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary RelitLRM is a new way to create 3D models that look good in different lights. It uses a special kind of computer program called a transformer to take pictures from many angles and make a 3D model. This program can also change the lighting on the 3D model to make it look like it’s being lit by different things. The model is trained using fake data that looks like real pictures, which helps it learn how to do this. RelitLRM is better than other methods because it can work with fewer pictures and be faster. |
Keywords
» Artificial intelligence » Diffusion » Multi modal » Optimization » Transformer
