Summary of Lvsm: a Large View Synthesis Model with Minimal 3d Inductive Bias, by Haian Jin et al.
LVSM: A Large View Synthesis Model with Minimal 3D Inductive Bias
by Haian Jin, Hanwen Jiang, Hao Tan, Kai Zhang, Sai Bi, Tianyuan Zhang, Fujun Luan, Noah Snavely, Zexiang Xu
First submitted to arxiv on: 22 Oct 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Graphics (cs.GR); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed Large View Synthesis Model (LVSM) is a transformer-based approach for scalable and generalizable novel view synthesis from sparse-view inputs. The model consists of two architectures: an encoder-decoder LVSM, which encodes input image tokens into a fixed number of 1D latent tokens, functioning as a fully learned scene representation, and decodes novel-view images from them; and a decoder-only LVSM, which directly maps input images to novel-view outputs, completely eliminating intermediate scene representations. Both models bypass the 3D inductive biases used in previous methods, addressing novel view synthesis with a fully data-driven approach. The encoder-decoder model offers faster inference due to its independent latent representation, while the decoder-only LVSM achieves superior quality, scalability, and zero-shot generalization, outperforming previous state-of-the-art methods by 1.5 to 3.5 dB PSNR. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The Large View Synthesis Model (LVSM) is a new way to create new views of objects from just a few images. It’s like taking a picture from a different angle without having to take the actual photo. The model uses special computer vision techniques called transformers and it works really well! Two types of models were tested: one that breaks down the image into smaller parts, and another that directly creates the new view. Both models do a great job, but the second one is better at creating high-quality views. |
Keywords
» Artificial intelligence » Decoder » Encoder decoder » Generalization » Inference » Transformer » Zero shot
