Summary of Webcode2m: a Real-world Dataset For Code Generation From Webpage Designs, by Yi Gui et al.
WebCode2M: A Real-World Dataset for Code Generation from Webpage Designs
by Yi Gui, Zhen Li, Yao Wan, Yemin Shi, Hongyu Zhang, Yi Su, Bohua Chen, Dongping Chen, Siyuan Wu, Xing Zhou, Wenbin Jiang, Hai Jin, Xiangliang Zhang
First submitted to arxiv on: 9 Apr 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Software Engineering (cs.SE)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A multimodal large language model (MLLM) has the potential to significantly reduce the workload of front-end developers by automatically generating webpage code from designs. However, existing MLLMs are constrained by the lack of high-quality datasets, resulting in inadequate performance. This paper introduces WebCode2M, a new dataset comprising 2.56 million instances of design images with corresponding webpage code and layout details. The dataset is sourced from real-world web resources and offers a rich and valuable resource for webpage code generation across various applications. A baseline model based on the Vision Transformer (ViT), named WebCoder, is introduced to validate the effectiveness of the dataset. Additionally, a new metric, TreeBLEU, is introduced to measure structural hierarchy recall. The benchmarking results demonstrate that the dataset significantly improves the ability of MLLMs to generate code from webpage designs, confirming its effectiveness and usability for future applications in front-end design tools. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Webpage code generation can be automated by using multimodal large language models (MLLMs). This helps reduce the workload of front-end developers. However, existing MLLMs aren’t very good because they don’t have high-quality datasets to work with. The paper introduces a new dataset called WebCode2M that has 2.56 million instances of design images and corresponding webpage code and layout details. This dataset is useful for generating webpage code across different applications. |
Keywords
» Artificial intelligence » Large language model » Recall » Vision transformer » Vit
