Summary of Text Role Classification in Scientific Charts Using Multimodal Transformers, by Hye Jin Kim et al.
Text Role Classification in Scientific Charts Using Multimodal Transformers
by Hye Jin Kim, Nicolas Lell, Ansgar Scherp
First submitted to arxiv on: 8 Feb 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Computation and Language (cs.CL); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes a finetuning approach for text role classification in scientific charts, using two multimodal document layout analysis models: LayoutLMv3 and UDOP. The transformers take as input the three modalities of text, image, and layout. The authors investigate the impact of data augmentation and balancing methods on model performance. The results show that LayoutLMv3 outperforms UDOP in all experiments, achieving an F1-macro score of 82.87 on the ICPR22 test dataset. Additionally, the paper evaluates the models’ robustness on a synthetic noisy dataset (ICPR22-N) and generalizability on three chart datasets (CHIME-R, DeGruyter, and EconBiz). |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The researchers developed a new method to identify the roles of text in scientific charts. They used special computer models that combine information from text, images, and layouts. The team tested their approach on several datasets and found that one model, LayoutLMv3, performed best. This model was able to correctly identify most text roles even when given limited training data. The results show the potential of this technique in analyzing scientific charts. |
Keywords
* Artificial intelligence * Classification * Data augmentation
