Summary of The Solution For the Iccv 2023 1st Scientific Figure Captioning Challenge, by Dian Chao et al.
The Solution for the ICCV 2023 1st Scientific Figure Captioning Challenge
by Dian Chao, Xin Song, Shupeng Zhong, Boyuan Wang, Xiangyu Wu, Chen Zhu, Yang Yang
First submitted to arxiv on: 26 Mar 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed solution aims to improve the quality of image captions for figures in papers by adopting a summarization approach that generates captions based on textual content. To address discrepancies in OCR information, PaddleOCR is employed to extract OCR data from all images. The study also recognizes noise introduced during caption generation due to irrelevant text and leverages LLaMA to filter out extraneous information. Additionally, the paper highlights the gap between maximum likelihood estimation during text generation and evaluation metrics like ROUGE. To bridge this gap, the BRIO model framework is integrated, enabling a more coherent alignment between generation and evaluation processes. The approach ranked first in the final test with a score of 4.49. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The study aims to improve the quality of image captions for figures in papers. It proposes a new approach that generates captions by summarizing textual content. To make sure captions are accurate, it uses special tools to get correct information from images. The study also finds that some text is not helpful for caption generation and filters it out using LLaMA. Another problem found was the difference between how captions are generated and how they’re evaluated. This paper solves this by using a new model framework called BRIO. With this approach, the study got a high score of 4.49 in the final test. |
Keywords
» Artificial intelligence » Alignment » Likelihood » Llama » Rouge » Summarization » Text generation
