Summary of Fleur: An Explainable Reference-free Evaluation Metric For Image Captioning Using a Large Multimodal Model, by Yebin Lee et al.
FLEUR: An Explainable Reference-Free Evaluation Metric for Image Captioning Using a Large Multimodal Model
by Yebin Lee, Imseong Park, Myungjoo Kang
First submitted to arxiv on: 10 Jun 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel approach to image captioning evaluation is proposed, departing from traditional methods that rely on reference captions. FLEUR, an explainable reference-free metric, leverages a large multimodal model to assess the caption’s quality and provide insights into its assigned score. By introducing score smoothing to align with human judgment and user-defined grading criteria, FLEUR achieves high correlations across various benchmarks and reaches state-of-the-art results on Flickr8k-CF, COMPOSITE, and Pascal-50S. The proposed metric not only addresses the limitations of traditional methods but also provides a deeper understanding of the evaluation process. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Imagine trying to describe what’s happening in a picture without any help. That’s basically how most computers evaluate image captions today. They compare your caption with another one they have, and give it a score based on that. But this doesn’t really explain why you got that score. What if there was a way for the computer to understand what’s going on in the picture and tell you exactly why your caption is good or bad? That’s basically what FLEUR does. It’s an innovative new way of evaluating image captions that doesn’t need those other captions at all, and it gives you a score with reasons behind it. This can help us make better computers that understand pictures like we do. |
Keywords
» Artificial intelligence » Image captioning
