Summary of Who Evaluates the Evaluations? Objectively Scoring Text-to-image Prompt Coherence Metrics with T2iscorescore (ts2), by Michael Saxon et al.
Who Evaluates the Evaluations? Objectively Scoring Text-to-Image Prompt Coherence Metrics with T2IScoreScore (TS2)
by Michael Saxon, Fatima Jahara, Mahsa Khoshnoodi, Yujie Lu, Aditya Sharma, William Yang Wang
First submitted to arxiv on: 5 Apr 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes a rigorous comparison and benchmarking of various text-to-image (T2I) faithfulness metrics. The authors leverage advances in cross-modal embeddings and vision-language models (VLMs) to develop a range of metrics that measure the semantic coherence of generated images with their prompts. The paper highlights the need for a robust evaluation framework, as current metrics are presented with correlation to human Likert scores over a limited set of easy-to-discriminate images against weak baselines. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper explores how to accurately evaluate text-to-image models that generate images from prompts. Right now, people use different ways to measure how well these models do, but they haven’t been compared or tested thoroughly. The authors want to change this by creating a system to test and compare these metrics against each other. |
