Summary of Faithful Chart Summarization with Chats-pi, by Syrine Krichene et al.
Faithful Chart Summarization with ChaTS-Pi
by Syrine Krichene, Francesco Piccinno, Fangyu Liu, Julian Martin Eisenschlos
First submitted to arxiv on: 29 May 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper presents a novel approach for evaluating the faithfulness of summaries generated from charts, called CHATS-CRITIC. The proposed metric is reference-free and consists of an image-to-text model to recover the table from a chart and a tabular entailment model that scores the summary sentence by sentence. The authors demonstrate that CHATS-CRITIC outperforms traditional metrics in evaluating summary quality, as measured by human ratings. Furthermore, they introduce CHATS-PI, a chart-to-summary pipeline that leverages CHATS-CRITIC to fix and rank candidate summaries generated from any chart-summarization model. The proposed approach achieves state-of-the-art results on two popular chart-to-summary datasets. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper helps us create better summaries of charts by making sure they are accurate and make sense. They developed a new way to evaluate the quality of these summaries, called CHATS-CRITIC, which doesn’t need any reference information. This approach is more effective than previous methods at judging how well a summary captures the original chart’s meaning. The authors also created a pipeline that uses this metric to fix and rank candidate summaries generated by other models. This new method performs better than others on two important datasets. |
Keywords
» Artificial intelligence » Summarization
