Summary of Less Is More For Improving Automatic Evaluation Of Factual Consistency, by Tong Wang et al.
Less is More for Improving Automatic Evaluation of Factual Consistency
by Tong Wang, Ninad Kulkarni, Yanjun Qi
First submitted to arxiv on: 9 Apr 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper investigates the factual consistency of automatically generated texts in relation to their source context. The authors focus on a previous method called AlignScore, which uses a unified alignment model to evaluate factual consistency. They find that using a smaller number of data points can actually improve performance and develop an improved factual consistency evaluation model called LIM-RA (Less Is More for Robust AlignScore). LIM-RA consistently outperforms AlignScore and other strong baselines like ChatGPT across four benchmarks, achieving the highest score on 24 of the 33 test datasets. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about making sure that computer-generated text makes sense in relation to where it came from. The authors look at a previous method called AlignScore, which tries to figure out if the text is accurate. They discover that using less data can actually make things better! They create a new way of checking accuracy called LIM-RA and test it on lots of different datasets. It does really well, beating other strong methods like ChatGPT in many cases. |
Keywords
* Artificial intelligence * Alignment
