Summary of Peerarg: Argumentative Peer Review with Llms, by Purin Sukpanichnant et al.
PeerArg: Argumentative Peer Review with LLMs
by Purin Sukpanichnant, Anna Rapberger, Francesca Toni
First submitted to arxiv on: 25 Sep 2024
Categories
- Main: Artificial Intelligence (cs.AI)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This AI research paper proposes a novel pipeline called PeerArg to support and understand the reviewing and decision-making processes of peer review. The pipeline combines large language models (LLMs) with methods from knowledge representation. PeerArg takes in input a set of reviews for a paper and outputs the paper acceptance prediction. To evaluate its performance, the authors tested it on three different datasets compared to a novel end-2-end LLM that uses few-shot learning to predict paper acceptance given reviews. The results indicate that both models are capable of predicting paper acceptance from reviews, but a variant of the PeerArg pipeline outperforms this LLM. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Peer review is important for deciding which papers get published in scientific journals or conferences. However, it can be biased and hard to understand. Some AI techniques have been used to help with peer review, but they are difficult to explain and trust. This paper introduces a new system called PeerArg that uses special language models and knowledge representation methods. It takes reviews for a paper as input and predicts whether the paper should be accepted or not. The authors tested PeerArg on three different datasets and compared it to another AI model that can also predict acceptance based on reviews. The results show that both models work well, but PeerArg is slightly better. |
Keywords
» Artificial intelligence » Few shot
