Summary of Peer Review As a Multi-turn and Long-context Dialogue with Role-based Interactions, by Cheng Tan et al.
Peer Review as A Multi-Turn and Long-Context Dialogue with Role-Based Interactions
by Cheng Tan, Dongxin Lyu, Siyuan Li, Zhangyang Gao, Jingxuan Wei, Siqi Ma, Zicheng Liu, Stan Z. Li
First submitted to arxiv on: 9 Jun 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper presents a reformulation of the peer-review process as a multi-turn, long-context dialogue, incorporating distinct roles for authors, reviewers, and decision makers. Large Language Models (LLMs) are used to simulate this dynamic and iterative process, capturing the complexities of real-world academic peer review. A comprehensive dataset containing over 26,841 papers with 92,017 reviews is constructed, facilitating the application of LLMs for multi-turn dialogues. The paper proposes a series of metrics to evaluate the performance of LLLs for each role under this reformulated peer-review setting. The work aims to enhance the LLM-driven peer-review process by incorporating dynamic, role-based interactions. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper is about making computers help with peer review, which is important because it can make academic publishing faster and more fair. The computer models are trained on lots of real reviews from top journals and conferences. The goal is to create a system that can have conversations like a human would during the peer-review process. The paper includes a big dataset that the computer models can use to learn how to do this. It also comes up with new ways to measure how well the computer models are doing. |
