Summary of Large Language Models For Classical Chinese Poetry Translation: Benchmarking, Evaluating, and Improving, by Andong Chen et al.
Large Language Models for Classical Chinese Poetry Translation: Benchmarking, Evaluating, and Improving
by Andong Chen, Lianzhang Lou, Kehai Chen, Xuefeng Bai, Yang Xiang, Muyun Yang, Tiejun Zhao, Min Zhang
First submitted to arxiv on: 19 Aug 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper introduces a challenging task in classical Chinese poetry translation, requiring both adequate and fluent translations while maintaining linguistic poetic elegance. The authors propose a new benchmark, PoetMT, and a custom metric based on GPT-4 to evaluate large language models (LLMs) in meeting these demands. Empirical evaluation reveals that existing LLMs fall short in this task. To address this challenge, the authors introduce a Retrieval-Augmented Machine Translation (RAT) method incorporating knowledge related to classical poetry. Experimental results show that RAT outperforms all comparison methods, including BLEU, COMET, BLEURT, and human evaluation. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research paper is about translating ancient Chinese poems into modern languages while keeping the original poetic style and cultural significance. The authors created a special test to evaluate how well computers can do this job. They found that current computer models are not good enough at doing this task. To improve, they developed a new way of using computer models with information about classical poetry. This new method worked better than others in tests. |
Keywords
» Artificial intelligence » Bleu » Gpt » Translation
