Summary of Cmoraleval: a Moral Evaluation Benchmark For Chinese Large Language Models, by Linhao Yu et al.
CMoralEval: A Moral Evaluation Benchmark for Chinese Large Language Models
by Linhao Yu, Yongqi Leng, Yufei Huang, Shang Wu, Haixin Liu, Xinmeng Ji, Jiahui Zhao, Jinwang Song, Tingting Cui, Xiaoqing Cheng, Tao Liu, Deyi Xiong
First submitted to arxiv on: 19 Aug 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper presents CMoralEval, a large benchmark for morality evaluation of Chinese language models. The dataset is curated from two sources: Chinese TV programs discussing moral norms and news articles on morality. A taxonomy of morals and fundamental principles are established to ensure diversity and authenticity. An AI-assisted platform is developed to streamline instance annotation. The resulting CMoralEval contains 30,388 instances, including explicit moral scenarios and moral dilemmas. Experimental results demonstrate that CMoralEval is a challenging benchmark for Chinese language models. The dataset is publicly available. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper creates a big test to see how well large language models can understand what’s right or wrong in Chinese culture. They make a special set of examples (called a “dataset”) with stories and news articles that teach good values and principles. A computer tool helps people mark the examples, making it easier to use. The dataset has many different kinds of situations where you have to decide if something is right or wrong. It’s a big challenge for language models, but it can help us make them better. |
