Summary of Cmmath: a Chinese Multi-modal Math Skill Evaluation Benchmark For Foundation Models, by Zhong-zhi Li et al.
CMMaTH: A Chinese Multi-modal Math Skill Evaluation Benchmark for Foundation Models
by Zhong-Zhi Li, Ming-Liang Zhang, Fei Yin, Zhi-Long Ji, Jin-Feng Bai, Zhen-Ru Pan, Fan-Hu Zeng, Jian Xu, Jia-Xin Zhang, Cheng-Lin Liu
First submitted to arxiv on: 28 Jun 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed paper aims to address the lack of evaluation tools and datasets for assessing mathematical capabilities in multimodal scenarios, specifically in the context of K12 education in Chinese language. To achieve this, the authors propose a benchmark called CMMaTH (Chinese Multi-modal Math Skill Evaluation Benchmark), which contains 23k multimodal K12 math-related questions covering elementary to high school levels. The CMMaTH dataset provides increased diversity in problem types, solution objectives, visual elements, detailed knowledge points, and standard solution annotations. A corresponding open-source tool called GradeGPT is integrated with the CMMaTH dataset, facilitating stable, rapid, and cost-free model evaluation. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper proposes a new benchmark to help evaluate large language models’ ability to solve Chinese math problems in different ways. This is important because right now there aren’t many tools or datasets that can do this well. The authors created a big dataset of 23,000 questions that cover math topics from elementary school to high school and include different types of problems and visuals. They also made an open-source tool called GradeGPT that makes it easy to use the benchmark and get quick results. |
Keywords
» Artificial intelligence » Multi modal
