Summary of Can Large Language Models Automatically Score Proficiency Of Written Essays?, by Watheq Mansour et al.
Can Large Language Models Automatically Score Proficiency of Written Essays?
by Watheq Mansour, Salam Albatarni, Sohaila Eltanbouly, Tamer Elsayed
First submitted to arxiv on: 10 Mar 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper investigates the ability of Large Language Models (LLMs) to analyze and effectively score written essays. Two popular LLMs, ChatGPT and Llama, were tested on the ASAP dataset using four different prompts designed through prompt-engineering tactics. The results show that choosing the right prompt depends on the model and task nature, with ChatGPT slightly outperforming Llama in average AES performance. While the LLMs’ predictions do not match those of state-of-the-art (SOTA) models, their feedback can help improve essay quality for both teachers and students. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper looks at how well Large Language Models can score essays written by humans. Two powerful AI models, ChatGPT and Llama, were tested to see if they could do a better job than other scoring methods that have been tried in the past. The results show that these AI models are pretty good at scoring essays, but not as good as some other methods. Still, the feedback they give can be helpful for teachers and students trying to improve their writing. |
Keywords
» Artificial intelligence » Llama » Prompt
