Summary of Emobench: Evaluating the Emotional Intelligence Of Large Language Models, by Sahand Sabour et al.
EmoBench: Evaluating the Emotional Intelligence of Large Language Models
by Sahand Sabour, Siyang Liu, Zheyuan Zhang, June M. Liu, Jinfeng Zhou, Alvionna S. Sunaryo, Juanzi Li, Tatia M.C. Lee, Rada Mihalcea, Minlie Huang
First submitted to arxiv on: 19 Feb 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed EmoBench benchmark aims to comprehensively evaluate the Emotional Intelligence (EI) of Large Language Models (LLMs). Current benchmarks have limitations, such as focusing primarily on emotion recognition and relying on existing datasets with annotation errors. EmoBench draws from established psychological theories and defines machine EI as comprising Emotional Understanding and Emotional Application. The benchmark consists of 400 hand-crafted questions in English and Chinese, designed to require thorough reasoning and understanding. Results show a significant gap between the EI of existing LLMs and that of average humans, indicating a promising direction for future research. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary EmoBench is a new way to test how well language models understand and work with emotions. Right now, we don’t have good tests for this ability, so researchers are using old datasets that might not be accurate. EmoBench changes that by creating a set of 400 questions in English and Chinese that require people to think carefully about emotions. The results show that current language models are far from being as good at understanding emotions as humans are. |
