Summary of Interintent: Investigating Social Intelligence Of Llms Via Intention Understanding in An Interactive Game Context, by Ziyi Liu et al.
InterIntent: Investigating Social Intelligence of LLMs via Intention Understanding in an Interactive Game Context
by Ziyi Liu, Abhishek Anand, Pei Zhou, Jen-tse Huang, Jieyu Zhao
First submitted to arxiv on: 18 Jun 2024
Categories
- Main: Artificial Intelligence (cs.AI)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This study examines large language models’ (LLMs) ability to demonstrate human-like social intelligence through a novel framework called InterIntent. The researchers created a game-based assessment that tests LLMs on four dimensions of social intelligence: situational awareness, self-regulation, self-awareness, and theory of mind. The results show that while LLMs excel in selecting intentions (88% accuracy), they struggle to infer the intentions of others, lagging behind human performance by 20%. Additionally, game performance is linked to intention understanding, underscoring the importance of this component in evaluating LLMs’ social intelligence. This study contributes a structured approach to assessing LLMs’ social intelligence through multiplayer games. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research investigates how well large language models (LLMs) can understand and work with people-like intentions. The scientists created a game to test LLMs on four important skills: knowing what’s happening, controlling their actions, understanding themselves, and guessing what others think. They found that while LLMs are great at choosing the right intention, they struggle to figure out what others really want. This study shows how we can use games to better understand and evaluate LLMs’ social skills. |
