Summary of Do Language Models Exhibit the Same Cognitive Biases in Problem Solving As Human Learners?, by Andreas Opedal et al.
Do Language Models Exhibit the Same Cognitive Biases in Problem Solving as Human Learners?
by Andreas Opedal, Alessandro Stolfo, Haruki Shirakami, Ying Jiao, Ryan Cotterell, Bernhard Schölkopf, Abulhair Saparov, Mrinmaya Sachan
First submitted to arxiv on: 31 Jan 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This research investigates how well large language models (LLMs) can mimic human cognition when solving arithmetic word problems. The study aims to identify which aspects of human problem-solving are accurately modeled by LLMs and which areas require improvement. To achieve this, the authors divide the problem-solving process into three stages: text comprehension, solution planning, and solution execution. They develop tests for each stage to examine whether current LLMs display similar cognitive biases as children in these steps. The authors create a set of novel word problems using a neuro-symbolic approach, allowing them to fine-tune control over the problem features. The results show that LLMs, with or without instruction tuning, exhibit human-like biases in both text comprehension and solution planning stages, but not in the final stage where arithmetic expressions are executed. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research looks at how well big computer models (LLMs) can think like humans when solving math problems. The study wants to see which parts of our brain’s problem-solving process the LLMs get right and which they don’t. To do this, the authors break down the problem-solving process into three steps: reading the problem, thinking about how to solve it, and doing the math. They make tests for each step to see if the LLMs have similar “biases” (mistakes) as humans in these areas. The researchers create new word problems using a special approach that lets them control what’s in the problems. The results show that the computer models act like humans in some parts of the process, but not in others. |
Keywords
* Artificial intelligence * Instruction tuning
