Summary of What Really Is Commonsense Knowledge?, by Quyet V. Do et al.
What Really is Commonsense Knowledge?
by Quyet V. Do, Junze Li, Tung-Duong Vuong, Zhaowei Wang, Yangqiu Song, Xiaojuan Ma
First submitted to arxiv on: 6 Nov 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper addresses the issue of genuineness in commonsense reasoning benchmarks in Natural Language Processing. The study argues that some existing datasets, developed through crowdsource human annotation, do not actually concern commonsense knowledge. This problem undermines the measurement of a model’s true commonsense reasoning ability. To resolve this, the authors provide a unified definition of commonsense knowledge, consolidating various frameworks. They then apply this definition to annotate and experiment on two popular datasets, CommonsenseQA and CommonsenseQA 2.0. The results show that Large Language Models (LLMs) perform worse on instances that truly require commonsense knowledge. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research paper looks at how well computer models can understand common sense. Right now, there are some problems with the way we test these models. Some tests might not even be asking questions that use common sense! To fix this, the researchers created a new definition of what common sense means and used it to check two big datasets. They found out that many of the questions in these datasets aren’t actually about common sense at all. This makes it hard to know how well the models are really doing when they’re tested on common sense. |
Keywords
» Artificial intelligence » Natural language processing
