Summary of Adversarial Transformer Language Models For Contextual Commonsense Inference, by Pedro Colon-hernandez et al.
Adversarial Transformer Language Models for Contextual Commonsense Inference
by Pedro Colon-Hernandez, Henry Lieberman, Yida Xin, Claire Yin, Cynthia Breazeal, Peter Chin
First submitted to arxiv on: 10 Feb 2023
Categories
- Main: Computation and Language (cs.CL)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A transformer-based model is developed for contextualized commonsense inference, addressing limitations such as lack of control over topics, limited commonsense knowledge during training, and potential hallucination of false facts. The approach introduces “hinting,” a language model prompting technique that utilizes both hard and soft prompts to advise the model on what to talk about. Additionally, a methodology is established for joint inference with multiple commonsense knowledge bases, aligning textual versions of assertions from three knowledge graphs (ConceptNet, ATOMIC2020, and GLUCOSE) with stories and target sentences. Experimental results demonstrate the effectiveness of this approach for joint inference with each knowledge graph. Furthermore, a GAN architecture is explored to generate contextualized commonsense assertions and score their plausibility through a discriminator. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary A new way to understand what’s true or false in a story is being developed. This helps by giving clues (called “hinting”) about what the story is talking about. It also combines information from many different sources, like ConceptNet, ATOMIC2020, and GLUCOSE, to make sure the answers still make sense for the context. The result is a way to generate true and believable answers based on what’s in a story. |
Keywords
» Artificial intelligence » Gan » Hallucination » Inference » Knowledge graph » Language model » Prompting » Transformer
