Summary of Investigating Wit, Creativity, and Detectability Of Large Language Models in Domain-specific Writing Style Adaptation Of Reddit’s Showerthoughts, by Tolga Buz et al.
Investigating Wit, Creativity, and Detectability of Large Language Models in Domain-Specific Writing Style Adaptation of Reddit’s Showerthoughts
by Tolga Buz, Benjamin Frost, Nikola Genchev, Moritz Schneider, Lucie-Aimée Kaffee, Gerard de Melo
First submitted to arxiv on: 2 May 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Recent Large Language Models (LLMs) have demonstrated the ability to generate content that is indistinguishable from human writing. This paper investigates the capability of differently-sized LLMs to replicate human writing style in short, creative texts within the Showerthoughts domain, focusing on thoughts that may occur during mundane activities. The study compares GPT-2 and GPT-Neo fine-tuned on Reddit data with GPT-3.5 invoked in a zero-shot manner against human-authored texts. Human preference is measured across specific dimensions accounting for the quality of creative, witty texts. Additionally, the paper explores the ability of humans versus fine-tuned RoBERTa classifiers to detect AI-generated texts. The results show that human evaluators rate generated texts slightly worse on average regarding their creative quality but are unable to reliably distinguish between human-written and AI-generated texts. Furthermore, a dataset for creative, witty text generation based on Reddit Showerthoughts posts is provided. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper studies whether computers can write like humans. It looks at different computer programs that generate writing, called Large Language Models (LLMs). The researchers tested these models to see if they could write funny and creative sentences similar to those found in a website called Showerthoughts. They compared the computer-generated texts with ones written by humans and asked people which ones they liked better. Surprisingly, even though the computer-written texts weren’t as good on average, people couldn’t tell them apart from human-written texts. The researchers also created a dataset of funny sentences that computers can use to improve their writing skills. |
Keywords
» Artificial intelligence » Gpt » Text generation » Zero shot
