Summary of Humor in Ai: Massive Scale Crowd-sourced Preferences and Benchmarks For Cartoon Captioning, by Jifan Zhang et al.
Humor in AI: Massive Scale Crowd-Sourced Preferences and Benchmarks for Cartoon Captioning
by Jifan Zhang, Lalit Jain, Yang Guo, Jiayi Chen, Kuan Lok Zhou, Siddharth Suresh, Andrew Wagenmaker, Scott Sievert, Timothy Rogers, Kevin Jamieson, Robert Mankoff, Robert Nowak
First submitted to arxiv on: 15 Jun 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper presents a novel dataset for creative tasks, consisting of over 250 million human ratings on more than 2.2 million captions, collected through crowdsourcing rating data for The New Yorker’s weekly cartoon caption contest. This unique dataset supports the development and evaluation of multimodal large language models and preference-based fine-tuning algorithms for humorous caption generation. Novel benchmarks are proposed to judge the quality of model-generated captions, utilizing both GPT4 and human judgments to establish ranking-based evaluation strategies. Experimental results highlight the limitations of current fine-tuning methods when applied to creative tasks. Furthermore, even state-of-the-art models like GPT4 and Claude currently underperform top human contestants in generating humorous captions. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper creates a big dataset with lots of ratings on funny caption ideas. People have been voting on these captions for eight years! The data can help make better artificial intelligence (AI) that can come up with good jokes. The researchers also made new ways to test how well AI does at making people laugh. They found that even the best AI models are not as good as humans at coming up with funny captions. |
Keywords
» Artificial intelligence » Claude » Fine tuning
