Summary of Empowering the Deaf and Hard Of Hearing Community: Enhancing Video Captions Using Large Language Models, by Nadeen Fathallah et al.
Empowering the Deaf and Hard of Hearing Community: Enhancing Video Captions Using Large Language Models
by Nadeen Fathallah, Monika Bhole, Steffen Staab
First submitted to arxiv on: 30 Nov 2024
Categories
- Main: Artificial Intelligence (cs.AI)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper presents a comprehensive study on improving video caption quality for the Deaf and Hard of Hearing community by leveraging Large Language Models (LLMs). The authors propose a novel pipeline that corrects automatic speech recognition (ASR) generated captions using advanced LLMs, such as GPT-3.5 and Llama2-13B. The methodology focuses on models with robust performance in language comprehension and generation tasks. A dataset representative of real-world challenges is introduced to evaluate the proposed pipeline. The results show that LLM-enhanced captions significantly improve accuracy, achieving a Word Error Rate (WER) of 9.75% compared to the original ASR captions’ WER of 23.07%. This improvement represents an approximate 57.72% reduction in error. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper is about making videos more accessible for people who are Deaf or hard of hearing by using special computer models called Large Language Models (LLMs). These models can help correct mistakes made by automatic speech recognition systems, which currently struggle to provide accurate captions for videos. The authors develop a new way to use these LLMs to improve caption accuracy and test it with real-world data. The results show that this approach works well, reducing errors by a significant amount. |
Keywords
» Artificial intelligence » Gpt
