Summary of Enhancing Bidirectional Sign Language Communication: Integrating Yolov8 and Nlp For Real-time Gesture Recognition & Translation, by Hasnat Jamil Bhuiyan et al.
Enhancing Bidirectional Sign Language Communication: Integrating YOLOv8 and NLP for Real-Time Gesture Recognition & Translation
by Hasnat Jamil Bhuiyan, Mubtasim Fuad Mozumder, Md. Rabiul Islam Khan, Md. Sabbir Ahmed, Nabuat Zaman Nahim
First submitted to arxiv on: 18 Nov 2024
Categories
- Main: Computer Vision and Pattern Recognition (cs.CV)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper explores the development of a system that can convert American Sign Language (ASL) data from real-time camera footage into text. The authors also propose a framework for converting text into ASL in real-time, aiming to break language barriers for individuals with hearing impairments. To recognize ASL, the researchers employ the You Only Look Once (YOLO) model and Convolutional Neural Network (CNN) model. The YOLO model extracts spatial-temporal characteristics from raw video streams without prior knowledge, while the CNN model is used for sign language detection. A novel method is introduced for converting text-based input into ASL by identifying keywords from a given sentence and displaying a real-time video of sign language performance. This study demonstrates bidirectional ASL communication in real-time, making it a rare contribution to the field. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research develops a system that can translate American Sign Language (ASL) into text, and also convert text into ASL in real time. The goal is to help people who are deaf or hard of hearing communicate more easily. To make this happen, the researchers use special computer models called YOLO and CNN. These models can look at video footage and understand what’s being signed. The system works by identifying important words from a sentence and then showing a video of someone signing those words in real time. |
Keywords
» Artificial intelligence » Cnn » Neural network » Yolo
