Loading Now

Summary of Less Is More: Accurate Speech Recognition & Translation Without Web-scale Data, by Krishna C. Puvvada et al.


Less is More: Accurate Speech Recognition & Translation without Web-Scale Data

by Krishna C. Puvvada, Piotr Żelasko, He Huang, Oleksii Hrinchuk, Nithin Rao Koluguri, Kunal Dhawan, Somshubra Majumdar, Elena Rastorgueva, Zhehuai Chen, Vitaly Lavrukhin, Jagadeesh Balam, Boris Ginsburg

First submitted to arxiv on: 28 Jun 2024

Categories

  • Main: Computation and Language (cs.CL)
  • Secondary: Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)

     Abstract of paper      PDF of paper


GrooveSquid.com Paper Summaries

GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!

Summary difficulty Written by Summary
High Paper authors High Difficulty Summary
Read the original abstract here
Medium GrooveSquid.com (original content) Medium Difficulty Summary
The paper presents a multilingual ASR and speech translation model called Canary that outperforms state-of-the-art models Whisper, OWSM, and Seamless-M4T on English, French, Spanish, and German languages using an order of magnitude less data. The key factors enabling this data-efficient approach are: (1) a FastConformer-based attention encoder-decoder architecture, (2) training on synthetic data generated with machine translation, and (3) advanced training techniques such as data-balancing, dynamic data blending, dynamic bucketing, and noise-robust fine-tuning. The model’s weights and training code will be open-sourced.
Low GrooveSquid.com (original content) Low Difficulty Summary
Canary is a new speech recognition and translation model that can recognize languages like English, French, Spanish, and German without needing huge amounts of internet data. This model actually performs better than other top models like Whisper, OWSM, and Seamless-M4T while using much less data. The way it works is by combining three important things: a special attention encoder-decoder architecture, training on fake data created with machine translation, and advanced techniques to balance the data. These innovations make Canary a more efficient model that can be used for many applications.

Keywords

» Artificial intelligence  » Attention  » Encoder decoder  » Fine tuning  » Synthetic data  » Translation