Summary of Less Is More: Accurate Speech Recognition & Translation Without Web-scale Data, by Krishna C. Puvvada et al.
Less is More: Accurate Speech Recognition & Translation without Web-Scale Data
by Krishna C. Puvvada, Piotr Żelasko, He Huang, Oleksii Hrinchuk, Nithin Rao Koluguri, Kunal Dhawan, Somshubra Majumdar, Elena Rastorgueva, Zhehuai Chen, Vitaly Lavrukhin, Jagadeesh Balam, Boris Ginsburg
First submitted to arxiv on: 28 Jun 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Machine Learning (cs.LG); Sound (cs.SD); Audio and Speech Processing (eess.AS)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper presents a multilingual ASR and speech translation model called Canary that outperforms state-of-the-art models Whisper, OWSM, and Seamless-M4T on English, French, Spanish, and German languages using an order of magnitude less data. The key factors enabling this data-efficient approach are: (1) a FastConformer-based attention encoder-decoder architecture, (2) training on synthetic data generated with machine translation, and (3) advanced training techniques such as data-balancing, dynamic data blending, dynamic bucketing, and noise-robust fine-tuning. The model’s weights and training code will be open-sourced. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Canary is a new speech recognition and translation model that can recognize languages like English, French, Spanish, and German without needing huge amounts of internet data. This model actually performs better than other top models like Whisper, OWSM, and Seamless-M4T while using much less data. The way it works is by combining three important things: a special attention encoder-decoder architecture, training on fake data created with machine translation, and advanced techniques to balance the data. These innovations make Canary a more efficient model that can be used for many applications. |
Keywords
» Artificial intelligence » Attention » Encoder decoder » Fine tuning » Synthetic data » Translation
