Summary of Acoustic Feature Mixup For Balanced Multi-aspect Pronunciation Assessment, by Heejin Do et al.
Acoustic Feature Mixup for Balanced Multi-aspect Pronunciation Assessment
by Heejin Do, Wonjun Lee, Gary Geunbae Lee
First submitted to arxiv on: 22 Jun 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Sound (cs.SD); Audio and Speech Processing (eess.AS)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed Acoustic Feature Mixup strategies aim to address data scarcity and score-label imbalances in automated pronunciation assessment for non-native language learners’ speech. The approach involves linearly and non-linearly interpolating acoustic features with the in-batch averaged feature, leveraging goodness-of-pronunciation as a primary acoustic feature. Additionally, fine-grained error-rate features are integrated by comparing speech recognition results with original answer phonemes, providing direct hints for mispronunciation. Experimental results on the speechocean762 dataset demonstrate enhanced scoring performances and highlight potential to predict unseen distortions. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Automated pronunciation assessment is important for non-native language learners’ speech. The challenge is acquiring data to provide enriched feedback on multiple aspects of pronunciation. Researchers have proposed a solution called Acoustic Feature Mixup, which combines different features to create more training data. This approach also helps with imbalanced score distributions. In this paper, the authors use goodness-of-pronunciation and fine-grained error-rate features to improve scoring performances. The results show that this method can be effective in predicting distortions. |
