FrameAugment: A Simple Data Augmentation Method for Encoder–Decoder Speech Recognition
Source abstract
Abstract
As the architecture of deep learning-based speech recognizers has recently changed to the end-to-end style, increasing the effective amount of training data has become an important issue. To tackle this issue, various data augmentation techniques to create additional training data by transforming labeled data have been studied. We propose a method called FrameAugment to augment data by changing the speed of speech locally for selected sections, which is different from the conventional speed perturbation technique that changes the speed of speech uniformly for the entire utterance. To change the speed of the selected sections of speech, the number of frames for the randomly selected sections is adjusted through linear interpolation in the spectrogram domain. The proposed method is shown to achieve 6.8% better performance than the baseline in the WSJ database and 9.5% better than the baseline in the LibriSpeech database. It is also confirmed that the proposed method further improves speech recognition performance when it is combined with the previous data augmentation techniques.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Speech Recognition and Synthesis · Speech and Audio Processing · Music and Audio Processing
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Major in Control and Robot Engineering, Chungbuk National University, Cheongju 28644, Korea
Department of Intelligent Systems and Robotics, Chungbuk National University, Cheongju 28644, Korea
References (26)
Show source-provided references
1. Deep learning10.1038/nature14539 ↗
2. Unsupervised data augmentation for consistency training
3. Xie, Q., Luong, M.T., Hovy, E., and Le, Q.V. (2020, January 13–19). Self-training with Noisy Student Improves ImageNet Classification. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPR42600.2020.01070 ↗
4. Pham, H., Dai, Z., Xie, Q., and Le, Q.V. (2021, January 19–25). Meta Pseudo Labels. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Virtual.10.1109/CVPR46437.2021.01139 ↗
5. FixMatch: Simplifying semi-supervised learning with consistency and confidence
6. Zhong, Z., Zheng, L., Kang, G., Li, S., and Yang, Y. (2020, January 7–12). Random erasing data augmentation. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.
7. Cubuk, E.D., Zoph, B., Shlens, J., and Le, Q.V. (2020, January 13–19). RandAugment: Practical automated data augmentation with a reduced search space. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.10.1109/CVPRW50498.2020.00359 ↗
8. Yu, D., and Deng, L. (2016). Automatic Speech Recognition—A Deep Learning Approach, Springer.10.1007/978-1-4471-5779-3 ↗
9. Ko, T., Peddinti, V., Povey, D., and Khudanpur, S. (2015, January 6–10). Audio augmentation for speech recognition. Proceedings of the INTERSPEECH 2015, Dresden, Germany.10.21437/Interspeech.2015-711 ↗
10. Park, D.S., Chan, W., Zhang, Y., Chiu, C.C., Zoph, B., Cubuk, E.D., and Le, Q.V. (2019). SpecAugment: A simple data augmentation method for automatic speech recognition. arXiv.10.21437/Interspeech.2019-2680 ↗
11. Song, X., Wu, Z., Huang, Y., Su, D., and Meng, H. (2020, January 25–29). SpecSwap: A Simple Data Augmentation Method for End-to-End Speech Recognition. Proceedings of the INTERSPEECH 2020, Shanghai, China.10.21437/Interspeech.2020-2275 ↗
12. Paul, D.B., and Janet, B. (1992, January 23–26). The design for the Wall Street Journal-based CSR corpus. Proceedings of the Speech and Natural Language, New York, NY, USA.10.3115/1075527.1075614 ↗
13. Panayotov, V., Chen, G., Povey, D., and Khudanpur, S. (2015, January 19–24). LibriSpeech: An ASR corpus based on public domain audio books. Proceedings of the ICASSP, South Brisbane, QLD, Australia.10.1109/ICASSP.2015.7178964 ↗
14. Chan, W., Jaitly, N., Le, Q.V., and Vinyals, O. (2015). Listen, attend and spell. arXiv.
15. Graves, A., Fernández, S., Gomez, F., and Schmidhuber, J. (2006, January 25–29). Connectionist temporal classification: Labelling unsegmented sequence data with recurrent neural networks. Proceedings of the 23rd International Conference on Machine Learning (ICML), Pittsburgh, PA, USA.10.1145/1143844.1143891 ↗
16. Attention is all you need
17. Long short-term memory10.1162/neco.1997.9.8.1735 ↗
18. Multitask learning10.1023/A:1007379606734 ↗
19. Kim, S., Hori, T., and Watanabe, S. (2017, January 5–9). Joint CTC-attention based end-to-end speech recognition using multi-task learning. Proceedings of the 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LA, USA.10.1109/ICASSP.2017.7953075 ↗
20. (2022, July 17). Sox-Sound eXchange. Available online: http://sox.sourceforge.net/.
21. Steffensen, J.F. (2006). Interpolation, Courier Corporation.
22. Kudo., T., and Richardson, J. (November, January 31). SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Brussels, Belgium.10.18653/v1/D18-2012 ↗
23. Kudo, T. (2018, January 15–20). Subword regularization: Improving neural network translation models with multiple subword candidates. Proceedings of the Association for Computational Linguistics, Melbourne, VIC, Australia.10.18653/v1/P18-1007 ↗
24. Watanabe, S., Hori, T., Karita, S., Hayashi, T., Nishitoba, J., Unno, Y., Soplin, N.E.Y., Heymann, J., Wiesner, M., and Chen, N. (2018). ESPnet: End-to-End Speech Processing Toolkit. arXiv.10.21437/Interspeech.2018-1456 ↗
25. Dropout: A Simple Way to Prevent Neural Networks from Overfitting
26. When does label smoothing help?