A New Method for Detecting Onset and Offset for Singing in Real-Time and Offline Environments
Source abstract
Abstract
This paper introduces a new method for detecting onsets, offsets, and transitions of the notes in real-time solo singing performances. It identifies the onsets and offsets by finding the transitions from one note to another by considering trajectory changes in the fundamental frequencies. The accuracy of our approach is compared with eight well-known algorithms. It was tested with two datasets that contained 130 files of singing. The total duration of the datasets was more than seven hours and had more than 41,000 onset annotations. The analysis metrics used include the Average, the F-Measure Score, and ANOVA. The proposed algorithm was observed to determine onsets and offsets more accurately than the other algorithms. Additionally, unlike the other algorithms, the proposed algorithm can detect the transitions between notes.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Music and Audio Processing · Neuroscience and Music Perception · Music Technology and Sound Studies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Department of Computer Science, Maynooth University, W23 F2H6 Maynooth, Co. Kildare, Ireland
References (41)
Show source-provided references
1. Bittner, R.M., Pasalo, K., Bosch, J.J., Meseguer-Brocal, G., and Rubinstein, D. (2021, January 7–12). Vocadito: A Dataset of Solo Vocals with F0, Note, and Lyric Annotations. Proceedings of the International Society for Music Information Retrieval, Virtual.
2. Erkomaishvili Dataset: A Curated Corpus of Traditional Georgian Vocal Music for Computational Musicology
3. Choi, S., Kim, W., Park, S., Yong, S., and Nam, J. (2020, January 11–16). Children’s Song Dataset for Singing Voice Research Soonbeom. Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), Virtual.
4. Hoon, H., Dooyong, S., and Kyogu, L. (2013, January 15–19). Note Onset Detection Based on Harmonic Cepstrum Regularity. Proceedings of the 2013 IEEE International Conference on Multimedia and Expo (ICME), San Jose, CA, USA.10.1109/ICME.2013.6607461 ↗
5. Designing Interactive Music Systems with and for People with Dementia
6. Holland, S., Mudd, T., Wilkie-McKenna, K., McPherson, A., and Wanderley, M.M. (2019). A Design Workbench for Interactive Music Systems. New Directions in Music and Human-Computer Interaction, Springer.10.1007/978-3-319-92069-6 ↗
7. Modelling Melodic Variation and Extracting Melodic Templates from Flamenco Singing Performances10.1080/17459737.2019.1610194 ↗
8. A Tutorial on Onset Detection in Music Signals10.1109/TSA.2005.851998 ↗
9. Collins, N. (2005, January 28–31). A Comparison of Sound Onset Detection Algorithms with Emphasis on Psychoacoustically Motivated Detection Functions. Proceedings of the Audio Engineering Society Convention 118, Barcelona, Spain.
10. Onset Detection Revisited
11. A Supervised Classification Algorithm for Note Onset Detection10.1155/2007/43745 ↗
12. Eyben, F., Böck, S., Schuller, B., and Graves, A. (2010, January 9–13). Universal Onset Detection with Bidirectional Long-Short Term Memory Neural Networks. Proceedings of the 11th International Society for Music Information Retrieval Conference, ISMIR, Utrecht, The Netherlands.
13. Onset Event Decoding Exploiting the Rhythmic Structure of Polyphonic Music10.1109/JSTSP.2011.2146229 ↗
14. Schluter, J., and Bock, S. (2014, January 4–9). Improved Musical Onset Detection with Convolutional Neural Networks. Proceedings of the 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Florence, Italy.10.1109/ICASSP.2014.6854953 ↗
15. Toh, C.C., Zhang, B., and Wang, Y. (2008, January 14–18). Multiple-Feature Fusion Based Onset Detection for Solo Singing Voice. Proceedings of the ISMIR 2008—International Conference on Music Information Retrieval, Philadelphia, PA, USA.
16. Powers, D.M.W. (2020). Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness and Correlation. arXiv.
17. Gong, R., and Serra, X. (2018). Towards an Efficient Deep Learning Model for Musical Onset Detection. arXiv.
18. Chang, S., and Lee, K. (2014, January 4–9). A Pairwise Approach to Simultaneous Onset/Offset Detection for Singing Voice Using Correntropy. Proceedings of the 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Florence, Italy.10.1109/ICASSP.2014.6853672 ↗
19. Lindblom, B., and Sundberg, J. (2007). The Human Voice in Speech and Singing. Springer Handbook of Acoustics, Springer.10.1007/978-0-387-30425-0_16 ↗
20. Schindler, A., Lidy, T., and Böck, S. (2020). Deep Learning for MIR Tutorial. arXiv.
21. Mayor, O., Bonada, J., and Loscos, A. (2006, January 5–8). The Singing Tutor: Expression Categorization and Segmentation of the Singing Voice. Proceedings of the AES 121st Convention, San Francisco, CA, USA.
22. McFee, B., Lostanlen, V., Metsai, A., McVicar, M., Balke, S., Thomé, C., Raffel, C., Zalkow, F., Malek, A. (2021, June 01). Librosa/Librosa: 0.8.0. Available online: https://librosa.org/doc/latest/index.html.
23. McFee, B., Raffel, C., Liang, D., Ellis, D., McVicar, M., Battenberg, E., and Nieto, O. (2015, January 6–12). Librosa: Audio and Music Signal Analysis in Python. Proceedings of the 14th Python in Science Conference (SciPy 2015), Austin, TX, USA.10.25080/Majora-7b98e3ed-003 ↗
24. Böck, S., Korzeniowski, F., Schlüter, J., Krebs, F., and Widmer, G. (2016, January 15–19). Madmom: A New Python Audio and Music Signal Processing Library. Proceedings of the Proceedings of the 24th ACM International Conference on Multimedia, Amsterdam, The Netherlands.10.1145/2964284.2973795 ↗
25. (2021, June 01). Available online: https://aubio.org/.
26. Bogdanov, D., Wack, N., Gómez, E., Gulati, S., Herrera, P., Mayor, O., Roma, G., Salamon, J., Zapata, J., and Serra, X. (2013, January 4–8). Essentia: An Audio Analysis Library for Music Information Retrieval. Proceedings of the Proceedings of the 14th International Society for Music Information Retrieval Conference, Curitiba, Brazil. ISMIR 2013.10.1145/2502081.2502229 ↗
27. Sebastian, B., Krebs, F., and Schedl, M. (2012, January 8–12). Evaluating the Online Capabilities of Onset Detection Methods. Proceedings of the 13th International Society for Music Information Retrieval Conference (ISMIR), Porto, Portugal.
28. Jehan, T. (2005). Creating Music by Listening. [Ph.D. Thesis, Media Arts and Sciences Department, Massachusetts Institute of Technology].
29. Böck, S., Arzt, A., Krebs, F., and Schedl, M. (2012, January 17–21). Online Real-Time Onset Detection with Recurrent Neural Networks. Proceedings of the 15th International Conference on Digital Audio Effects (DAFx-12), York, UK.
30. Brossier, P.M. (2005). Fast Onset Detection Using Aubio (Brossier), Mirex 2005, Citeseer.
31. MasJri, P., and Bateman, A. (1996, January 19–24). Improved Modelling of Attack Transients in Music Analysis-Resynthesis. Proceedings of the 1996 International Computer Music Conference, ICMC 1996, Hong Kong, China.
32. On the Use of Phase and Energy for Musical Onset Detection in the Complex Domain10.1109/LSP.2004.827951 ↗
33. Raffel, C., McFee, B., Humphrey, E.J., Salamon, J., Nieto, O., Liang, D., and Ellis, D.P.W. (2014, January 27–31). Mir_eval: A Transparent Implementation of Common MIR Metrics. Proceedings of the Proceedings of the 15th International Society for Music Information Retrieval Conference, Taipei, Taiwan. ISMIR 2014.
34. An Algorithm for Determining the Endpoints of Isolated Utterances10.1002/j.1538-7305.1975.tb02840.x ↗
35. Faghih, B., and Timoney, J. (2019, January 17–18). An Investigation into Several Pitch Detection Algorithms for Singing Phrases Analysis. Proceedings of the 2019 30th Irish Signals and Systems Conference (ISSC), Maynooth, Ireland.10.1109/ISSC.2019.8904943 ↗
36. Faghih, B., and Timoney, J. (2022). Smart-Median: A New Real-Time Algorithm for Smoothing Singing Pitch Contours. Appl. Sci., 12.10.3390/app12147026 ↗
37. Faghih, B., and Timoney, J. (2022). Real-Time Monophonic Singing Pitch Detection. Preprint.
38. Mauch, M., and Dixon, S. (2014, January 4–9). PYIN: A Fundamental Frequency Estimator Using Probabilistic Threshold Distributions. Proceedings of the 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Florence, Italy.10.1109/ICASSP.2014.6853678 ↗
39. Normative Voice Range Profiles of Male and Female Professional Voice Users10.1016/S0892-1997(02)00065-6 ↗
40. Real-Time Music Following in Score Sheet Images via Multi-Resolution Prediction10.3389/fcomp.2021.718340 ↗
41. Wilkins, J., Seetharaman, P., Wahl, A., and Pardo, B. (2018, January 23–27). VocalSet: A Singing Voice Dataset. Proceedings of the 19th International Society for Music Information Retrieval Conference, Paris, France.