State-of-the-Art Model for Music Object Recognition with Deep Learning
Source abstract
Abstract
Optical music recognition (OMR) is an area in music information retrieval. Music object detection is a key part of the OMR pipeline. Notes are used to record pitch and duration and have semantic information. Therefore, note recognition is the core and key aspect of music score recognition. This paper proposes an end-to-end detection model based on a deep convolutional neural network and feature fusion. This model is able to directly process the entire image and then output the symbol categories and the pitch and duration of notes. We show a state-of-the-art recognition model for general music symbols which can get 0.92 duration accurary and 0.96 pitch accuracy .
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Music and Audio Processing · Diverse Musicological Studies · Digital Media Forensic Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
References (28)
Show source-provided references
1. Content-Based Music Information Retrieval: Current Directions and Future Challenges10.1109/JPROC.2008.916370 ↗
2. Roland, P. (2002, January 19–20). The Music Encoding Initiative (MEI). Proceedings of the First International Conference on Musical Applications Using XML, Milan, Italy.
3. Good, M., and Actor, G. (2003, January 15–17). Using MusicXML for file interchange. Proceedings of the Third International Conference on WEB Delivering of Music, Leeds, UK.
4. The challenge of optical music recognition10.1023/A:1002485918032 ↗
5. Optical music recognition: State-of-the-art and open issues10.1007/s13735-012-0004-6 ↗
6. Deep learning10.1038/nature14539 ↗
7. Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2015). You Only Look Once: Unified, Real-Time Object Detection. arXiv.10.1109/CVPR.2016.91 ↗
8. Redmon, J., and Farhadi, A. (2016). YOLO9000: Better, Faster, Stronger. arXiv.10.1109/CVPR.2017.690 ↗
9. Redmon, J., and Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv.
10. Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016). SSD: Single Shot MultiBox Detector. European Conference on Computer Vision, Springer.10.1007/978-3-319-46448-0_2 ↗
11. Lin, T.Y., Goyal, P., Girshick, R., He, K., and Dollár, P. (2017). Focal Loss for Dense Object Detection. arXiv.10.1109/ICCV.2017.324 ↗
12. Girshick, R. (2015). Fast R-CNN. arXiv.10.1109/ICCV.2015.169 ↗
13. Ren, S., He, K., Girshick, R., and Sun, J. (2015). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. arXiv.
14. Dai, J., Li, Y., He, K., and Sun, J. (2016). R-FCN: Object Detection via Region-based Fully Convolutional Networks. arXiv.
15. Pacha, A., Hajič, J., and Calvo-Zaragoza, J. (2018). A Baseline for General Music Object Detection with Deep Learning. Appl. Sci., 8.10.3390/app8091488 ↗
16. Ronneberger, O., Fischer, P., and Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. arXiv.10.1007/978-3-319-24574-4_28 ↗
17. Tuggener, L., Elezi, I., Schmidhuber, J., Pelillo, M., and Stadelmann, T. (2018). DeepScores—A Dataset for Segmentation, Detection and Classification of Tiny Objects. arXiv.10.1109/ICPR.2018.8545307 ↗
18. Hajič, J., and Pecina, P. (2017, January 10). The MUSCIMA++ Dataset for Handwritten Optical Music Recognition. Proceedings of the 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), Kyoto, Japan.10.1109/ICDAR.2017.16 ↗
19. Hajič, J., Dorfer, M., Widmer, G., and Pecina, P. (2018, January 23–27). Towards Full-Pipeline Handwritten OMR with Musical Symbol Detection by U-Nets. Proceedings of the 19th International Society for Music Information Retrieval Conference, Paris, France.
20. Tuggener, L., Elezi, I., Schmidhuber, J., and Stadelmann, T. (2018). Deep Watershed Detector for Music Object Recognition. arXiv.
21. He, K., Zhang, X., Ren, S., and Sun, J. (2015). Deep Residual Learning for Image Recognition. arXiv.10.1109/CVPR.2016.90 ↗
22. Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014). Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. arXiv.10.3115/v1/D14-1179 ↗
23. Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., and Weinberger, K.Q. (2014). Sequence to Sequence Learning with Neural Networks. Advances in Neural Information Processing Systems 27, Curran Associates, Inc.
24. van der Wel, E., and Ullrich, K. (2017). Optical Music Recognition with Convolutional Sequence-to-Sequence Models. arXiv.
25. Calvo-Zaragoza, J., Valero-Mas, J.J., and Pertusa, A. (2017, January 472–477). END-TO-END OPTICAL MUSIC RECOGNITION USING NEURAL NETWORKS. Proceedings of the 18th International Society for Music Information Retrieval Conference, Suzhou, China.
26. Calvo-Zaragoza, J., and Rizo, D. (2018). End-to-End Neural Optical Music Recognition of Monophonic Scores. Appl. Sci., 8.10.3390/app8040606 ↗
27. From Optical Music Recognition to Handwritten Music Recognition: A baseline10.1016/j.patrec.2019.02.029 ↗
28. Kingma, D.P., and Ba, J. (2014). Adam: A Method for Stochastic Optimization. arXiv.