A Frequency-Structured Dilated Conformer Architecture for Full-Kit Automatic Drum Transcription
Source abstract
Abstract
Automatic Drum Transcription (ADT) remains challenging due to overlapping spectral content among drum instruments, dense transient activity, and the increased difficulty associated with fine-grained full-kit transcription. This study investigates whether combining explicit spectral structuring with temporally extended modeling improves ADT performance under a controlled evaluation protocol. A frequency-structured architecture is proposed that integrates a convolutional front end, a Frequency-Structured Soft-Gated Module (FSSM) for coarse frequency-aware processing, and a Dilated Conformer encoder for multi-scale temporal context modeling while preserving frame-level output alignment. Experiments are conducted on the ENST Drums dataset using both 3-class and consolidated 8-class transcription settings. Under a fixed ±50 ms event-level evaluation protocol, the proposed model achieved mean Micro-F1 scores of 0.920 ± 0.001 and 0.872 ± 0.002 in the 3-class and 8-class settings, respectively, outperforming a reproduced CRNN baseline trained and evaluated under identical preprocessing, training, and evaluation protocols. Ablation studies indicate that FSSM contributes the largest performance improvement among the evaluated architectural components, while class-wise analyses show improvements across all instrument categories in the 8-class setting. Zero-shot evaluation on external datasets further demonstrates that the proposed architecture maintains higher performance than the baseline under distribution shift, although substantial degradation remains relative to in-domain evaluation. These findings suggest that combining explicit spectral structuring and temporally extended modeling can improve full-kit drum transcription performance under the evaluated conditions while highlighting the continuing challenges of cross-dataset generalization.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Music and Audio Processing · Music Technology and Sound Studies · Speech Recognition and Synthesis
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Division of Computer Science and Engineering, Jeonbuk National University, Jeonju 54896, Republic of Korea
Department of Computer Science and Applications, Tribhuvan University, Mechi Multiple Campus, Bhadrapur 57200, Nepal
Funding
National Research Foundation of Korea
References (28)
Show source-provided references
1. A Review of Automatic Drum Transcription10.1109/TASLP.2018.2830113 ↗
2. Bhattarai, B., and Lee, J. (2023). A Comprehensive Review on Music Transcription. Appl. Sci., 13.10.3390/app132111882 ↗
3. Weyers, P., Uhle, C., Müller, M., and Lang, M. (2025, January 21–25). Understanding Performance Limitations in Automatic Drum Transcription. Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), Daejeon, Republic of Korea.
4. High-Quality and Reproducible Automatic Drum Transcription from Crowdsourced Data10.3390/signals4040042 ↗
5. Jacques, C., and Roebel, A. (2018, January 4–8). Automatic Drum Transcription with Convolutional Neural Networks. Proceedings of the 21st International Conference on Digital Audio Effects (DAFx), Aveiro, Portugal.
6. Vogl, R., Dorfer, M., Widmer, G., and Knees, P. (2017, January 23–27). Drum Transcription via Joint Beat and Drum Modeling Using Convolutional Recurrent Neural Networks. Proceedings of the 18th International Society for Music Information Retrieval Conference (ISMIR), Suzhou, China.
7. Vogl, R., Widmer, G., and Knees, P. (2018, January 4–8). Towards Multi-Instrument Drum Transcription. Proceedings of the 21st International Conference on Digital Audio Effects (DAFx), Aveiro, Portugal.
8. Jacques, C., and Roebel, A. (2019, January 2–6). Data Augmentation for Drum Transcription with Convolutional Neural Networks. Proceedings of the 27th European Signal Processing Conference (EUSIPCO), A Coruña, Spain.10.23919/EUSIPCO.2019.8902980 ↗
9. Southall, C., Stables, R., and Hockman, J. (2017, January 23–27). Automatic Drum Transcription for Polyphonic Recordings Using Soft Attention Mechanisms and Convolutional Neural Networks. Proceedings of the 18th International Society for Music Information Retrieval Conference (ISMIR), Suzhou, China.
10. Global Structure-Aware Drum Transcription Based on Self-Attention Mechanisms10.3390/signals2030031 ↗
11. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention Is All You Need. Proceedings of the Advances in Neural Information Processing Systems 30 (NeurIPS 2017), Long Beach, CA, USA.
12. Gulati, A., Qin, J., Chiu, C.-C., Parmar, N., Zhang, Y., Yu, J., Han, W., Wang, S., Wu, Z., and Pang, R. (2020, January 25–29). Conformer: Convolution-Augmented Transformer for Speech Recognition. Proceedings of the INTERSPEECH, Shanghai, China.10.21437/Interspeech.2020-3015 ↗
13. Fitzgerald, D., Coyle, E., and Lawlor, B. (2002, January 26–28). Sub-Band Independent Subspace Analysis for Drum Transcription. Proceedings of the 5th International Conference on Digital Audio Effects (DAFx-02), Hamburg, Germany.
14. Music Source Separation with Band-Split RNN10.1109/TASLP.2023.3271145 ↗
15. Lu, W.-T., Wang, J.-C., Kong, Q., and Hung, Y.-N. (2024, January 14–19). Music Source Separation with Band-Split RoPE Transformer. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Seoul, Republic of Korea.10.1109/ICASSP48485.2024.10446843 ↗
16. Wang, J.-C., Lu, W.-T., and Chen, J. (2024, January 10–14). Mel-RoFormer for Vocal Separation and Vocal Melody Transcription. Proceedings of the 25th International Society for Music Information Retrieval Conference (ISMIR), San Francisco, CA, USA.
17. Gillet, O., and Richard, G. (2006, January 8–12). ENST-Drums: An Extensive Audio-Visual Database for Drum Signals Processing. Proceedings of the 7th International Society for Music Information Retrieval Conference (ISMIR), Victoria, BC, Canada.
18. Improving Time-Scale Modification of Music Signals Using Harmonic–Percussive Separation10.1109/LSP.2013.2294023 ↗
19. Wu, C.-W., and Lerch, A. (September, January 31). Drum Transcription Using Partially Fixed Non-Negative Matrix Factorization. Proceedings of the 23rd European Signal Processing Conference (EUSIPCO), Nice, France.
20. Automatic Drum Transcription with Label Augmentation Using Convolutional Neural Networks10.1007/978-3-030-92185-9_6 ↗
21. Zehren, M., Alunno, M., and Bientinesi, P. (2021, January 7–12). ADTOF: A Large Dataset of Non-Synthetic Music for Automatic Drum Transcription. Proceedings of the 22nd International Society for Music Information Retrieval Conference (ISMIR), Online.
22. Gillick, J., Roberts, A., Engel, J., Eck, D., and Bamman, D. (2019, January 9–15). Learning to Groove with Inverse Sequence Transformations. Proceedings of the 36th International Conference on Machine Learning (ICML), Long Beach, CA, USA.
23. Grais, E.M., Zhao, F., and Plumbley, M.D. (2020, January 24–28). Multi-Band Multi-Resolution Fully Convolutional Neural Networks for Singing Voice Separation. Proceedings of the 28th European Signal Processing Conference (EUSIPCO), Amsterdam, The Netherlands.
24. Southall, C., Wu, C.-W., Lerch, A., and Hockman, J. (2017, January 23–27). MDB Drums: An Annotated Subset of MedleyDB for Automatic Drum Transcription. Proceedings of the Extended Abstracts of the 18th International Society for Music Information Retrieval Conference (ISMIR), Suzhou, China.
25. Manilow, E., Wichern, G., Seetharaman, P., and Le Roux, J. (2019, January 20–23). Cutting Music Source Separation Some Slakh: A Dataset to Study the Impact of Training Data Quality and Quantity. Proceedings of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), New Paltz, NY, USA.10.1109/WASPAA.2019.8937170 ↗
26. Ridnik, T., Ben-Baruch, E., Zamir, N., Noy, A., Friedman, I., Protter, M., and Zelnik-Manor, L. (2021, January 11–17). Asymmetric Loss for Multi-Label Classification. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.10.1109/ICCV48922.2021.00015 ↗
27. Riley, X., and Dixon, S. (2024, January 10–14). Enhanced Automatic Drum Transcription via Drum Stem Source Separation. Proceedings of the Extended Abstracts of the 25th International Society for Music Information Retrieval Conference (ISMIR), San Francisco, CA, USA.
28. Loshchilov, I., and Hutter, F. (2019, January 6–9). Decoupled Weight Decay Regularization. Proceedings of the International Conference on Learning Representations (ICLR), New Orleans, LA, USA.