Rhythm-Based Attention Analysis: A Comprehensive Model for Music Hierarchy
Source abstract
Abstract
Deciphering the structural hierarchy of musical compositions is indispensable for a range of music analysis applications, encompassing feature extraction, data compression, interpretation, and visualization. In this paper, we introduce a quantitative model grounded in fractal theory to evaluate the significance of individual notes within a musical piece. To analyze the quantized note importance, we adopt a rhythm-based approach and propose a series of detection operators informed by fundamental rhythmic combinations. Employing the Mamba model, we carry out recursive detection operations that offer a hierarchic understanding of musical structures. By organizing the composition into a tree data structure, we achieve an ordered layer traversal that highlights the music piece’s multi-dimensional features. Musical compositions often exhibit intrinsic symmetry in their temporal organization, manifested through repetition, variation, and self-similar patterns across scales. Among these symmetry properties, fractality stands out as a prominent characteristic, reflecting recursive structures both rhythmically and melodically. Our model effectively captures this property, providing insights into the fractal-like regularities within music. It also proves effective in musical phrase boundary detection tasks, enhancing the clarity and visualization of musical information. The findings illustrate the model’s potential to advance the quantitative analysis of music hierarchy, promoting novel methodologies in musicological research.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Neuroscience and Music Perception · Music and Audio Processing · Vehicle Noise and Vibration Control
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China
Institute of Artificial Intelligence, Shanghai University, Shanghai 200444, China
References (50)
Show source-provided references
1. Music information retrieval10.1002/aris.1440370108 ↗
2. The scientific evaluation of music information retrieval systems: Foundations and future10.1162/014892604323112211 ↗
3. Music retrieval: A tutorial and review10.1561/1500000002 ↗
4. Content-based music information retrieval: Current directions and future challenges10.1109/JPROC.2008.916370 ↗
5. Kirthika, P., and Chattamvelli, R. (2012, January 19–21). A review of raga based music classification and music information retrieval (MIR). Proceedings of the 2012 IEEE International Conference on Engineering Education: Innovative Practices and Future Trends (AICERA), Kottayam, India.10.1109/AICERA.2012.6306752 ↗
6. Contextual music information retrieval and recommendation: State of the art and challenges10.1016/j.cosrev.2012.04.002 ↗
7. McKay, C., Cumming, J., and Fujinaga, I. (2018, January 7–16). JSYMBOLIC 2.2: Extracting Features from Symbolic Music for use in Musicological and MIR Research. Proceedings of the ISMIR, Izmir, Turkey.
8. Temperley, D. (2004). The Cognition of Basic Musical Structures, MIT Press.
9. Pearce, M.T. (2005). The Construction and Evaluation of Statistical Models of Melodic Structure in Music Perception and Composition. [Ph.D. Thesis, City University London].
10. An overview of hierarchical structure in music10.2307/40285257 ↗
11. Processing of hierarchical syntactic structure in music10.1073/pnas.1300272110 ↗
12. Zhong, Z., Hirano, M., Shimada, K., Tateishi, K., Takahashi, S., and Mitsufuji, Y. (2023, January 4–10). An attention-based approach to hierarchical multi-label music instrument classification. Proceedings of the ICASSP 2023—2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Rhodes Island, Greece.10.1109/ICASSP49357.2023.10095162 ↗
13. Bhandari, K., and Colton, S. (2024, January 3–5). Motifs, Phrases, and Beyond: The Modelling of Structure in Symbolic Music Generation. Proceedings of the International Conference on Computational Intelligence in Music, Sound, Art and Design (Part of EvoStar), Aberystwyth, UK.10.1007/978-3-031-56992-0_3 ↗
14. Pesek, M., Leonardis, A., and Marolt, M. (2014, January 27–31). A Compositional Hierarchical Model for Music Information Retrieval. Proceedings of the ISMIR, Taipei, China.
15. Content-based music information retrieval (cb-mir) and its applications toward the music industry: A review10.1145/3177849 ↗
16. The Interconnections of Music Structure, Harmony, Melody, Rhythm, and Predictivity10.1177/20592043241234758 ↗
17. Buisson, M., Mcfee, B., Essid, S., and Crayencour, H.C. (2022, January 4–8). Learning multi-level representations for hierarchical music structure analysis. Proceedings of the International Society for Music Information Retrieval (ISMIR), Bengaluru, India.
18. Hierarchical multi-head attention LSTM for polyphonic symbolic melody generation10.1007/s11042-024-18491-7 ↗
19. Cooper, M., and Foote, J. (2003, January 19–22). Summarizing popular music via structural similarity analysis. Proceedings of the 2003 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (IEEE Cat. No. 03TH8684), New Paltz, NY, USA.
20. Maddage, N.C., Xu, C., Kankanhalli, M.S., and Shao, X. (2004, January 10–16). Content-based music structure analysis with applications to music semantics understanding. Proceedings of the 12th Annual ACM International Conference on Multimedia, New York, NY, USA.10.1145/1027527.1027549 ↗
21. Shiu, Y., Jeong, H., and Kuo, C.C.J. (2006, January 23–27). Similarity matrix processing for music structure analysis. Proceedings of the 1st ACM Workshop on Audio and Music Computing Multimedia, Santa Barbara, CA, USA.10.1145/1178723.1178734 ↗
22. Music structure analysis using a probabilistic fitness measure and a greedy search algorithm10.1109/TASL.2009.2020533 ↗
23. Peeters, G. (2003, January 26–27). Deriving musical structures from signal analysis for music audio summary generation: “Sequence” and “state” approach. Proceedings of the International Symposium on Computer Music Modeling and Retrieval, Montpellier, France.10.1007/978-3-540-39900-1_14 ↗
24. Wu, H.H., and Bello, J.P. (2010, January 21–24). Audio-based music visualization for music structure analysis. Proceedings of the Sound and Music Computing Conference (SMC), Barcelona, Spain.
25. Audio-based music structure analysis: Current trends, open challenges, and applications
26. Mandelbrot, B.B., and Aizenman, M. (1979). Fractals: Form, Chance, and Dimension, W. H. Freeman and Co.10.1063/1.2995555 ↗
27. Towards a mathematical foundation for music theory and composition: A theory of structure10.1080/17459737.2024.2379788 ↗
28. MusicAOG: An Energy-Based Model for Learning and Sampling a Hierarchical Representation of Symbolic Music10.1109/TCSS.2024.3521445 ↗
29. Rhythm implicitly affects temporal orienting of attention across modalities10.1016/j.actpsy.2012.11.012 ↗
30. Musical meter modulates the allocation of attention across time10.1162/jocn_a_00862 ↗
31. Elbaz, A., and Yeshurun, Y. (2020). Can rhythm-induced attention improve the perceptual representation?. PLoS ONE, 15.10.1371/journal.pone.0231200 ↗
32. Fractal dimension and classification of music10.1016/S0960-0779(99)00137-X ↗
33. Music walk, fractal geometry in music10.1016/j.physa.2007.02.079 ↗
34. Musical rhythm spectra from Bach to Joplin obey a 1/f power law10.1073/pnas.1113828109 ↗
35. Fractal patterns in music10.1016/j.chaos.2023.113315 ↗
36. Exact deconvolution for multiple convolution operators-an overview, plus performance characterizations for imaging sensors10.1109/5.54810 ↗
37. Istvanek, M., Smekal, Z., Spurny, L., and Mekyska, J. (2020). Enhancement of conventional beat tracking system using Teager–Kaiser energy operator. Appl. Sci., 10.10.3390/app10010379 ↗
38. Cooper, G.W., Cooper, G., and Meyer, L.B. (1963). The Rhythmic Structure of Music, University of Chicago Press.
39. 1/f noise10.1109/PROC.1982.12282 ↗
40. Mandelbrot, B.B., and Mandelbrot, B.B. (1997). The Variation of Certain Speculative Prices, Springer.10.1007/978-1-4757-2763-0_14 ↗
41. Multifractal detrended fluctuation analysis of nonstationary time series10.1016/S0378-4371(02)01383-3 ↗
42. Gu, A., and Dao, T. (2023). Mamba: Linear-Time Sequence Modeling with Selective State Spaces. arXiv.
43. Dao, T., and Gu, A. (2024, January 21–27). Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality. Proceedings of the International Conference on Machine Learning (ICML), Vienna, Austria.
44. Schaffrath, H. (2025, May 27). The Essen Folksong Collection in Kern Format. Computer Database. Available online: https://github.com/ccarh/essen-folksong-collection.
45. Cenkerová, Z., Hartmann, M., and Toiviainen, P. (2018, January 4–7). Crossing phrase boundaries in music. Proceedings of the Sound and Music Computing Conferences, Limassol, Cyprus.
46. van Kranenburg, P. (2020, January 11–16). Rule mining for local boundary detection in melodies. Proceedings of the 21st International Society for Music Information Retrieval Conference, ISMIR, Montreal, QC, Canada.
47. Bassan, S., Adi, Y., and Rosenschein, J.S. (2022). Unsupervised symbolic music segmentation using ensemble temporal prediction errors. arXiv.10.21437/Interspeech.2022-10379 ↗
48. A review of the literature on the relationship of music education to the development of socio-emotional learning10.1177/21582440211068501 ↗
49. A knowledge-based pilot study on assessing the music influence
50. Worschech, F., Marie, D., Jünemann, K., Sinke, C., Krüger, T.H., Grossbach, M., Scholz, D.S., Abdili, L., Kliegel, M., and James, C.E. (2021). Improved speech in noise perception in the elderly after 6 months of musical instruction. Front. Neurosci., 15.10.3389/fnins.2021.696240 ↗