Applying Acoustical and Musicological Analysis to Detect Brain Responses to Realistic Music: A Case Study
Source abstract
Abstract
Music information retrieval (MIR) methods offer interesting possibilities for automatically identifying time points in music recordings that relate to specific brain responses. However, how the acoustical features and the novelty of the music structure affect the brain response is not yet clear. In the present study, we tested a new method for automatically identifying time points of brain responses based on MIR analysis. We utilized an existing database including brain recordings of 48 healthy listeners measured with electroencephalography (EEG) and magnetoencephalography (MEG). While we succeeded in capturing brain responses related to acoustical changes in the modern tango piece Adios Nonino, we obtained less reliable brain responses with a metal rock piece and a modern symphony orchestra musical composition. However, brain responses might also relate to the novelty of the music structure. Hence, we added a manual musicological analysis of novelty in the musical structure to the computational acoustic analysis, obtaining strong brain responses even to the rock and modern pieces. Although no standardized method yet exists, these preliminary results suggest that analysis of novelty in music is an important aid to MIR analysis for investigating brain responses to realistic music.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Neuroscience and Music Perception · Music and Audio Processing · Multisensory perception and integration
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Center for Music in the Brain, Department of Clinical Medicine, Aarhus University and Royal Academy of Music, Aarhus/Aalborg, Nørrebrogade 44, Building 10G, 4th and 5th floor, 8000 Aarhus C, Denmark
BioMag Laboratory, HUS Medical Imaging Center, University of Helsinki and Helsinki University Hospital, 00100 Helsinki, Finland
Funding
Danmarks Grundforskningsfond
References (60)
Show source-provided references
1. Lartillot, O., and Toiviainen, P. (2007, January 10–15). A matlab toolbox for musical feature extraction from audio. Proceedings of the International Conference on Digital Audio Effects, Bordeaux, France.
2. Alías, F., Socoró, J., and Sevillano, X. (2016). A review of physical and perceptual feature extraction techniques for speech, music and environmental sounds. Appl. Sci., 6.10.3390/app6050143 ↗
3. Large-scale brain networks emerge from dynamic processing of musical timbre, key and rhythm10.1016/j.neuroimage.2011.11.019 ↗
4. The reliability of continuous brain responses during naturalistic listening to music10.1016/j.neuroimage.2015.09.005 ↗
5. Event-related brain responses while listening to entire pieces of music10.1016/j.neuroscience.2015.10.061 ↗
6. Early auditory processing in musicians and dancers during a contemporary dance piece10.1038/srep33056 ↗
7. Sturm, I., Dahne, S., Blankertz, B., and Curio, G. (2015). Multi-variate eeg analysis as a novel tool to examine brain responses to naturalistic music stimuli. PLoS ONE, 10.10.1371/journal.pone.0141281 ↗
8. Haumann, N.T., Santacruz, J.L., Kliuchko, M., Vuust, P., and Brattico, E. (2018). Automatic capture of acoustic changes in real pieces of music: Transient and non-transient brain responses, in preparation.
9. Magnetoencephalography—Theory, instrumentation, and applications to noninvasive studies of the working human brain10.1103/RevModPhys.65.413 ↗
10. Predictive coding of music—Brain responses to rhythmic incongruity10.1016/j.cortex.2008.05.014 ↗
11. Ever-changing cycles of musical pleasure: The role of dopamine and anticipation10.1037/a0031126 ↗
12. Predictive information processing in music cognition. A critical review10.1016/j.ijpsycho.2011.12.010 ↗
13. Toward a neural chronometry for the aesthetic experience of music10.3389/fpsyg.2013.00206 ↗
14. The role of expectation in music: From the score to emotions and the brain10.1002/wcs.1262 ↗
15. Tracking timbral changes in metal productions from 1990 to 201310.1386/mms.1.1.39_1 ↗
16. The N1 wave of the human electric and magnetic response to sound—A review and an analysis of the component structure10.1111/j.1469-8986.1987.tb00311.x ↗
17. Distinct neural responses to chord violations: A multiple source analysis study10.1016/j.brainres.2011.02.089 ↗
18. Influence of musical enculturation on brain responses to metric deviants10.3389/fnins.2018.00218 ↗
19. Kliuchko, M., Vuust, P., Tervaniemi, M., Bogert, B., Sams, M., Toiviainen, P., and Brattico, E. (2018). Fractionating music-derived neuroplasticity: Neural correlates of active versus passive musical style preference, in preparation.
20. Koelsch, S., Kilches, S., Steinbeis, N., and Schelinski, S. (2008). Effects of unexpected chords and of performer’s expression on brain responses and electrodermal activity. PLoS ONE, 3.10.1371/journal.pone.0002631 ↗
21. Perceiving musical scale structures—A cross-cultural event-related brain potentials study10.1196/annals.1284.026 ↗
22. Unsupervised statistical learning underpins computational, behavioural, and neural manifestations of musical expectation10.1016/j.neuroimage.2009.12.019 ↗
23. Practiced musical style shapes auditory skills
24. Musical expertise modulates functional connectivity of limbic regions during continuous music listening10.1037/pmu0000124 ↗
25. Burunat, I., Brattico, E., Puolivali, T., Ristaniemi, T., Sams, M., and Toiviainen, P. (2015). Action in perception: Prominent visuo-motor functional symmetry in musicians during music listening. PLoS ONE, 10.10.1371/journal.pone.0138238 ↗
26. Maladaptive and adaptive emotion regulation through music: A behavioral and neuroimaging study of males and females10.3389/fnhum.2015.00466 ↗
27. The association of noise sensitivity with music listening, training, and aptitude10.4103/1463-1741.165065 ↗
28. Comparing the performance of popular MEG/EEG artifact correction methods in an evoked-response study10.1155/2016/7489108 ↗
29. A window into the brain mechanisms associated with noise sensitivity10.1038/srep39236 ↗
30. Connectivity patterns during music listening: Evidence for action-based processing in musicians10.1002/hbm.23565 ↗
31. Risk of depression enhances auditory pitch discrimination in the brain as indexed by the mismatch negativity10.1016/j.clinph.2017.07.004 ↗
32. Coupling of action-perception brain networks during musical pulse processing: Evidence from region-of-interest-based independent component analysis10.3389/fnhum.2017.00230 ↗
33. Neuroanatomical substrate of noise sensitivity10.1016/j.neuroimage.2017.11.041 ↗
34. Decoding musical training from dynamic processing of musical features in the brain10.1038/s41598-018-19177-5 ↗
35. Thiede, A., Suppanen, E., Brattico, E., Sams, M., and Parkkonen, L. (2018). Magnetoencephalographic inter-subject correlation using continuous music stimuli, in preparation.
36. Bonetti, L., Haumann, N.T., Brattico, E., Kliuchko, M., Vuust, P., and Näätänen, R. (2018). Working memory modulates frontal mismatch negativity responses to sound intensity and slide deviants. Brain Res., submitted.
37. Fieldtrip: Open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data10.1155/2011/156869 ↗
38. Tonal consonance and critical bandwidth10.1121/1.1909741 ↗
39. Consonance-based spectral mappings10.2307/3681045 ↗
40. Rioting with stravinsky: A particular analysis of the “rite of spring”10.1111/j.1468-2249.2007.00250.x ↗
41. Evoked-potential audiometry
42. Human auditory sustained potentials. 2. Stimulus relationships10.1016/0013-4694(78)90004-4 ↗
43. Long latency auditory evoked potentials: Intensity, inter-stimulus interval, and habituation10.1007/BF02910543 ↗
44. Müller, M., Prätzlich, T., and Driedger, J. (2012, January 8–12). A cross-version approach for stabilizing tempo-based novelty detection. Proceedings of the ISMIR, Porto, Portugal.
45. Flexer, A., Pampalk, E., and Widmer, G. (2005, January 11–15). Novelty detection based on spectral similarity of songs. Proceedings of the ISMIR, London, UK.
46. Structural and affective aspects of music from statistical audio signal analysis10.1002/asi.20429 ↗
47. Music structure analysis using self-similarity matrix and two-stage categorization10.1007/s11042-013-1761-9 ↗
48. Lokki, T., Pätynen, J., and Välimäki, V. (2017, January 5–8). Musical Feature and Novelty Curve Characterizations as Predictors of Segmentation Accuracy. Proceedings of the Smc 2017: Proceedings of the 14th Sound and Music Computing Conference, Espoo, Finland.
49. Emotional ratings of music excerpts in the western art music repertoire and their self-organization in the kohonen neural network10.1177/0305735605056147 ↗
50. Implicit learning of tonality: A self-organizing approach10.1037/0033-295X.107.4.885 ↗
51. Measuring and modeling real-time responses to music: The dynamics of tonality induction10.1068/p3312 ↗
52. Visualization of tonal content with self-organizing maps and self-similarity matrices
53. A combined model of sensory and cognitive representations underlying tonal expectations in music: From audio signals to behavior10.1037/a0034695 ↗
54. Rao, Z., Guan, X., and Teng, J. (2016). Chord recognition based on temporal correlation support vector machine. Appl. Sci., 6.10.3390/app6050157 ↗
55. Jensen, K. (2015). Sensory dissonance using memory model. Dan. Musicol. Online Spec. Ed., 59–68. http://www.danishmusicologyonline.dk/arkiv/arkiv_dmo/dmo_saernummer_2015/dmo_saernummer_2015_musik_hjerneforskning_03.pdf.
56. Sak, H., Senior, A., and Beaufays, F. (2014, January 14–18). Long short-term memory recurrent neural network architectures for large scale acoustic modeling. Proceedings of the INTERSPEECH, Singapore.10.21437/Interspeech.2014-80 ↗
57. Why there are complementary learning-systems in the hippocampus and neocortex—Insights from the successes and failures of connectionist models of learning and memory10.1037/0033-295X.102.3.419 ↗
58. Interactions between frontal cortex and basal ganglia in working memory: A computational model10.3758/CABN.1.2.137 ↗
59. O’Reilly, R.C., Munakata, Y., Frank, M.J., and Hazy, T.E. (2012). Computational Cognitive Neuroscience, Wiki Book. [1st ed.].
60. The perceptual attack time of musical tones10.1121/1.395441 ↗