Analyzing Free-Hand Sound-Tracings of Melodic Phrases
Source abstract
Abstract
In this paper, we report on a free-hand motion capture study in which 32 participants ‘traced’ 16 melodic vocal phrases with their hands in the air in two experimental conditions. Melodic contours are often thought of as correlated with vertical movement (up and down) in time, and this was also our initial expectation. We did find an arch shape for most of the tracings, although this did not correspond directly to the melodic contours. Furthermore, representation of pitch in the vertical dimension was but one of a diverse range of movement strategies used to trace the melodies. Six different mapping strategies were observed, and these strategies have been quantified and statistically tested. The conclusion is that metaphorical representation is much more common than a ‘graph-like’ rendering for such a melodic sound-tracing task. Other findings include a clear gender difference for some of the tracing strategies and an unexpected representation of melodies in terms of a small object for some of the Hindustani music examples. The data also show a tendency of participants moving within a shared ‘social box’.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Action Observation and Synchronization · Neuroscience and Music Perception · Language, Metaphor, and Cognition
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
University of Oslo, Department of Musicology, RITMO Centre for Interdisciplinary Studies in Rhythm, Time and Motion, 0371 Oslo, Norway
References (68)
Show source-provided references
1. Kennedy, M., Rutherford-Johnson, T., and Kennedy, J. (2013). The Oxford Dictionary of Music, Oxford University Press. [6th ed.]. Available online: http://www.oxfordreference.com/view/10.1093/acref/9780199578108.001.0001/acref-9780199578108.10.1093/acref/9780199578108.001.0001 ↗
2. Godøy, R.I., Haga, E., and Jensenius, A.R. (2005). Playing “Air Instruments”: Mimicry of Sound-Producing Gestures by Novices and Experts, Springer.10.1007/11678816_29 ↗
3. Kendon, A. (2004). Gesture: Visible Action as Utterance, Cambridge University Press.10.1017/CBO9780511807572 ↗
4. In time with the music: The concept of entrainment and its significance for ethnomusicology
5. Zbikowski, L.M. (2011). Musical gesture and musical grammar: A cognitive approach. New Perspectives on Music and Gesture, Ashgate Publishing Ltd.
6. Conceptual metaphor in everyday language10.2307/2025464 ↗
7. Metaphor and music theory: Reflections from cognitive science10.30535/mto.4.1.1 ↗
8. The thickness of pitch: Crossmodal metaphors in Farsi, Turkish, and Zapotec10.2752/174589311X12893982233911 ↗
9. Lower pitch is larger, yet falling pitches shrink10.1027/1618-3169/a000246 ↗
10. Beethoven’s last piano sonata and those who follow crocodiles: Cross-domain mappings of auditory pitch in a musical context10.1016/j.cognition.2009.10.013 ↗
11. Spatial representation of pitch height: The SMARC effect10.1016/j.cognition.2005.01.004 ↗
12. The melodic arch in Western folksongs
13. Structural integration in language and music: Evidence for a shared system10.3758/MC.37.1.1 ↗
14. Patel, A.D. (2010). Music, Language, and the Brain, Oxford University Press.
15. Adami, A.G., Mihaescu, R., Reynolds, D.A., and Godfrey, J.J. (2003, January 6–10). Modeling prosodic dynamics for speaker recognition. Proceedings of the 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP’03), Hong Kong, China.
16. Scale and contour: Two components of a theory of memory for melodies10.1037/0033-295X.85.4.341 ↗
17. Recognition of melodic transformations: Inversion, retrograde, and retrograde inversion10.3758/BF03205852 ↗
18. Coding of melodic gestalt in human auditory cortex10.1093/cercor/bhs289 ↗
19. Cross-cultural perspectives on music and musicality10.1098/rstb.2014.0096 ↗
20. Infants’ perception of melodies: The role of melodic contour
21. Infants’ perception of melodies: Changes in a single tone10.1016/S0163-6383(85)80007-2 ↗
22. Children’s perception of melodies: The role of contour, frequency, and rate of presentation10.1016/0022-0965(85)90090-6 ↗
23. Melodic contour typology10.2307/851015 ↗
24. The motor origins of human and avian song structure10.1073/pnas.1103882108 ↗
25. Automatic infant cry analysis for the identification of qualitative features to help opportune diagnosis10.1016/j.bspc.2011.06.011 ↗
26. Parsons, D. (1975). The Directory of Tunes and Musical Themes, S. Brown.
27. The combinatorial model of pitch contour10.2307/40285803 ↗
28. Narmour, E. (1992). The Analysis and Cognition of Melodic Complexity: The Implication-Realization Model, University of Chicago Press.
29. Marvin, E.W. (1988). A Generalized Theory of Musical Contour: Its Application to Melodic and Rhythmic Analysis of Non-Tonal Music and Its Perceptual and Pedagogical Implications. [Ph.D. Thesis, University of Rochester].
30. Testing models of melodic contour similarity10.2307/40285795 ↗
31. Melodic contour similarity using folk melodies10.1525/mp.2010.28.2.169 ↗
32. Expectation in music: Investigation of melodic and harmonic processes10.2307/40285454 ↗
33. Perceived complexity of western and African folk melodies by western and African listeners10.1177/0305735606064842 ↗
34. Melodic and contextual similarity of folk song phrases10.1177/102986490701100109 ↗
35. Are the emotions expressed in music genre-specific? An audio-based evaluation of datasets spanning classical, film, pop and mixed genres10.1080/09298215.2011.602195 ↗
36. Salamon, J., Peeters, G., and Röbel, A. (2012, January 8–12). Statistical Characterisation of Melodic Pitch Contours and its Application for Melody Extraction. Proceedings of the ISMIR, Porto, Portugal.
37. Bittner, R.M., Salamon, J., Bosch, J.J., and Bello, J.P. (2017, January 22–24). Pitch Contours as a Mid-Level Representation for Music Informatics. Proceedings of the Audio Engineering Society Conference: 2017 AES International Conference on Semantic Audio, Erlangen, Germany.
38. Classification of melodic motifs in raga music with time-series matching10.1080/09298215.2013.873470 ↗
39. Towards computer-assisted flamenco transcription: An experimental comparison of automatic transcription algorithms as applied to a cappella singing10.1162/COMJ_a_00180 ↗
40. Preverbal infants are sensitive to cross-sensory correspondences much ado about the null results of Lewkowicz and Minar (2014)10.1177/0956797613520170 ↗
41. Representation of pitch in horizontal space and its dependence on musical and instrumental experience10.1037/pmu0000146 ↗
42. Leman, M. (2008). Embodied Music Cognition and Mediation Technology, MIT Press.10.7551/mitpress/7476.001.0001 ↗
43. Motor-mimetic music cognition10.1162/002409403322258781 ↗
44. Jensenius, A.R. (2007). Action-Sound: Developing Methods and Tools to Study Music-Related Body Movement. [Ph.D. Thesis, University of Oslo].
45. Jensenius, A.R., Kvifte, T., and Godøy, R.I. (2006, January 4–8). Towards a gesture description interchange format. Proceedings of the 2006 Conference on New Interfaces for Musical Expression, IRCAM—Centre Pompidou, Paris, France.
46. Gritten, A., and King, E. (2006). Music and Gesture, Ashgate Publishing, Ltd.
47. Gritten, A., and King, E. (2011). New Perspectives on Music and Gesture, Ashgate Publishing, Ltd.
48. Music and mirror neurons: From motion to ‘e’motion10.1093/scan/nsl029 ↗
49. Music and movement share a dynamic structure that supports universal expressions of emotion10.1073/pnas.1209023110 ↗
50. From contour similarity to motivic topologies10.1177/102986490000400201 ↗
51. Clayton, M., and Leante, L. (2013). Embodiment in Music Performance. Experience and Meaning in Music Performance, Oxford University Press.10.1093/acprof:oso/9780199811328.001.0001 ↗
52. Rahaim, M. (2012). Musicking Bodies: Gesture and Voice in Hindustani Music, Wesleyan University Press.
53. Eyebrow movements and vocal pitch height: Evidence consistent with an ethological signal10.1121/1.4798801 ↗
54. Global music recordings support the motor constraint hypothesis for human and avian song contour10.1525/mp.2017.34.3.327 ↗
55. Godøy, R.I., Haga, E., and Jensenius, A.R. (2018, January 16). Exploring Music-Related Gestures by Sound-Tracing: A Preliminary Study. Available online: https://www.duo.uio.no/bitstream/handle/10852/26899/Godxy_2006b.pdf?sequence=1&isAllowed=y.
56. Glette, K.H., Jensenius, A.R., and Godøy, R.I. (2018, January 16). Extracting Action-Sound Features From a Sound-Tracing Study. Available online: https://www.duo.uio.no/bitstream/handle/10852/8848/Glette_2010.pdf?sequence=1&isAllowed=y.
57. Music and shape10.1093/llc/fqs071 ↗
58. Roy, U., Kelkar, T., and Indurkhya, B. (July, January 30). TrAP: An Interactive System to Generate Valid Raga Phrases from Sound-Tracings. Proceedings of the NIME, London, UK.
59. Kelkar, T. (2015). Applications of Gesture and Spatial Cognition in Hindustani Vocal Music. [Ph.D. Thesis, International Institute of Information Technology].
60. Analyzing Correspondence Between Sound Objects and Body Motion10.1145/2465780.2465783 ↗
61. Nymoen, K., Caramiaux, B., Kozak, M., and Torresen, J. (2011, January 30). Analyzing Sound-Tracings: A Multimodal Approach to Music Information Retrieval. Proceedings of the 1st International ACM Workshop on Music Information Retrieval with User-centered and Multimodal Strategies (MIRUM ’11), Scottsdale, AZ, USA.10.1145/2072529.2072541 ↗
62. Ollen, J.E. (2006). A Criterion-Related Validity Test of Selected Indicators of Musical Sophistication Using Expert Ratings. [Ph.D. Thesis, The Ohio State University].
63. Nymoen, K., Torresen, J., Godøy, R., and Jensenius, A. (2012). A statistical approach to analyzing sound-tracings. Speech, Sound and Music Processing: Embracing Research in India, Springer.10.1007/978-3-642-31980-8_11 ↗
64. Schmuckler, M.A. (2009). Components of melodic processing. Oxford Handbook of Music Psychology, Oxford Uniersity Press.
65. Godøy, R.I. (2010). Gestural affordances of musical sound. Musical Gestures: Sound, Movement, and Meaning, Routledge.10.4324/9780203863411 ↗
66. Paschalidou, S., Clayton, M., and Eerola, T. (2017, January 18–22). Effort in Interactions with Imaginary Objects in Hindustani Vocal Music—Towards Enhancing Gesture-Controlled Virtual Instruments. Proceedings of the 2017 International Symposium on Musical Acoust, Montreal, QC, Canada.
67. Paschalidou, S., and Clayton, M. (2015, January 23–25). Towards a sound-gesture analysis in Hindustani Dhrupad vocal music: Effort and raga space. Proceedings of the ICMEM, University of Sheffield, Sheffield, UK.
68. Pearson, L. (2016). Gesture in Karnatak Music: Pedagogy and Musical Structure in South India. [Ph.D. Thesis, Durham University].