journal article·2026-04-21Open access

Processing of Amplitude-Temporal Acoustic Parameters in the Auditory System During Signal Coding for Image Recognition: Analytical Review

Sergey Lytaev

Applied Sciences· Volume 16· Issue 8

Abstract

In the study of sensory processes, the visual system has received the most research compared to other sensory systems. The primary difference between visual and auditory perception lies in the nature of the stimuli and the reception processes: vision perceives electromagnetic radiation, while auditory perception perceives acoustic signals of mechanical origin. This review aims to analyze modern approaches and controversies to the mechanisms of auditory perception related to psychophysics, psychophysiology, psychopathology, modern research on hearing in human–computer interaction (HCI) systems, and machine learning methods. Modern studies of acoustic patterns include a comprehensive assessment of the physical characteristics of perception, complex nonverbal auditory cues, verbalization, perception and memory, as well as individual differences in auditory perception. An analysis of the scientific literature allowed us to conclude that acoustic signals transformed in the brain into auditory images retain (encode) a number of amplitude-temporal parameters of acoustic signals that facilitate auditory discrimination (filtering), but interfere with auditory detection (recognition). Signal processing often, but not necessarily, involves brain regions involved in other forms of perception. It depends on subvocalization, includes semantically interpreted information and expectations, pictorial (visual) and descriptive components, functions as a mnemonic, and is linked to individual musical ability and experience (although the mechanisms of this connection are unclear).
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Neuroscience and Music Perception · Multisensory perception and integration · Human auditory perception and evaluation

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Department of Normal Physiology, St. Petersburg State Pediatric Medical University, 194100 Saint Petersburg, Russia

Funding

Saint Petersburg State Pediatric Medical University

References (100+)

Show source-provided references
1. Shah, P., and Miyake, A. (2005). Visuospatial images. The Cambridge Handbook of Visuospatial Thinking, Cambridge University Press.10.1017/CBO9780511610448 ↗
2. Neural Activity When People Solve Verbal Problems with Insight
3. Brain electrical activity evoked by mental formation of auditory expectations and images10.1023/A:1007803102254 ↗
4. Lytaev, S. (2022). Long-latency event-related potentials (300–1000 ms) of the visual insight. Sensors, 22.10.3390/s22041323 ↗
5. Lytaev, S. (2024). Interaction of Sensitivity, Emotions, and Motivations during Visual Perception. Sensors, 24.10.3390/s24227414 ↗
6. Human discrimination of dynamic changes in the spatial position of sound images (electrophysiological and psychophysical study)10.1134/S0362119710010093 ↗
7. Total electrical potentials of the human brain during sound source localization
8. Remembering the past and imaging the future: A role for nonvisual imagery in the everyday cognition of blind and sighted people10.1080/09658210600859582 ↗
9. Neuropsychological analysis of cognitive deficits in schizophrenia10.1134/S0362119722010029 ↗
10. How does mismatch negativity reflect auditory motion?10.1016/j.heares.2010.06.001 ↗
11. Neurophysiological mechanisms of auditory adaptation. I. Adaptation during the stimulus action
12. Neurophysiological mechanisms of auditory adaptation. II. Poststimulus effects
13. Carlile, S. (2014). The plastic ear and perceptual relearning in auditory spatial perception. Front. Neurosci., 8.10.3389/fnins.2014.00237 ↗
14. Computerized Auditory Training in Students: Electrophysiological and Subjective Analysis of Therapeutic Effectiveness10.1055/s-0037-1600121 ↗
15. EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces10.1088/1741-2552/aace8c ↗
16. Levi-Aharoni, H., Shriki, O., and Tishby, N. (2020). Surprise response as a probe for compressed memory states. PLoS Comp. Biol., 16.10.1371/journal.pcbi.1007065 ↗
17. The frustration status and noise proof feature during perception of the auditory images10.1007/978-3-642-21852-1_23 ↗
18. Integration and Disintegration of Auditory Images Perception10.1007/978-3-319-20816-9_45 ↗
19. Short-term plasticity in the auditory system: Differential neural responses to perception and imagery of speech and music
20. Estimates of frequency resolving power of humans by different methods: The role of sensory and cognitive factors
21. Regular developmental changes in EEG multifractal characteristics10.1080/00207450390240086 ↗
22. Deep learning with convolutional neural networks for EEG decoding and visualization10.1002/hbm.23730 ↗
23. PET-Neuroimaging and Neuropsychological Study for Early Cognitive Impairment in Parkinson’s Disease
24. A P300-based brain–computer interface: Initial tests by ALS patients10.1016/j.clinph.2005.06.027 ↗
25. Acoustic characteristics of speech signal as an indicator of human functional state
26. Methods for classifying EEG patterns of imaged movements
27. Luo, H., and Poeppel, D. (2012). Cortical oscillations in auditory perception and speech: Evidence for two temporal windows in human auditory cortex. Front. Psychol., 3.10.3389/fpsyg.2012.00170 ↗
28. Neural Response Phase Tracks How Listeners Learn New Acoustic Representations10.1016/j.cub.2013.04.031 ↗
29. Sound Texture Perception via Statistics of the Auditory Periphery: Evidence from Sound Synthesis10.1016/j.neuron.2011.06.032 ↗
30. Summary statistics in auditory perception10.1038/nn.3347 ↗
31. Deep Neural Network with Joint Distribution Matching for Cross-Subject Motor Imagery Brain-Computer Interfaces10.1155/2020/7285057 ↗
32. Acuity of auditory images in pitch and time10.3758/BF03193705 ↗
33. Auditory Evoked Potentials to Spectro-Temporal Modulation of Complex Tones in Normal Subjects and Patients with Severe Brain Injury10.1093/brain/123.5.1007 ↗
34. A review of classification algorithms for EEG-based brain-computer interfaces: A 10 year update10.1088/1741-2552/aab2f2 ↗
35. Modeling and Estimation of Physiological, Psychological and Sensory Indicators for Working Capacity
36. Behavioral and neural correlates of perceived and imagined musical timbre10.1016/j.neuropsychologia.2003.12.017 ↗
37. Carrer, L.R. (2015). Music and sound in time processing of children with ADHD. Front. Psychiatry, 6.10.3389/fpsyt.2015.00127 ↗
38. Grinspun, N., Nijs, L., Kausel, L., Onderdijk, K., Sepúlveda, N., and Rivera-Hutinel, A. (2020). Selective attention and inhibitory control of attention are correlated with music audiation. Front. Psychol., 11.10.3389/fpsyg.2020.01109 ↗
39. Global-Local Processing in ADHD Is Not Limited to the Visuospatial Domain: Novel Evidence from the Auditory Domain10.1177/10870547231153952 ↗
40. Speech-evoked Brainstem Auditory Responses and Auditory Processing Skills: A Correlation in Adults with Hearing Loss10.1055/s-0037-1603109 ↗
41. Hearing sensitivity to shifts of rippled-spectrum patterns10.1121/1.4820789 ↗
42. Auditory images: How complex sounds are represented in the auditory system
43. Multisensory identification of natural objects in a two-way cross modal priming paradigm10.1027/1618-3169.55.2.121 ↗
44. Scanning silence: Mental imagery of complex sounds10.1016/j.neuroimage.2005.03.013 ↗
45. Event-related potentials during mental imagery of animal sounds10.1111/j.1469-8986.2006.00464.x ↗
46. Sound of silence activates auditory cortex10.1038/434158a ↗
47. Perception of emotional intonation of noisy speech signal with different acoustic parameters by adults of different age and gender
48. The Perception of Auditory Motion
49. Van Dinteren, R., Arns, M., Jongsma, M.L.A., and Kessels, R.P.C. (2014). P300 Development across the Lifespan: A Systematic Review and Meta-Analysis. PLoS ONE, 9.10.1371/journal.pone.0087347 ↗
50. Functional MRI of language: New approaches to understanding the cortical organization of semantic processing10.1146/annurev.neuro.25.112701.142946 ↗
51. TMS produces two dissociable types of speech disruption10.1006/nimg.2000.0701 ↗
52. Covert speech arrest induced by rTMS over both motor and nonmotor left hemisphere frontal sites10.1162/0898929054021157 ↗
53. Auditory hallucinations: A review of psychological treatments10.1016/S0920-9964(98)00052-8 ↗
54. A functional study of auditory verbal imagery10.1017/S003329170100335X ↗
55. Auditory grouping is necessary to understand interrupted mosaic speech stimuli10.1121/10.0013425 ↗
56. The functional neuroanatomy of metrical stress evaluation of perceived and imaged spoken words10.1093/cercor/bhh124 ↗
57. Different interference effects in musicians and a control group10.1027/1618-3169.53.2.111 ↗
58. Mismatch negativity and its hemodynamic equivalent (based on fMRI) in research of speech perception in healthy and in speech disorders
59. Compelling imagery, unanticipated speech, and deceptive memory: Neurocognitive models of auditory verbal hallucinations in schizophrenia10.1080/13546800344000156 ↗
60. Short Time Algorithms for Screening Examinations of the Collective and Personal Stress Resilience10.1007/978-3-031-35392-5_34 ↗
61. Auditory Spatial Perception Dynamically Realigns with Changing Eye Position10.1523/JNEUROSCI.0938-07.2007 ↗
62. Humans use summary statistics to perceive auditory sequences10.1177/0956797612473759 ↗
63. Minatoya, M., Daikoku, T., and Kuniyoshi, Y. (2025). Emotional responses to auditory hierarchical structures is shaped by bodily sensations and listeners’ sensory traits. Front. Psychol., 16.10.3389/fpsyg.2025.1599430 ↗
64. Do we enjoy what we sense and perceive? A dissociation between aesthetic appreciation and basic perception of environmental objects or events10.3758/s13415-022-01004-0 ↗
65. What are aesthetic emotions?10.1037/rev0000135 ↗
66. Schindler, I., Hosoya, G., Menninghaus, W., Beermann, U., Wagner, V., Eid, M., and Scherer, K.R. (2017). Measuring aesthetic emotions: A review of the literature and a new assessment tool. PLoS ONE, 12.10.1037/t79669-000 ↗
67. Egermann, H., and Reuben, F. (2020). “Beauty is how you feel inside”: Aesthetic judgments are related to emotional responses to contemporary music. Front. Psychol., 11.10.3389/fpsyg.2020.510029 ↗
68. List, A. (2023). Global and local priming in a multi-modal context. Front. Hum. Neurosci., 16.10.3389/fnhum.2022.1043475 ↗
69. Levels of Consciousness in Psychopathology According to Monitoring of Neural Network Centers Alpha Rhythm rs-EEG10.11591/eei.v14i4.9546 ↗
70. Neural Network of Alpha Rhythm rs-EEG Activity for the Assessment of Consciousness in Psychopathology10.21926/obm.neurobiol.2503299 ↗
71. Is auditory imagery defective in patients with auditory hallucinations?10.1017/S0033291799001555 ↗
72. Musical and auditory hallucinations: A spectrum10.1111/j.1440-1819.2004.01200.x ↗
73. Auditory system: Development, genetics, function, aging and diseases10.1007/s00441-015-2218-4 ↗
74. Sacks, O. (2007). Musicophilia: Tales of Music and the Brain, Knoft.
75. Musical hallucinations: Prevalence in psychotic and non-psychotic outpatients10.4088/JCP.v65n0208 ↗
76. Desynchronization in the right auditory cortex during musical hallucinations: A MEG study10.1046/j.1479-8301.2003.00009.x ↗
77. Processing of auditory motion in inferior parietal lobule: Evidence from transcranial magnetic stimulation10.1016/j.neuropsychologia.2010.11.038 ↗
78. On the Possible Role of Brain Rhythms in Speech Perception: Intelligibility of Time-Compressed Speech with Periodic and Aperiodic Insertions of Silence10.1159/000208934 ↗
79. Auditory Imagery: Empirical Findings10.1037/a0018436 ↗
80. The internal noise of the visual system and cognitive functions in schizophrenia10.1016/j.procs.2020.02.158 ↗
81. The functional anatomy of auditory hallucinations in schizophrenia10.1016/S0925-4927(00)00068-8 ↗
82. The role of head movements and signal spectrum in an auditory front/back illusion10.1068/i7173sas ↗
83. Nosulenko, V.N. (1988). Psychology of Auditory Perception, Nauka. (In Russian).
84. Endogenous Cortical Rhythms Determine Cerebral Specialization for Speech Perception and Production10.1016/j.neuron.2007.09.038 ↗
85. Imagery for musical timbre10.1037/0096-1523.15.3.472 ↗
86. The mental representation of music notation: Notational audition10.1037/0096-1523.34.2.427 ↗
87. Frolov, A.A., Mokienko, O., Lyukmanov, R., Biryukova, E., Kotov, S., Turbina, L., Nadareyshvily, G., and Bushkova, Y. (2017). Post-stroke Rehabilitation Training with a Motor-Imagery-Based Brain-Computer Interface (BCI)-Controlled Hand Exoskeleton: A Randomized Controlled Multicenter Trial. Front. Neurosci., 11.10.3389/fnins.2017.00400 ↗
88. Hearing sensitivity to gliding rippled spectrum patterns10.1121/1.5033898 ↗
89. Auditory motion direction encoding in auditory cortex and high-level visual cortex10.1002/hbm.21263 ↗
90. A review of EEG, ERP, and neuroimaging studies of creativity and insight10.1037/a0019749 ↗
91. Spectral information for detection of acoustic time to arrival10.3758/s13414-013-0424-2 ↗
92. Chait, M., Greenberg, S., Arai, T., Simon, J.Z., and Poeppel, D. (2015). Multi-time resolution analysis of speech: Evidence from psychophysics. Front. Neurosci., 9.10.3389/fnins.2015.00214 ↗
93. Modern Neurophysiological Research of the Human Brain in Clinic and Psychophysiology10.1007/978-3-030-88163-4_21 ↗
94. Cortical oscillations and speech processing: Emerging computational principles and operations10.1038/nn.3063 ↗
95. Auditory attention to frequency and time: An analogy to visual local–global stimuli10.1016/j.cognition.2004.11.001 ↗
96. Mismatch negativity (MMN) in basic research of central auditory processing: A review10.1016/j.clinph.2007.04.026 ↗
97. Action planning in sequential skills: Relations to music performance10.1080/17470210601160864 ↗
98. The Interaction between Somatosensory and Auditory Cognitive Processing Assessed with Event-Related Potentials10.1097/WNP.0b013e31816a8ffa ↗
99. Mismatch negativity as an indicator of human discriminative localization ability
100. Gaussian-noise discrimination and its relation to auditory object formation10.1121/1.3119626 ↗