A Biologically Inspired Sound Localisation System Using a Silicon Cochlea Pair
Source abstract
Abstract
We present a biologically inspired sound localisation system for reverberant environments using the Cascade of Asymmetric Resonators with Fast-Acting Compression (CAR-FAC) cochlear model. The system exploits a CAR-FAC pair to pre-process binaural signals that travel through the inherent delay line of the cascade structures, as each filter acts as a delay unit. Following the filtering, each cochlear channel is cross-correlated with all the channels of the other cochlea using a quantised instantaneous correlation function to form a 2-D instantaneous correlation matrix (correlogram). The correlogram contains both interaural time difference and spectral information. The generated correlograms are analysed using a regression neural network for localisation. We investigate the effect of the CAR-FAC nonlinearity on the system performance by comparing it with a CAR only version. To verify that the CAR/CAR-FAC and the quantised instantaneous correlation provide a suitable basis with which to perform sound localisation tasks, a linear regression, an extreme learning machine, and a convolutional neural network are trained to learn the azimuthal angle of the sound source from the correlogram. The system is evaluated using speech data recorded in a reverberant environment. We compare the performance of the linear CAR and nonlinear CAR-FAC models with current sound localisation systems as well as with human performance.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Adaptive Filtering Techniques · Speech and Audio Processing · Hearing Loss and Rehabilitation
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
International Centre for Neuromorphic Systems, The MARCS Institute for Brain, Behaviour, and Development, Western Sydney University, Kingswood, NSW 2751, Australia
Department of Electronic Systems Engineering, Indian Institute of Science, Bangalore 560012, India
School of Electrical and Data Engineering, University of Technology Sydney, Sydney, NSW 2000, Australia
Funding
Australian Research Council Grant · Indian Science and Engineering Research Board
References (66)
Show source-provided references
1. The function of the medial superior olive in small mammals: Temporal receptive fields in auditory analysis10.1007/s003590050441 ↗
2. Neural mechanisms of encoding Binaural localization cues in the auditory brainstem10.1007/978-1-4757-3654-0_4 ↗
3. Park, T.J., Klug, A., Holinstat, M., and Grothe, B. (2004). Interaural level difference processing in the lateral superior olive and the inferior colliculus. J. Neurophysiol., 92.10.1152/jn.00961.2003 ↗
4. On our perception of sound direction10.1080/14786440709463595 ↗
5. Just noticeable differences in dichotic phase10.1121/1.1908495 ↗
6. Localization of pure tones10.1121/1.1913586 ↗
7. Detectability of interaural delay in high-frequency complex waveforms10.1121/1.1928135 ↗
8. Temporal coding of envelopes and their interaural delays in the inferior colliculus of the unanesthetized rabbit10.1152/jn.1989.61.2.257 ↗
9. Envelope coding in the lateral superior olive. II. Characteristic delays and comparison with responses in the medial superior olive10.1152/jn.1996.76.4.2137 ↗
10. Mechanisms of Sound localization in mammals10.1152/physrev.00026.2009 ↗
11. Yin, T.C.T., and Kuwada, S. (2012). Binaural localization cues. The Oxford Handbook of Auditory Science: The Auditory Brain, Oxford University Press.
12. Lyon, R.F. (1983, January 14–16). A computational model of binaural localization and separation. Proceedings of the ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing, Boston, MA, USA.
13. Shamma, S.A., Shen, N., and Gopalaswamy, P. (1989). Stereausis: Binaural processing without neural delays. J. Acoust. Soc. Am., 86.10.1121/1.398734 ↗
14. A place theory of sound localization
15. Heckmann, M., Rodemann, T., Joublin, F., Goerick, C., and Schölling, B. (2006, January 9–15). Auditory inspired binaural robust sound source localization in echoic and noisy environments. Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Beijing, China.10.1109/IROS.2006.281758 ↗
16. Early Binaural hearing: The Comparison of temporal differences at the two ears10.1146/annurev-neuro-080317-061925 ↗
17. A silicon model of auditory localization10.1162/neco.1989.1.1.47 ↗
18. Sound localization: Jeffress and beyond10.1016/j.conb.2011.05.008 ↗
19. Bhadkamkar, N., and Fowler, B. (April, January 28). A sound localization system based on biological analogy. Proceedings of the 1993 IEEE International Conference on Neural Networks, San Francisco, CA, USA.
20. Analog CMOS chipset for a 2-D sound localization system10.1023/B:ALOG.0000041634.92147.0d ↗
21. Analog VLSI model of binaural hearing10.1109/72.80333 ↗
22. A neuromorphic sound localizer for a smart MEMS system10.1023/B:ALOG.0000029662.37528.c7 ↗
23. Ponca, M., and Schauer, C. (2001). FPGA implementation of a spike-based sound localization system. Artificial Neural Networks and Genetic Algorithms, Springer.10.1007/978-3-7091-6230-9_52 ↗
24. Finger, H., and Liu, S.C. (2011, January 15–18). Estimating the location of a sound source with a spike-timing localization algorithm. Proceedings of the IEEE International Symposium on Circuits and Systems, Rio de Janeiro, Brazil.10.1109/ISCAS.2011.5938102 ↗
25. Iwasa, K., Kugler, M., Kuroyanagi, S., and Iwata, A. (2007, January 12–17). A sound localization and recognition system using pulsed neural networks on FPGA. Proceedings of the International Joint Conference on Neural Networks (IJCNN), Orlando, FL, USA.10.1109/IJCNN.2007.4371078 ↗
26. A complete hardware implementation of an integrated sound localization and classification system based on spiking neural networks
27. Adaptive sound localization with a silicon cochlea pair10.3389/fnins.2010.00196 ↗
28. Schauer, C., Zahn, T., Paschke, P., and Gross, H.M. (2000, January 5–9). Binaural sound localization in an artificial neural network. Proceedings of the ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing, Istanbul, Turkey.
29. Youssef, K., Argentieri, S., and Zarader, J.L. (2013, January 3–7). A learning-based approach to robust binaural sound localization. Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Tokyo, Japan.10.1109/IROS.2013.6696771 ↗
30. Exploiting deep neural networks and head movements for robust binaural localization of multiple sources in reverberant environments10.1109/TASLP.2017.2750760 ↗
31. Wang, J., Wang, J., Qian, K., Xie, X., and Kuang, J. (2020). Binaural sound localization based on deep neural network and affinity propagation clustering in mismatched HRTF condition. Eurasip J. Audio Speech Music Process.10.1186/s13636-020-0171-y ↗
32. Deep and CNN fusion method for binaural sound source localisation
33. Xu, Y., Afshar, S., Singh, R.K., Hamilton, T.J., Wang, R., and van Schaik, A. (2018, January 27–30). A machine hearing system for binaural sound localization based on instantaneous correlation. Proceedings of the 2018 IEEE International Symposium on Circuits and Systems (ISCAS), Florence, Italy.10.1109/ISCAS.2018.8351367 ↗
34. The precedence effect in sound localization10.2307/1418275 ↗
35. The precedence effect in scenarios with projected sound10.17743/jaes.2018.0074 ↗
36. Xu, Y., Afshar, S., Singh, R.K., Wang, R., van Schaik, A., and Hamilton, T.J. (2019, January 26–29). A binaural sound localization system using deep convolutional neural networks. Proceedings of the IEEE International Symposium on Circuits and Systems, Sapporo, Japan.10.1109/ISCAS.2019.8702345 ↗
37. Lyon, R.F. (2017). Human and Machine Hearing—Extracting Meaning from Sound, Cambridge University Press.10.1017/9781139051699 ↗
38. Xu, Y., Singh, R.K., Thakur, C.S., Wang, R., and van Schaik, A. (2016, January 17–19). CAR-FAC Model of the cochlea on the FPGA. Proceedings of the BioMedical Circuits and Systems Conference (BIOCAS), Shanghai, China.
39. A FPGA implementation of the CAR-FAC cochlear model10.3389/fnins.2018.00198 ↗
40. A cochlear frequency-position function for several species—29 years later10.1121/1.399052 ↗
41. CAR-lite: A Multi-rate cochlear model on FPGA for Spike-based sound encoding10.1109/TCSI.2018.2868247 ↗
42. Singh, R.K., Xu, Y., Wang, R., Hamilton, T.J., van Schaik, A., and Denham, S.L. (2018, January 27–30). CAR-lite: A Multi-rate cochlea model on FPGA. Proceedings of the IEEE International Symposium on Circuits and Systems, Florence, Italy.10.1109/ISCAS.2018.8351394 ↗
43. Katsiamis, A.G., Drakakis, E.M., and Lyon, R.F. (2007). Practical gammatone-like filters for auditory processing. Eurasip J. Audio Speech Music Process.10.1155/2007/63685 ↗
44. Chi, T., Ru, P., and Shamma, S.A. (2005). Multiresolution spectrotemporal analysis of complex sounds. J. Acoust. Soc. Am., 118.10.1121/1.1945807 ↗
45. Seidner, D. (2008, January 13–14). Efficient implementation of log10 lookup table in FPGA. Proceedings of the IEEE International Conference on Microwaves, Communications, Antennas and Electronic Systems, Tel-Aviv, Israel.10.1109/COMCAS.2008.4562810 ↗
46. Bangqiang, L., Ling, H., and Xiao, Y. (2013, January 8–10). Base-N logarithm implementation on FPGA for the data with random decimal point positions. Proceedings of the IEEE 9th International Colloquium on Signal Processing and Its Applications (CSPA), Kuala Lumpur, Malaysia.
47. Extreme learning machine: Theory and applications10.1016/j.neucom.2005.12.126 ↗
48. Fast, simple and accurate handwritten digit classification by training shallow neural network classifiers with the ‘extreme learning machine’ algorithm10.1371/journal.pone.0134254 ↗
49. Online and adaptive pseudoinverse solutions for ELM weights10.1016/j.neucom.2014.01.071 ↗
50. Wang, R., Cohen, G., Thakur, C.S., Tapson, J., and van Schaik, A. (2016, January 17–19). An SRAM-based implementation of a convolutional neural network. Proceedings of the IEEE Biomedical Circuits and Systems Conference, Shanghai, China.10.1109/BioCAS.2016.7833856 ↗
51. High-performance CNN accelerator on FPGA using unified winograd-GEMM architecture10.1109/TVLSI.2019.2941250 ↗
52. A high-throughput and power-efficient fpga implementation of yolo CNN for object detection10.1109/TVLSI.2019.2905242 ↗
53. High-performance FPGA-based CNN accelerator with block-floating-point arithmetic10.1109/TVLSI.2019.2913958 ↗
54. Al-Rfou, R. (2016). Theano: A Python Framework for Fast Computation of Mathematical Expressions, Cornell University.
55. He, K., Zhang, X., Ren, S., and Sun, J. (2015, January 7–13). Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.10.1109/ICCV.2015.123 ↗
56. Ruder, S. (2016). An Overview of Gradient Descent Optimization Algorithms, Cornell University.
57. Theano Development Team (2016). Theano: A Python Framework for Fast Computation of Mathematical Expressions, Cornell University.
58. Zhou, J., Hong, X., Su, F., and Zhao, G. (July, January 26). Recurrent convolutional neural network regression for continuous pain intensity estimation in video. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Las Vegas, NV, USA.
59. Burnham, D., Ambikairajah, E., Arciuli, J., Bennamoun, M., Best, C.T., Bird, S., Butcher, A.R., Cassidy, S., Chetty, G., and Cox, F.M. A blueprint for a comprehensive Australian English auditory-visual speech corpus. Selected Proceedings of the 2008 HCSNet Workshop on Designing the Australian National Corpus, Cascadilla Proceedings Project.
60. Burnham, D., Estival, D., Fazio, S., Viethen, J., Cox, F., Dale, R., Cassidy, S., Epps, J., Togneri, R., and Wagner, M. (2011, January 27–31). Building an audio-visual corpus of Australian English: Large corpus collection with an economical portable and replicable black box. Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH), Florence, Italy.10.21437/Interspeech.2011-309 ↗
61. Ko, T., Peddinti, V., Povey, D., and Khudanpur, S. (2015, January 6–10). Audio augmentation for speech recognition. Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH), Dresden, Germany.10.21437/Interspeech.2015-711 ↗
62. Cascades of two-pole-two-zero asymmetric resonators are good models of peripheral auditory function10.1121/1.3658470 ↗
63. A low-power correlation-derivative CMOS VLSI circuit for bearing estimation10.1109/TVLSI.2005.863740 ↗
64. Real-time neuro-inspired sound source localization and tracking architecture applied to a robotic platform10.1016/j.neucom.2017.12.041 ↗
65. Sound localization by human listeners10.1146/annurev.ps.42.020191.001031 ↗
66. The nature and distribution of errors in sound localization by human listeners10.1016/S0378-5955(97)00161-5 ↗