DataVXApplied SciencesPaper
journal article·2022-06-09Open access

A Review of Neural Network-Based Emulation of Guitar Amplifiers

Tara Vanhatalo, Pierrick Legrand, Myriam Desainte-Catherine, Pierre Hanna, Antoine Brusco, Guillaume Pille, Yann Bayle

Applied Sciences· Volume 12· Issue 12
Publisher pageOpen full text
Source abstract

Abstract

Vacuum tube amplifiers present sonic characteristics frequently coveted by musicians, that are often due to the distinct nonlinearities of their circuits, and accurately modelling such effects can be a challenging task. A recent rise in machine learning methods has lead to the ubiquity of neural networks in all fields of study including virtual analog modelling. This has lead to the appearance of a variety of architectures tailored to this task. This article aims to provide an overview of the current state of the research in neural emulation of analog distortion circuits by first presenting preceding methods in the field and then focusing on a complete review of the deep learning landscape that has appeared in recent years, detailing each subclass of available architectures. This is done in order to bring to light future possible avenues of work in this field.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Model Reduction and Neural Networks · Analog and Mixed-Signal Circuit Design · Music and Audio Processing

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

Inria Bordeaux Sud-Ouest, Institute of Mathematics of Bordeaux, UMR 5251 CNRS, University of Bordeaux, F-33405 Talence, France

University of Bordeaux, CNRS, Bordeaux INP, LaBRI, UMR 5800, F-33400 Talence, France

Orosys, F-34980 Saint-Gély-du-Fesc, France

Funding

Association Nationale de la Recherche et de la Technologie (ANRT) · Orosys

References (60)

Show source-provided references
1. Cool Sound of Tubes10.1109/6.708439 ↗
2. Zölzer, U. (2011). DAFX: Digital Audio Effects, John Wiley & Sons Ltd.. Available online: https://onlinelibrary.wiley.com/doi/book/10.1002/9781119991298.10.1002/9781119991298 ↗
3. Presonus (2022, April 08). Digital Audio Latency Explained. Available online: https://www.presonus.com/learn/technical-articles/Digital-Audio-Latency-Explained.
4. Kuznetsov, B., Parker, J.D., and Esqueda, F. (2020, January 8–12). Differentiable IIR Filters for Machine Learning Applications. Proceedings of the 23rd International Conference on Digital Audio Effects, Vienna, Austria. Available online: http://www.dafx.de/paper-archive/2020/proceedings/papers/DAFx2020_paper_52.pdf.
5. A Review of Digital Techniques for Modeling Vacuum-tube Guitar Amplifiers10.1162/comj.2009.33.2.85 ↗
6. Yeh, D.T. (2009). Digital Implementation of Musical Distortion Circuits by Analysis and Simulation. [Ph.D. Thesis, Stanford University]. Available online: https://ccrma.stanford.edu/~dtyeh/papers/DavidYehThesissinglesided.pdf.
7. Yeh, D.T., and Smith, J.O. (2008, January 1–4). Simulating Guitar Distortion Circuits using Wave Digital and Nonlinear State-Space formulations. Proceedings of the 11th International Conference on Digital Audio Effects (DAFx-08), Espoo, Finland. Available online: http://legacy.spa.aalto.fi/dafx08/papers/dafx08_04.pdf.
8. Elimination of Delay-free Loops in Discrete-time Models of Nonlinear Acoustic Systems10.1109/89.861380 ↗
9. Automated Physical Modeling of Nonlinear Audio Circuits for Real-time Audio Effects—Part II: BJT and Vacuum Tube Examples10.1109/TASL.2011.2173677 ↗
10. Falaize, A., and Hélie, T. (2016). Passive Guaranteed Simulation of Analog Audio Circuits: A Port-Hamiltonian Approach. Appl. Sci., 6.10.3390/app6100273 ↗
11. Schattschneider, J., and Zölzer, U. (1999, January 9–11). Discrete-Time Models for Nonlinear Audio Systems. Proceedings of the 2nd COST G-6 Workshop on Digital Audio Effects (DAFx99), Trondheim, Norway. Available online: https://www.dafx.de/paper-archive/1999/schattschneider.pdf.
12. Fractal Audio Systems (2021, June 23). Multipoint Iterative Matching and Impedance Correction Technology (MIMIC™). Available online: https://www.fractalaudio.com/downloads/manuals/axe-fx-2/Fractal-Audio-Systems-MIMIC-(tm)-Technology.pdf.
13. Kemper, C.K.G. (2021, June 23). Musical Instrument with Acoustic Transducer. Available online: https://worldwide.espacenet.com/patent/search/family/038596197/publication/US2008134867A1?q=pn.
14. Eichas, F., Möller, S., and Zölzer, U. (December, January 30). Block-oriented Modeling of Distortion Audio Effects using Iterative Minimization. Proceedings of the 18th International Conference on Digital Audio Effects, Trondheim, Norway. Available online: https://www.ntnu.edu/documents/1001201110/1266017954/DAFx-15_submission_21.pdf.
15. Eichas, F., Möller, S., and Zölzer, U. (2017, January 5–9). Block-oriented Gray Box Modeling of Guitar Amplifiers. Proceedings of the 20th International Conference on Digital Audio Effects, Edinburgh, UK. Available online: http://www.dafx17.eca.ed.ac.uk/papers/DAFx17_paper_35.pdf.
16. Gray-box modeling of guitar amplifiers10.17743/jaes.2018.0052 ↗
17. Kemp, M.J. (2022, February 02). Analysis and Simulation of Non-Linear Audio Processes using Finite Impulse Responses Derived at Multiple Impulse Amplitudes. The 106th AES Convention, Available online: http://www.sintefex.com/docs/appnotes/dynaconv.PDF.
18. Gillespie, D.J., and Ellis, D.P. (2013, January 20–23). Modeling Nonlinear Circuits with Linearized Dynamical Models via Kernel Regression. Proceedings of the 2013 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, New Paltz, NY, USA .10.1109/WASPAA.2013.6701830 ↗
19. Rugh, W.J. (1981). Nonlinear System Theory: The Volterra/Wiener Approach, The Johns Hopkins University Press.
20. Hélie, T. (2006, January 18–20). On the Use of Volterra Series for Real-time Simulations of Weakly Nonlinear Analog Audio Devices: Application to the Moog Ladder Filter. Proceedings of the 9th International Conference on Digital Audio Effects, DAFx, Montreal, QC, Canada. Available online: http://articles.ircam.fr/textes/Helie06a/index.pdf.
21. Eichas, F., and Zölzer, U. (2016, January 5–9). Black-box Modeling of Distortion Circuits with Block-oriented Models. Proceedings of the 19th International Conference on Digital Audio Effects, DAFx, Brno, Czech Republic. Available online: http://dafx.de/paper-archive/2016/dafxpapers/06-DAFx-16_paper_16-PN.pdf.
22. Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press. Available online: http://www.deeplearningbook.org.
23. Wright, A., Damskägg, E.P., Juvela, L., and Välimäki, V. (2020). Real-time Guitar Amplifier Emulation with Deep Learning. Appl. Sci., 10.10.3390/app10030766 ↗
24. Society, A.E. (2022, May 30). AES Recommended Practicefor Professional Digital Audio–Preferred Sampling Frequencies for Applications Employing Pulse-Code Modulation. Available online: https://www.aes.org/publications/standards/search.cfm?docID=14.
25. Damskägg, E.P., Juvela, L., Thuillier, E., and Välimäki, V. (2019, January 12–17). Deep Learning for Tube Amplifier Emulation. Proceedings of the 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK. Available online: https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8682805.10.1109/ICASSP.2019.8682805 ↗
26. Damskägg, E.P., Juvela, L., and Välimäki, V. (2019, January 19–20). Real-time Modeling of Audio Distortion Circuits with Deep Learning. Proceedings of the Sound and Music Computing Conferences, Stockholm, Sweden. Available online: http://smc2019.uma.es/articles/S5/S5_02_SMC2019_paper.pdf.
27. van den Oord, A., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., and Kavukcuoglu, K. (2016). WaveNet: A Generative Model for Raw Audio. arXiv.
28. Steinmetz, C.J., and Reiss, J.D. (2021). Efficient Neural Networks for Real-time Analog Audio Effect Modeling. arXiv.
29. Perez, E., Strub, F., De Vries, H., Dumoulin, V., and Courville, A. (2017, January 4–9). FiLM: Visual Reasoning with a General Conditioning Layer. Proceedings of the 32nd AAAI Conference on Artificial Intelligence, AAAI 2018, San Francisco, CA, USA.10.1609/aaai.v32i1.11671 ↗
30. Covert, J., and Livingston, D.L. (2013, January 4–7). A Vacuum-tube Guitar Amplifier Model using a Recurrent Neural Network. Proceedings of the Conference Proceedings–IEEE Southeastcon, Jacksonville, FL, USA. Available online: https://ieeexplore.ieee.org/document/6567472.10.1109/SECON.2013.6567472 ↗
31. Zhang, Z., Olbrych, E., Bruchalski, J., McCormick, T.J., and Livingston, D.L. (2018, January 19–22). A Vacuum-Tube Guitar Amplifier Model Using Long/Short-Term Memory Networks. Proceedings of the Conference Proceedings–IEEE Southeastcon, St. Petersburg, FL, USA. Available online: https://ieeexplore.ieee.org/document/8479039.10.1109/SECON.2018.8479039 ↗
32. Learning to Forget: Continual Prediction with LSTM10.1162/089976600300015015 ↗
33. Wright, A., Damskägg, E.P., and Välimäki, V. (2019, January 2–6). Real-Time Black-Box Modelling with Recurrent Neural Networks. Proceedings of the 22nd International Conference on Digital Audio Effects (DAFx-19), Birmingham, UK. Available online: http://dafx.de/paper-archive/2019/DAFx2019_paper_43.pdf.
34. Cho, K., van Merriënboer, B., Bahdanau, D., and Bengio, Y. (2014, January 25). On the Properties of Neural Machine Translation: Encoder–decoder Approaches. Proceedings of the SSST 2014—8th Workshop on Syntax, Semantics and Structure in Statistical Translation, Doha, Qatar. Available online: https://arxiv.org/abs/1409.1259.10.3115/v1/W14-4012 ↗
35. Schmitz, T. (2019). Nonlinear Modeling of the Guitar Signal Chain Enabling its Real-Time Emulation. [Ph.D. Thesis, University of Liège]. Available online: https://pdfs.semanticscholar.org/18e8/0acdd9d704a61a1f174a2a4a1a9411801785.pdf.
36. Schmitz, T., and Embrechts, J.J. (2018). Real Time Emulation of Parametric Guitar Tube Amplifier with LSTM Neural Network. arXiv.10.5121/csit.2018.80511 ↗
37. Ramírez, M.A.M., Benetos, E., and Reiss, J.D. (2019). A General-purpose Deep Learning Approach to Model Time-varying Audio Effects. arXiv.
38. Ramírez, M.A.M., Benetos, E., and Reiss, J.D. (2020). Deep Learning for Black-box Modeling of Audio Effects. Appl. Sci., 10.10.3390/app10020638 ↗
39. Martinez Ramirez, M.A., and Reiss, J.D. (2019, January 12–17). Modeling Nonlinear Audio Effects with End-to-end Deep Neural Networks. Proceedings of the ICASSP 2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK. Available online: https://ieeexplore.ieee.org/document/8683529.10.1109/ICASSP.2019.8683529 ↗
40. Kehling, C., Männchen, A., and Eppler, A. (2022, March 17). IDMT-SMT-Audio-Effects. Available online: https://www.idmt.fraunhofer.de/en/publications/datasets/audio_effects.html.
41. Parker, J.D., Esqueda, F., and Bergner, A. (2019, January 2–6). Modelling of Nonlinear State-Space Systems using a Deep Neural Network. Proceedings of the 22nd International Conference on Digital Audio Effects, DAFx, Birmingham, UK. Available online: http://dafx.de/paper-archive/2019/DAFx2019_paper_42.pdf.
42. Peussa, A. (2020). State-Space Virtual Analog Modelling of Audio Circuits. [Master’s Thesis, Aalto University]. Available online: https://aaltodoc.aalto.fi/handle/123456789/97585.
43. Nercessian, S., Sarroff, A., and Werner, K.J. (2021, January 6–11). Lightweight and Interpretable Neural Modeling of an Audio Distortion Effect using Hyperconditioned Differentiable Biquads. Proceedings of the CASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada. Available online: https://arxiv.org/pdf/2103.08709.pdf.10.1109/ICASSP39728.2021.9413996 ↗
44. Engel, J., Hantrakul, L., Gu, C., Roberts, A., Team, B., and View, M. (2020). DDSP: Differentiable Digital Signal Processing. arXiv.
45. Lyons, R. (2001). Understanding Digital Signal Processing, Bernard Goodwin. Available online: https://www.mikrocontroller.net/attachment/341426/Understanding_digital_signal_processing.pdf.
46. Esqueda, F., Kuznetsov, B., and Parker, J.D. (2021, January 8–10). Differentiable White-Box Virtual Analog Modeling. Proceedings of the 24th International Conference on Digital Audio Effects, Vienna, Austria. Available online: https://dafx2020.mdw.ac.at/proceedings/papers/DAFx20in21_paper_39.pdf.10.23919/DAFx51585.2021.9768272 ↗
47. Fonseca, E., Pons Puig, J., Favory, X., Font Corbera, F., Bogdanov, D., Ferraro, A., Oramas, S., Porter, A., and Serra, X. (2017, January 23–27). Freesound Datasets: A Platform for the Creation of Open Audio Datasets. Proceedings of the International Society for Music Information Retrieval (ISMIR), Suzhou, China. Available online: https://repositori.upf.edu/handle/10230/33299#.YjNpl3P9P-E.mendeley.
48. Kehling, C., Männchen, A., and Eppler, A. (2022, March 17). IDMT-SMT-Guitar. Available online: https://www.idmt.fraunhofer.de/en/publications/datasets/guitar.html.
49. Kehling, C., Männchen, A., and Eppler, A. (2022, March 17). IDMT-SMT-Bass. Available online: https://www.idmt.fraunhofer.de/en/publications/datasets/bass.html.
50. Schmitz, T., and Embrechts, J.J. (2018, January 17–20). Introducing a Dataset of Guitar Amplifier Sounds for Nonlinear Emulation Benchmarking. Proceedings of the Audio Engineering Society 145th Conference (AES 145), New York, USA. Available online: http://www.aes.org/e-lib/browse.cfm?elib=19754&fbclid=IwAR0fXUmzclYzhp_wIPDP1T6ihtC2IOTuVYo8wh0TegMohS9P2fwHtjTiJCM.
51. Hawley, S., Colburn, B., and Mimilakis, S.I. (2020, January 16–19). SignalTrain LA2A Dataset. Proceedings of the Audio Engineering Society 147th Conference (AES 147), New York, NY, USA. Available online: https://doi.org/10.5281/zenodo.3824876#.YjNzFbJm-jk.mendeley.
52. Loudness, Its Definition, Measurement and Calculation10.1121/1.1915637 ↗
53. Wright, A., and Välimäki, V. (2020, January 4–8). Perceptual loss function for neural modeling of audio systems. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Barcelona, Spain. Available online: https://acris.aalto.fi/ws/portalfiles/portal/43946100/ICASSP_2020_RNN_Loss_Functions.pdf.10.1109/ICASSP40776.2020.9052944 ↗
54. Fast Spectrogram Inversion using Multi-head Convolutional Neural Networks
55. ITU-R (2015). ITU-R BS.1534-3: Method for the Subjective Assessment of Intermediate Quality Level of Audio Systems. Intern. Telecom. Union Radiocom. Sect., BS Series, 34. Available online: https://www.itu.int/dms_pubrec/itu-r/rec/bs/R-REC-BS.1534-3-201510-I!!PDF-E.pdf.
56. Jillings, N., Moffat, D., De Man, B., and Reiss, J. (–1, January 25). Web Audio Evaluation Tool: A Browser-Based Listening Test Environment. Proceedings of the Sound and Music Computing, Maynooth, Ireland. Available online: https://www.researchgate.net/publication/282328219_Web_Audio_Evaluation_Tool_A_Browser-Based_Listening_Test_Environment.
57. ITU-R (2015). ITU-R BS.1770-4. Algorithms to Measure Audio Programme Loudness and True-peak Audio Level BS Series Broadcasting Service (sound). Intern. Telecom. Union Radiocom. Sect., 4, Available online: https://www.itu.int/dms_pubrec/itu-r/rec/bs/R-REC-BS.1770-4-201510-I!!PDF-E.pdf.
58. webMUSHRA—A Comprehensive Framework for Web-based Listening Tests10.5334/jors.187 ↗
59. Steinmetz, C.J., and Reiss, J.D. (2020, January 15). auraloss: Audio focused loss functions in PyTorch. Proceedings of the Digital Music Research Network One-day Workshop (DMRN+15), London, UK. Available online: https://github.com/csteinmetz1/auraloss.
60. Carvalho, D.V., Pereira, E.M., and Cardoso, J.S. (2019). Machine Learning Interpretability: A survey on Methods and Metrics. Electronics, 8.10.3390/electronics8080832 ↗