Spectral Envelope Transformation in Singing Voice for Advanced Pitch Shifting
Source abstract
Abstract
The aim of the present work is to perform a step towards more natural pitch shifting techniques in singing voice for its application in music production and entertainment systems. In this paper, we present an advanced method to achieve natural modifications when applying a pitch shifting process to singing voice by modifying the spectral envelope of the audio excerpt. To this end, an all-pole model has been selected to model the spectral envelope, which is estimated using a constrained non-linear optimization. The analysis of the global variations of the spectral envelope was carried out by identifying changes of the parameters of the model along with the changes of the pitch. With the obtained spectral envelope transformation functions, we applied our pitch shifting scheme to some sustained vowels in order to compare results with the same transformation made by using the Flex Pitch plugin of Logic Pro X and pitch synchronous overlap and add technique (PSOLA). This comparison has been carried out by means of both an objective and a subjective evaluation. The latter was done with a survey open to volunteers on our website.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Music and Audio Processing · Speech and Audio Processing · Music Technology and Sound Studies
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
ATIC Research Group, Dept. Ingeniería de Comunicaciones, E.T.S.I. Telecomunicación, Universidad de Málaga, Andalucía Tech, Campus de Teatinos s/n, Málaga 29071, Spain
Funding
Ministerio de Economia y Competitividad of the Spanish Government · Universidad de Malaga, Campus de Excelencia Internacional (CEI) Andalucia TECH
References (39)
Show source-provided references
1. Celemony Software: Melodyne Editor. Available online: http://www.celemony.com.
2. Kenmochi, H., and Ohshita, H. (2007, January 27–31). VOCALOID—Commercial singing synthesizer based on sample concatenation. Proceedings of INTERSPEECH-2007, Antwerp, Belgium.
3. Nakano, T., and Goto, M. (2009, January 23–25). VocaListener: A singing-to-singing synthesis system based on iterative parameter estimation. Proceedings of the 6th Sound and Music Computing Conference, Porto, Portugal.
4. Röbel, A. (2005, January 20–22). Efficient Spectral Envelope Estimation and its application to pitch shifting and envelope preservation. Proceedings of the 8th International Conference on Digital Audio Effects, Madrid, Spain.
5. Roucos, S., and Wilgus, A.M. (1985, January 26–29). High quality time-scale modification for speech. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP’85), Tampa, FL, USA.
6. Hamon, C., Mouline, E., and Charpentier, F. (1989, January 23–26). A diphone synthesis system based on time-domain prosodic modifications of speech. Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP-89), Glasgow, UK.
7. Charpentier, F.J., and Stella, M. (1986, January 7–11). Diphone synthesis using an overlap-add technique for speech waveforms concatenation. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP’86), Tokyo, Japan.10.1109/ICASSP.1986.1168657 ↗
8. Verhelst, W., and Roelands, M. (1993, January 27–30). An overlap-add technique based on waveform similarity (WSOLA) for high quality time-scale modification of speech. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP-93), Minneapolis, MN, USA.10.1109/ICASSP.1993.319366 ↗
9. Pitch-synchronous waveform processing techniques for text-to-speech synthesis using diphones10.1016/0167-6393(90)90021-Z ↗
10. Chládková, K., Boersma, P., and Podlipský, V.J. (2009, January 6–10). Online formant shifting as a function of F0. Proceedings of INTERSPEECH-2009, Brighton, UK.10.21437/Interspeech.2009-164 ↗
11. Formants of children, women and men: The effects of vocal intensity variation10.1121/1.427150 ↗
12. Zölzer, U. (2011). DAFX: Digital Audio Effects, John Wiley & Sons.10.1002/9781119991298 ↗
13. Development of an F0 control model based on F0 dynamic characteristics for singing-voice synthesis10.1016/j.specom.2005.01.010 ↗
14. Molina, E., Barbancho, I., Barbancho, A.M., and Tardón, L.J. (2014, January 4–9). Parametric model of spectral envelope to synthesize realistic intensity variations in singing voice. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP2014), Florence, Italy.10.1109/ICASSP.2014.6853673 ↗
15. Linear predictive coding10.1109/45.1890 ↗
16. Atal, B.S., and Remde, J.R. (1982, January 3–5). A new model of LPC excitation for producing natural-sounding speech at low bit rates. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP’82), Paris, France.
17. The computation of line spectral frequencies using Chebyshev polynomials10.1109/TASSP.1986.1164983 ↗
18. Bonada, J., Celma, Ò., Loscos, À., Ortolà, J., Serra, X., Yoshioka, Y., Kayama, H., Hisaminato, Y., and Kenmochi, H. (2001, January 17–22). Singing voice synthesis combining excitation plus resonance and sinusoidal plus residual models. Proceedings of the International Computer Music Conference, La Habana, Cuba.
19. Software for a cascade/parallel formant synthesizer10.1121/1.383940 ↗
20. Formant location from LPC analysis data10.1109/89.222882 ↗
21. Combining Auditory Preprocessing and Bayesian Estimation for Robust Formant Tracking10.1109/TASL.2009.2025536 ↗
22. Accuracy of formant measurement for synthesized vowels using the reassigned spectrogram and comparison with linear prediction10.1121/1.3308476 ↗
23. Bonada, J., Serra, X., Amatriain, X., and Loscos, A. (2011). DAFX: Digital Audio Effects, John Wiley & Sons. [2nd ed.].
24. Dissonance reduction in polyphonic audio using harmonic reorganization10.1109/TASLP.2013.2287056 ↗
25. Age, sex, and vowel dependencies of acoustic measures related to the voice source10.1121/1.2697522 ↗
26. Integrating a robust option into a multiple regression computing environment
27. Ohmura, H., and Tanaka, K. (1996, January 3–6). Speech synthesis using a nonlinear energy damping model for the vocal folds vibration effect. Proceedings of the Fourth International Conference on Spoken Language Processing (ICSLP 96), Philadelphia, PA, USA.10.21437/ICSLP.1996-320 ↗
28. Linear least squares regression10.1214/aoms/1177698603 ↗
29. Catford, J.C. (1988). A Practical Introduction to Phonetics, Clarendon Press.
30. Mattheyses, W., Verhelst, W., and Verhoeve, P. (2006, January 28–29). Robust pitch marking for prosodic modification of speech using TD-PSOLA. Proceedings of the IEEE Benelux/DSP Valley Signal Processing Symposium, SPS-DARTS, Antwerp, Belgium.
31. Lin, C.Y., and Jang, J.S.R. (2004, January 4–8). A two-phase pitch marking method for TD-PSOLA synthesis. Proceedings of INTERSPEECH-2004, Jeju Island, Korea.10.21437/Interspeech.2004-443 ↗
32. YIN, a fundamental frequency estimator for speech and music10.1121/1.1458024 ↗
33. Melody Transcription From Music Audio: Approaches and Evaluation10.1109/TASL.2006.889797 ↗
34. Ryynänen, M., and Klapuri, A. (2004, January 2–3). Modelling of note events for singing transcription. Proceedings of the ISCA Tutorial and Research Workshop on Statistical and Perceptual Audio Processing, Jeju Island, Korea.
35. Drugman, T., and Dutoit, T. (2009, January 6–10). Glottal closure and opening instant detection from speech signals. Proceedings of INTERSPEECH-2009, Brighton, UK.10.21437/Interspeech.2009-47 ↗
36. Ykhlef, F., and Bendaouia, L. (2013, January 9–11). Pitch Marking Using the Fundamental Signal for Speech Modifications via TD-PSOLA. Proceedings of the 2013 IEEE International Symposium on Multimedia (ISM), Anaheim, CA, USA.10.1109/ISM.2013.28 ↗
37. Dutilleux, P., De Poli, G., von dem Knesebeck, A., and Zoelzer, U. (2002). DAFX: Digital Audio Effects, John Wiley & Sons. [2nd ed.].
38. Logic Pro X. Available online: http://www.apple.com/logic-pro/.
39. Radiocommunication Sector of the International Telecommunication Union (ITU-R) Method for the subjective assessment of intermediate quality level of audio systems. Broadcasting Service (sound) (BS) Series. Available online: https://www.itu.int/dms_pubrec/itu-r/rec/bs/R-REC-BS.1534-3-201510-I!!PDF-E.pdf.