journal article·2024-06-01Open access

Pyramid Feature Attention Network for Speech Resampling Detection

Xinyu Zhou, Yujin Zhang, Yongqi Wang, Jin Tian, Shaolun Xu

Applied Sciences· Volume 14· Issue 11

Abstract

Speech forgery and tampering, increasingly facilitated by advanced audio editing software, pose significant threats to the integrity and privacy of digital speech avatars. Speech resampling is a post-processing operation of various speech-tampering means, and the forensic detection of speech resampling is of great significance. For speech resampling detection, most of the previous works used traditional methods of feature extraction and classification to distinguish original speech from forged speech. In view of the powerful ability of deep learning to extract features, this paper converts the speech signal into a spectrogram with time-frequency characteristics, and uses the feature pyramid network (FPN) with the Squeeze and Excitation (SE) attention mechanism to learn speech resampling features. The proposed method combines the low-level location information and the high-level semantic information, which dramatically improves the detection performance of speech resampling. Experiments were carried out on a resampling corpus made on the basis of the TIMIT dataset. The results indicate that the proposed method significantly improved the detection accuracy of various resampled speech. For the tampered speech with a resampling factor of 0.9, the detection accuracy is increased by nearly 20%. In addition, the robustness test demonstrates that the proposed model has strong resistance to MP3 compression, and the overall performance is better than the existing methods.
Source: crossref · Quality: baseline

Research context

OpenAlex topics: Digital Media Forensic Detection · Speech and Audio Processing · Speech Recognition and Synthesis

SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.

Affiliations

School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai 201620, China

School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China

Funding

National Natural Science Foundation of China · Natural Science Foundation of Shanghai · Opening Project of Shanghai Key Laboratory of Integrated Administration Technologies for Information Security · Innovation Fund for Industry-University-Research of Chinese Universities

References (44)

Show source-provided references
1. (2023, December 17). Audacity: Free Audio Editor and Recorder. Available online: http://www.audacityteam.org/.
2. (2023, August 06). Cool Edit Pro Is Now Adobe Audition. Available online: http://www.adobe.com/products/audition.html.
3. (2024, February 03). Gold Wave-Audio Editor, Recorder, Converter, Restoration, and Analysis Software. Available online: http://www.goldwave.ca/.
4. Detection of speech smoothing on very short clips10.1109/TIFS.2019.2900935 ↗
5. Bevinamarad, P.R., and Shirldonkar, M. (2020, January 15–17). Audio forgery detection techniques: Present and past review. Proceedings of the 2020 4th International Conference on Trends in Electronics and Informatics (ICOEI)(48184), Tirunelveli, India.10.1109/ICOEI48184.2020.9143014 ↗
6. Detection of impostor and tampered segments in audio by using an intelligent system10.1016/j.compeleceng.2021.107122 ↗
7. Saleem, S., Dilawari, A., and Khan, U.G. (2021, January 5–7). Spoofed voice detection using dense features of stft and mdct spectrograms. Proceedings of the 2021 International Conference on Artificial Intelligence (ICAI), Islamabad, Pakistan.10.1109/ICAI52203.2021.9445259 ↗
8. Capoferri, D., Borrelli, C., Bestagini, P., Antonacci, F., Sarti, A., and Tubaro, S. (2020, January 6–11). Speech audio splicing detection and localization exploiting reverberation cues. Proceedings of the 2020 IEEE International Workshop on Information Forensics and Security (WIFS), New York, NY, USA.10.1109/WIFS49906.2020.9360900 ↗
9. Huang, X., Liu, Z., Lu, W., Liu, H., and Xiang, S. (2020). Digital Forensics and Forensic Investigations: Breakthroughs in Research and Practice, IGI Global.
10. Zhao, J., Lu, B., Huang, L., Huang, M., and Huang, J. (2022, January 21–22). Digital audio tampering detection using ENF feature and LST-MInception net. Proceedings of the AIIPCC 2022; The Third International Conference on Artificial Intelligence, Information Processing and Cloud Computing, Online.
11. Gallagher, A.C. (2005, January 9–11). Detection of linear and cubic interpolation in JPEG compressed images. Proceedings of the 2nd Canadian Conference on Computer and Robot Vision (CRV’05), Victoria, BC, Canada.
12. Blind authentication using periodic properties of interpolation10.1109/TIFS.2004.924603 ↗
13. Audio re-sampling detection in audio forensics based on EM algorithm
14. A method of detecting re-sampling based on expectation maximization applied in audio blind forensics
15. Shi, Q., and Ma, X. (2011, January 27–30). Detection of audio interpolation based on singular value decomposition. Proceedings of the 2011 3rd International Conference on Awareness Science and Technology (iCAST), Dalian, China.
16. Audio re-sampling detection in audio forensics based on second-order derivative
17. Exposing digital forgeries by detecting traces of resampling10.1109/TSP.2004.839932 ↗
18. Speech resampling detection based on inconsistency of band energy
19. Zhang, Y., Dai, S., Song, W., Zhang, L., and Li, D. (2019). Exposing speech resampling manipulation by local texture analysis on spectrogram images. Electronics, 9.10.3390/electronics9010023 ↗
20. Multiscale Attention Network for Detection and Localization of Image Splicing Forgery
21. MFL Image Recognition Method of Pipeline Corrosion Defects Based on Multilayer Feature Fusion Multiscale GhostNet10.1109/TIM.2022.3199247 ↗
22. Wani, T.M., Gunawan, T.S., Qadri, S.A.A., Mansor, H., Arifin, F., and Ahmad, Y.A. (2021, January 23–25). Stride Based Convolutional Neural Network for Speech Emotion Recognition. Proceedings of the 2021 IEEE 7th International Conference on Smart Instrumentation, Measurement and Applications (ICSIMA), Bandung, Indonesia.10.1109/ICSIMA50015.2021.9526320 ↗
23. A Robust Bio-Signal Steganography With Lost-Data Recovery Architecture Using Deep Learning
24. ARWGAN: Attention-Guided Robust Image Watermarking Model Based on GAN
25. Küçükuğurlu, B., Ustubioglu, B., and Ulutas, G. (2020, January 7–9). Duplicated Audio Segment Detection with Local Binary Pattern. Proceedings of the 2020 43rd International Conference on Telecommunications and Signal Processing (TSP), Milan, Italy.10.1109/TSP49548.2020.9163568 ↗
26. Exposing speech transsplicing forgery with noise level inconsistency
27. Ulutas, G., Tahaoglu, G., and Ustubioglu, B. (2022, January 13–15). Forge Audio Detection Using Keypoint Features on Mel Spectrograms. Proceedings of the 2022 45th International Conference on Telecommunications and Signal Processing (TSP), Virtual.10.1109/TSP55681.2022.9851327 ↗
28. Jadhav, S., Patole, R., and Rege, P. (2019, January 6–8). Audio splicing detection using convolutional neural network. Proceedings of the 2019 10th International Conference on Computing, Communication and Networking Technologies (ICCCNT), Kanpur, India.10.1109/ICCCNT45670.2019.8944345 ↗
29. Mel spectrogram-based audio forgery detection using CNN10.1007/s11760-022-02436-4 ↗
30. A Deep Learning Approach for Splicing Detection in Digital Audios10.1007/978-981-16-9416-5_39 ↗
31. Hu, J., Shen, L., and Sun, G. (2018, January 18–23). Squeeze-and-Excitation Networks. Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake, UT, USA.10.1109/CVPR.2018.00745 ↗
32. Investigation of the effect of spectrogram images and different texture analysis methods on speech emotion recognition10.1016/j.apacoust.2018.08.003 ↗
33. Spectrogram based multi-task audio classification10.1007/s11042-017-5539-3 ↗
34. Pyrovolakis, K., Tzouveli, P., and Stamou, G. (2022). Multi-modal song mood detection with deep learning. Sensors, 22.10.3390/s22031065 ↗
35. Savić, N., Milivojević, Z., Prlinčević, B., and Kostić, D. (2022, January 26–28). Septic-convolution Kernel-Comparative Analysis of the Interpolation Error. Proceedings of the 2022 International Conference on Development and Application Systems (DAS), Suceava, Romania.10.1109/DAS54948.2022.9786191 ↗
36. A Study on the Impact of Data Augmentation for Training Convolutional Neural Networks in the Presence of Noisy Labels10.1109/SIBGRAPI55357.2022.9991791 ↗
37. Attention-Guided Pyramid Context Network for Polyp Segmentation in Colonoscopy Images
38. He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27–30). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.10.1109/CVPR.2016.90 ↗
39. Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21–26). Feature pyramid networks for object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.10.1109/CVPR.2017.106 ↗
40. Zhang, H., Zu, K., Lu, J., Zou, Y., and Meng, D. (2022, January 4–8). EPSANet: An efficient pyramid squeeze attention block on convolutional neural network. Proceedings of the Asian Conference on Computer Vision, Macao, China.10.1007/978-3-031-26313-2_33 ↗
41. Xiang, Z., Bestagini, P., Tubaro, S., and Delp, E.J. (2022, January 22–27). Forensic Analysis and Localization of Multiply Compressed MP3 Audio Using Transformers. Proceedings of the ICASSP 2022—2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Singapore.10.1109/ICASSP43922.2022.9747639 ↗
42. Hailu, N., Siegert, I., and Nürnberger, A. (2020, January 21–24). Improving automatic speech recognition utilizing audio-codecs for data augmentation. Proceedings of the 2020 IEEE 22nd International Workshop on Multimedia Signal Processing (MMSP), Tampere, Finland.10.1109/MMSP48831.2020.9287127 ↗
43. Guo, T., Wen, C., and Jiang, D. (2021, January 6–11). Didispeech: A large scale mandarin speech corpus. Proceedings of the ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Toronto, ON, Canada.10.1109/ICASSP39728.2021.9414423 ↗
44. Fu, Y., Cheng, L., and Lv, S. (2021). Aishell-4: An open source dataset for speech enhancement, separation, recognition and speaker diarization in conference scenario. arXiv.10.21437/Interspeech.2021-1397 ↗