A Recurrent Neural Network for Multimodal Anomaly Detection by Using Spatio-Temporal Audio-Visual Data

1. H. Lin, J. D. Deng, B. J. Woodford, and A. Shahi, “Online weighted clustering for real-time abnormal event detection in video surveillance,” presented at the ACM Int. Conf. Multimed., Amsterdam, Netherlands, Oct. 2016, pp. 536–540.10.1145/2964284.2967279 ↗

2. Combining motion and appearance cues for anomaly detection10.1016/j.patcog.2015.09.005 ↗

3. Stauffer · Automated Audio-Visual Activity Analysis · 2005

4. Using deep autoencoders for in-vehicle audio anomaly detection10.1016/j.procs.2021.08.031 ↗

5. M. Pooyan, X. Zhang, M. Hamidi, and J. Zhang, “Deep learning-based anomaly detection for compressors using audio data,” presented at the Annu. Reliabil. Maintainability Symp. (RAMS), Orlando, FL, USA, Jan. 2021, pp. 1–7.10.1109/RAMS48097.2021.9605720 ↗

6. A novel approach for automatic acoustic novelty detection using a denoising autoencoder with bidirectional LSTM neural networks

7. Towards Data Science, “Denoising Autoencoders (DAE): How to use neural networks to cleanup your data,” Towards Data Science. Accessed: May 20, 2022. [Online]. Available: https://towardsdatascience.com/denoising-autoencoders-dae-how-to-use-neural-networks-to-cleanup-your-data-cd9c19bc6915

8. Towards Data Science, “Sparse autoencoder neural networks: How to utilize sparsity for robust information encoding,” Towards Data Science. Accessed: May 20, 2022. [Online]. Available: https://towardsdatascience.com/sparse-autoencoder-neural-networks-how-to-utilise-sparsity-for-robust-information-encoding-6aa9ff542bc9

9. Keras, “Building Autoencoders in Keras,” Keras. Accessed: May 20, 2022. [Online]. Available: https://blog.keras.io/building-autoencoders-in-keras.html

10. Keras, “Variational Autoencoders (VAE),” Keras. Accessed: May 20, 2022. [Online]. Available: https://keras.io/examples/generative/vae/

11. P. Kumari and M. Saini, “Anomaly detection in audio with concept drift using adaptive Huffman coding,” 2021, arXiv:2102.10515.10.1109/JSEN.2022.3193969 ↗

12. Unsupervised log message anomaly detection10.1016/j.icte.2020.06.003 ↗

13. IRESE: An intelligent rare-event detection system using unsupervised learning on the IoT edge10.1016/j.engappai.2019.05.011 ↗

14. Anomalous sound event detection: A survey of machine learning based methods and applications10.1007/s11042-021-11817-9 ↗

15. Y. Koizumi et al., “Description and discussion on DCASE2020 challenge task2: Unsupervised anomalous sound detection for machine condition monitoring,” 2020, arXiv:2006.05822.

16. G. Pang, Y. Cheng, C. Shen, A. Van den Hengel, and X. Bai, “Self-trained deep ordinal regression for end-to-end video anomaly detection,” presented at the IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Seattle, WA, USA, Jun. 2020, pp. 12173–12182.10.1109/CVPR42600.2020.01219 ↗

17. An adaptive framework for anomaly detection in time-series audio-visual data10.1109/ACCESS.2022.3164439 ↗

18. R. Morais, V. Le, T. Tran, B. Saha, M. Mansour and S. Venkatesh, “Learning regularity in skeleton trajectories for anomaly detection in videos,” presented at the IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Long Beach, CA, USA, Jun. 2019, pp. 11996–12004.10.1109/CVPR.2019.01227 ↗

19. H. -S. Fang, S. Xie, Y. -W. Tai, and C. Lu, “RMPE: Regional multiperson pose estimation,” presented at the IEEE Int. Conf. Comput. Vis. (ICCV), Venice, Italy, Oct. 2017, pp. 2334–2343.10.1109/ICCV.2017.256 ↗

20. Learning phrase representations using RNN encoder-decoder for statistical machine translation

21. A survey of single-scene video anomaly detection

22. Sparse coding guided spatiotemporal feature learning for abnormal event detection in large videos

23. Y. Cong, J. Yuan, and J. Liu, “Sparse reconstruction cost for abnormal event detection,” presented at the IEEE Conf.Comput. Vis. Pattern Recognit. (CVPR), Colorado Springs, CO, USA, Jun. 2011, pp. 3449–3456.10.1109/CVPR.2011.5995434 ↗

24. C. Lu, J. Shi, and J. Jia, “Abnormal event detection at 150 fps in MATLAB,” presented at the IEEE Int. Conf. Comput. Vis. (ICCV), Sydney, Australia, Dec. 2013, pp. 2720–2727.10.1109/ICCV.2013.338 ↗

25. Y. Lu, C. Cao, and Y. Zhang, “Learnable locality-sensitive hashing for video anomaly detection,” 2021, arXiv:2111.07839.

26. Online growing neural gas for anomaly detection in changing surveillance scenes10.1016/j.patcog.2016.09.016 ↗

27. L. Naud and A. Lavin, “Manifolds for unsupervised visual anomaly detection,” 2020, arXiv:2006.11364.

28. P. Wu et al., “Not only look, but also listen: Learning multimodal violence detection under weak supervision,” presented at the Eur. Conf. Comput. Vis. (ECCV), Glasgow, UK, Aug. 2020, pp. 322–339.10.1007/978-3-030-58577-8_20 ↗

29. Multi-modal anomaly detection by using audio and visual cues10.1109/ACCESS.2021.3059519 ↗

30. Deep multimodal learning: A survey on recent advances and trends10.1109/MSP.2017.2738401 ↗

31. R. Gao, T. -H. Oh, K. Grauman, and L. Torresani, “Listen to look: Action recognition by previewing audio,” presented at the IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Seattle, WA, USA, Jun. 2020, pp. 10457–10467.10.1109/CVPR42600.2020.01047 ↗

32. A. R. Lejbølle, B. Krogh, K. Nasrollahi, and T. B. Moeslund, “Attention in multimodal neural networks for person re-identification,” presented at the IEEE Conf. Comput. Vis. Pattern Recognit. Workshops, Salt Lake City, UT, USA, Jun. 2018, pp. 179–187.10.1109/CVPRW.2018.00055 ↗

33. Deep multi-level fusion network for multi-source image pixel-wise classification

34. Deep multiple instance learning-based spatial-spectral classification for PAN and MS imagery

35. Multimodal fusion for multimedia analysis: A survey10.1007/s00530-010-0182-0 ↗

36. K. M. Imran, S. N. Foley, and B. O’Sullivan, “Database Intrusion Detection Systems (DIDs): Insider threat detection via behavioural-based anomaly detection systems—A brief survey of concepts and approaches,” 2020, arXiv:2011.02308.

37. Generalization of spectral flatness measure for non-gaussian linear processes10.1109/LSP.2004.831663 ↗

38. Trends in audio signal feature extraction methods10.1016/j.apacoust.2019.107020 ↗

39. Analytics India Magazine, “A tutorial on spectral feature extraction for audio analytics,” Analytics India Magazine. Accessed: Dec. 20, 2021. [Online]. Available: https://analyticsindiamag.com/a-tutorial-on-spectral-feature-extraction-for-audio-analytics/

40. Analytics India Magazine, “Hands-on guide to implement batch normalization in deep learning models,” Accessed: Jun. 13, 2021. [Online]. Available: https://analyticsindiamag.com/hands-on-guide-to-implement-batch-normalization-in-deep-learning-models/

41. Analytics India Magazine, “Convolutional layer,” Accessed: Jun. 12, 2021. [Online]. Available: https://databricks.com/glossary/convolutional-layer

42. M. L. Value, “Time distributed layer in Keras with example in Python,” Accessed: Jun. 13, 2021. [Online]. Available: https://valueml.com/time-distributed-layer-in-keras-with-example-in-python/

43. M. Hasan, J. Choi, J. Neumann, A. K. Roy Chowdhury, and L. S. Davis, “Learning temporal regularity in video sequences,” presented at the IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Las Vegas, NV, USA, Jun. 2016, pp. 733–742.10.1109/CVPR.2016.86 ↗

44. Towards Data Science, “Prototyping an anomaly detection system for videos: Step-by-step using LSTM & convolutional,” Towards Data Science. Accessed: Jun. 10, 2022. [Online]. Available: https://towardsdatascience.com/prototyping-an-anomaly-detection-system-for-videos-step-by-step-using-lstm-convolutional-4e06b7dcdd29

45. Github, “UCSD-UMN anomalous audio datasets,” Accessed: Aug. 05, 2022. [Online]. Available: https://github.com/tsameema/UCSD-UMN-Anomalous_Audio-Datasets

46. DCASE, “Task 1: Rare sound event detection,” Accessed: Jul. 24, 2021. [Online]. Available: https://dcase.community/challenge2017/taskrare-sound-event-detection

47. Anomaly detection in crowded scenes

48. Machine Learning Mastery, “How to choose loss functions when training deep learning neural networks,” Machine Learning Mastery. Accessed: Jan. 13, 2022. [Online]. Available: https://machinelearningmastery.com/how-to-choose-loss-functions-when-training-deep-learningneural-networks/

49. Analytics Vidhya, “Binary cross-entropy (Log Loss) for binary classification,” Analytics Vidhya. Accessed: Jan. 13, 2022. [Online]. Available: https://www.analyticsvidhya.com/blog/2021/03/binary-cross-entropy-log-loss-for-binary-classification/

50. Analytics Vidhya, “A comprehensive guide on deep learning optimizers,” Analytics Vidhya. Accessed: Jan. 20, 2022. [Online]. Available: https://www.analyticsvidhya.com/blog/2021/10/a-comprehensive-guide-on-deep-learning-optimizers/

51. Towards Data Science, “Adam: Latest trends in deep learning optimization,” Towards Data Science. Accessed: Jan. 20, 2022. [Online]. Available: https://towardsdatascience.com/adam-latest-trends-in-deep-learning-optimization-6be9a291375c

52. N. Rashmiranjan, U. C. Pati, and S. K. Das, “Video anomaly detection using convolutional spatiotemporal autoencoder,” presented at the Int. Conf. Contemp. Comput. Appl. (IC3A), Patna, India, Feb. 2020, pp. 175–180.10.1109/IC3A48958.2020.233292 ↗

53. Anomaly detection in videos using two-stream autoencoder with post hoc interpretability10.1155/2021/7367870 ↗

54. W. Tian, Z. Miao, Y. Chen, Y. Zhou, G. Shan and H. Snoussi, “AED-Net: An abnormal event detection network,” presented at the Eng. Conf., Nanjing, China, Sep. 2019, pp. 930–939.10.1016/j.eng.2019.02.008 ↗

55. Towards zero shot learning of geometry of motion streams and its application to anomaly recognition

56. Video anomaly detection with compact feature sets for online performance10.1109/TIP.2017.2695105 ↗

57. A novel framework for detection of motion and appearance-based anomaly using ensemble learning and LSTMs10.1016/j.eswa.2021.116394 ↗

58. M. R. Parate, K. M. Bhurchandi, and A. G. Kothari, “Anomaly detection in residential video surveillance on edge devices in IoT framework,” 2021, arXiv:2107.04767.

59. Monitoring human activity, Accessed: Jul. 24, 2021. [Online]. Available: http://mha.cs.umn.edu/proj_events.shtml