A Practical Framework for Event-Level Evaluation and Verifiable Counterfactual Explanation in Multivariate Time-Series Anomaly Detection
Source abstract
Abstract
Multivariate time-series anomaly detection is often evaluated with point-adjusted metrics, which can overstate practical performance when alarms are judged at the event level. Explanation results are also frequently reported as descriptive attributions without directly testing whether selected variables are useful for diagnosis. This study revisits these issues through unified event-level evaluation and repair-based explanation, using DCdetector as the main case study rather than proposing a new detector architecture. Experiments on SMAP, MSL, and HAI 21.03 use full-coverage score export and standard event-level control metrics. The results show that point-adjusted scores can be much higher than stricter event-level measurements. Event-aware refinement changes the detection trade-off by improving event recovery and reducing delay in several settings, but its effect is dataset- and calibration-dependent. For explanation, variables are ranked by exact marginal counterfactual repair effect and evaluated by whether repair reduces anomaly scores more than random or heuristic alternatives. The results provide quantitative evidence that the ranked variables are diagnostically informative, while exact marginal verification is computationally expensive and better suited to offline alarm review and post hoc diagnosis than latency-critical deployment. Auxiliary checks with TranAD, Anomaly-Transformer, and DADA support the plausibility of the main observations, but the evidence remains detector-conditioned rather than a fully backbone-agnostic benchmark. Overall, this work provides a stricter and more verifiable protocol for evaluating anomaly detection, event-aware refinement, and explanation quality in multivariate time-series monitoring.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Anomaly Detection Techniques and Applications · Software System Performance and Reliability · Network Security and Intrusion Detection
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
School of Advanced Manufacturing, Guangdong University of Technology, Jieyang 515200, China
References (33)
Show source-provided references
1. Darban, Z.Z., Webb, G.I., Pan, S., Aggarwal, C.C., and Salehi, M. (2024). Deep Learning for Time Series Anomaly Detection: A Survey. ACM Comput. Surv., 57.10.1145/3691338 ↗
2. Su, Y., Zhao, Y., Niu, C., Liu, R., Sun, W., and Pei, D. (2019, January 4–8). Robust Anomaly Detection for Multivariate Time Series through Stochastic Recurrent Neural Network. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Anchorage, AK, USA.10.1145/3292500.3330672 ↗
3. Zhang, C., Song, D., Chen, Y., Feng, X., Lumezanu, C., Cheng, W., Ni, J., Zong, B., Chen, H., and Chawla, N.V. (February, January 27). A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data. Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.
4. Audibert, J., Michiardi, P., Guyard, F., Zuluaga, M.A., and Michiardi, P. (2020, January 23–27). USAD: UnSupervised Anomaly Detection on Multivariate Time Series. Proceedings of the 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA.10.1145/3394486.3403392 ↗
5. Zhao, H., Wang, Y., Duan, J., Huang, C., Cao, D., Tong, Y., Xu, B., Bai, J., Tong, J., and Zhang, Q. (2020, January 17–20). Multivariate Time-Series Anomaly Detection via Graph Attention Network. Proceedings of the 2020 IEEE International Conference on Data Mining (ICDM), Sorrento, Italy.10.1109/ICDM50108.2020.00093 ↗
6. Deng, A., and Hooi, B. (2021, January 2–9). Graph Neural Network-Based Anomaly Detection in Multivariate Time Series. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual.10.1609/aaai.v35i5.16523 ↗
7. Xu, J., Wu, H., Wang, J., and Long, M. (2022, January 25–29). Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy. Proceedings of the International Conference on Learning Representations, Virtual.
8. TranAD: Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data10.14778/3514061.3514067 ↗
9. Yang, Y., Zhang, C., Zhou, T., Wen, Q., and Sun, L. (2023, January 6–10). DCdetector: Dual Attention Contrastive Representation Learning for Time Series Anomaly Detection. Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Long Beach, CA, USA.10.1145/3580305.3599295 ↗
10. Dai, E., and Chen, J. (2022, January 25–29). Graph-Augmented Normalizing Flows for Anomaly Detection of Multiple Time Series. Proceedings of the International Conference on Learning Representations, Virtual.
11. Feng, C., and Tian, P. (2021, January 14–18). Time Series Anomaly Detection for Cyber-physical Systems via Neural System Identification and Bayesian Filtering. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, Singapore.10.1145/3447548.3467137 ↗
12. Chen, W., Tian, L., Chen, B., Dai, L., Duan, Z., and Zhou, M. (2022, January 17–23). Deep Variational Graph Convolutional Recurrent Network for Multivariate Time Series Anomaly Detection. Proceedings of the 39th International Conference on Machine Learning, Baltimore, MD, USA. Proceedings of Machine Learning Research.
13. Li, Y., Chen, W., Chen, B., Wang, D., Tian, L., and Zhou, M. (2023, January 23–29). Prototype-oriented Unsupervised Anomaly Detection for Multivariate Time Series. Proceedings of the 40th International Conference on Machine Learning, Honolulu, HI, USA. Proceedings of Machine Learning Research.
14. Kim, S., Choi, K., Choi, H.S., Lee, B., and Yoon, S. (March, January 22). Towards a Rigorous Evaluation of Time-Series Anomaly Detection. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual.
15. Anomaly Detection in Time Series: A Comprehensive Evaluation10.14778/3538598.3538602 ↗
16. Navigating the Metric Maze: A Taxonomy of Evaluation Metrics for Anomaly Detection in Time Series10.1007/s10618-023-00988-8 ↗
17. Explainable Time Series Anomaly Detection Using Masked Latent Generative Modeling10.1016/j.patcog.2024.110826 ↗
18. Verma, S., Boonsanong, V., Hoang, M., Hines, K.E., Dickerson, J.P., and Shah, C. (2024). Counterfactual Explanations and Algorithmic Recourses for Machine Learning: A Review. ACM Comput. Surv., 56.10.1145/3677119 ↗
19. Ates, E., Aksar, B., Leung, V.J., and Coskun, A.K. (2021, January 19–21). Counterfactual Explanations for Multivariate Time Series. Proceedings of the 2021 International Conference on Applied Artificial Intelligence (ICAPAI), Halden, Norway.10.2172/1866905 ↗
20. Sulem, D., Donini, M., Zafar, M.B., Aubet, F.X., Gasthaus, J., Januschowski, T., Das, S., Kenthapadi, K., and Archambeau, C. (2022). Diverse Counterfactual Explanations for Anomaly Detection in Time Series. arXiv.
21. Ji, X., Xue, A., Wong, E., Sokolsky, O., and Lee, I. (2024, January 10–15). AR-Pro: Counterfactual Explanations for Anomaly Repair with Formal Properties. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada.10.52202/079017-0514 ↗
22. Bahri, O., Li, P., Boubrahimi, S.F., and Hamdi, S.M. (2022, January 12–14). Temporal Rule-Based Counterfactual Explanations for Multivariate Time Series. Proceedings of the 2022 21st IEEE International Conference on Machine Learning and Applications (ICMLA), Nassau, Bahamas.10.1109/ICMLA55696.2022.00200 ↗
23. Lang, J., Giese, M.A., Ilg, W., and Otte, S. (2023). Generating Sparse Counterfactual Explanations for Multivariate Time Series. Artificial Neural Networks and Machine Learning—ICANN 2023, Springer.10.1007/978-3-031-44223-0_15 ↗
24. DeepAnT: A Deep Learning Approach for Unsupervised Anomaly Detection in Time Series10.1109/ACCESS.2018.2886457 ↗
25. Zong, B., Song, Q., Min, M.R., Cheng, W., Lumezanu, C., Zheng, D., and Chen, H. (May, January 30). Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada.
26. Ruff, L., Vandermeulen, R.A., Görnitz, N., Binder, A., Müller, E., Müller, K.R., and Kloft, M. (2018, January 10–15). Deep One-Class Classification. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden.
27. Lin, S., Clark, R., Birke, R., Schönborn, S., Trigoni, N., and Roberts, S. (2020). Anomaly Detection for Time Series Using VAE-LSTM Hybrid Model. arXiv.10.1109/ICASSP40776.2020.9053558 ↗
28. Niu, Z., Yu, K., and Wu, X. (2020). LSTM-Based VAE-GAN for Time-Series Anomaly Detection. Sensors, 20.10.3390/s20133738 ↗
29. Liu, Q., and Paparrizos, J. (2024, January 10–15). The Elephant in the Room: Towards a Reliable Time-Series Anomaly Detection Benchmark. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada.10.52202/079017-3437 ↗
30. Hundman, K., Constantinou, V., Laporte, C., Colwell, I., and Söderström, T. (2018, January 19–23). Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, London, UK.10.1145/3219819.3219845 ↗
31. Shin, H.K., Lee, W., Yun, J.H., and Min, B.G. (2021, January 9). Two ICS Security Datasets and Anomaly Detection Contest on the HIL-Based Augmented ICS Testbed. Proceedings of the Cyber Security Experimentation and Test Workshop (CSET ’21), Virtual, CA, USA.10.1145/3474718.3474719 ↗
32. Temporal-contextual self-supervised time-series learning for automated fault detection and diagnosis of air handling units in buildings10.1016/j.buildenv.2026.114300 ↗
33. Class-aware temporal and contextual contrastive framework for semi-supervised automated fault detection and diagnosis in air handling units10.1016/j.enbuild.2026.117233 ↗