A Novel Reconstruction Method for Irregularly Sampled Observation Sequences for Digital Twin
Source abstract
Abstract
Various uncertainties such as communication delay, packet loss and disconnection in the Industrial Internet, as well as the asynchronous sampling of sensors, can cause irregularity, sparsity, and misalignment of sampling sequences, and thereby seriously affect the training and prediction performance of a digital twin model. Sequence reconstruction is an effective way to deal with the above problems, but if the measurement data become sparse or contain significant noise due to packet loss and electromagnetic interference, existing methods struggle to achieve ideal results. Therefore, a novel variational autoencoder model based on a parallel reference network and neural controlled differential equation (PRN-NCDE) is proposed in this article to solve the problem of reconstructing irregular series under sparse measurements and high noise levels. First, a multi-channel self-attention module is established, which can not only analyze the position and feature information of the sampled data to improve the reconstruction accuracy under sparse measurements, but also effectively tackle the misalignment and irregularity of the observation sequence through multi-channel and mask mechanisms. Second, to improve the accuracy of sequence reconstruction under large noise levels, a PRN is established to obtain reference features, which are weighted and fused with the features of observed data. Third, we use the NCDE to construct a decoder that can combine the control input of the system to predict the output values to solve the problem of sequence reconstruction in a controlled system. Finally, a weighted loss function is constructed to better train the network parameters of the model. This article takes the furnace of the boiler system in a coal-fired power plant as the test object to verify the effectiveness and fitting accuracy of the proposed PRN-NCDE model compared to the existing methods for a controlled system under sparse measurements and large noise levels. Simulation results show that the proposed PRN-NCDE model can improve the estimation accuracy by more than 50% and 70% compared with the recurrent neural network-NCDE (RNN-NCDE) under different sampling numbers and noise levels, and by more than 80% and 60% compared with the recurrent neural network-NODE (RNN-NODE).
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Machine Learning in Healthcare · Advanced Data Processing Techniques
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Faculty of Electronic and Information Engineering, Xi’an Jiaotong University, Xi’an 710049, China
Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100089, China
Funding
National Natural Science Foundation of China · China Scholarship Council
References (33)
Show source-provided references
1. Mikołajewska, E., Mikołajewski, D., Mikołajczyk, T., and Paczkowski, T. (2025). Generative AI in AI-Based Digital Twins for Fault Diagnosis for Predictive Maintenance in Industry 4.0/5.0. Appl. Sci., 15.10.3390/app15063166 ↗
2. Digital Twins: Review and Challenges10.1115/1.4050244 ↗
3. On the reconstruction of irregularly sampled time series10.1086/316495 ↗
4. Kreindler, D., and Lumsden, C. (2010). The effects of the irregular sample and missing data in time series analysis. Nonlinear Dynamical Systems Analysis for the Behavioral Sciences Using Real Data, CRC Press.
5. Fast piecewise polynomial fitting of time-series data for streaming computing10.1109/ACCESS.2020.2976494 ↗
6. Maximum likelihood fitting of ARMA models to time series with missing observations10.1080/00401706.1980.10486171 ↗
7. Adaptive order polynomial fitting: Bandwidth robustification and bias reduction10.1080/10618600.1995.10474678 ↗
8. Polynomial fitting algorithm based on neural network10.52810/TPRIS.2021.100019 ↗
9. Fitting discrete polynomial curve and surface to noisy data10.1007/s10472-014-9425-7 ↗
10. Kleinbaum, D.G., Dietz, K., Gail, M., Klein, M., and Klein, M. (2010). Logistic Regression, Springer.10.1007/978-1-4419-1742-3 ↗
11. Marlin, B.M., Kale, D.C., Khemani, R.G., and Wetzel, R.C. (2012, January 28–30). Unsupervised Pattern Discovery in Electronic Health Care Data Using Probabilistic Clustering Models. Proceedings of the 2nd ACM SIGHIT International Health Informatics Symposium, Miami, FL, USA.10.1145/2110363.2110408 ↗
12. Shukla, S., and Marlin, B. (2020). A survey on principles, models and methods for learning from irregularly sampled time series. arXiv.
13. Estimating Missing Data in Temporal Data Streams Using Multi-Directional Recurrent Neural Networks10.1109/TBME.2018.2874712 ↗
14. Shukla, S., and Marlin, B. (2019). Interpolation-prediction networks for irregularly sampled time series. arXiv.
15. Tan, Q., Ye, M., Yang, B., Liu, S.Q., Ma, A.J., Yip, T.C.F., Wong, G.L.H., and Yuen, P.C. (2020, January 7–12). DATA-GRU: Dual-Attention Time-Aware Gated Recurrent Unit for Irregular Multivariate Time Series. Proceedings of the 36th AAAI Conference on Artificial Intelligence, New York, NY, USA.
16. Zhang, Y., Yang, X., Ivy, J., and Chi, M. (2019, January 10–16). ATTAIN: Attention-based time-aware LSTM networks for disease progression modeling. Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao, China.10.24963/ijcai.2019/607 ↗
17. Shukla, S., and Marlin, B. (2021). Multi-time attention networks for irregularly sampled time series. arXiv.
18. Donahue, J., Krähenbühl, P., and Darrell, T. (2016). Adversarial feature learning. arXiv.
19. Cao, W., Wang, D., Li, J., Zhou, H., Li, L., and Li, Y. (2018, January 3–8). BRITS: Bidirectional Recurrent Imputation for Time Series. Proceedings of the International Conference on Neural Information Processing Systems, Montréal, QC, Canada.
20. Scalable and accurate deep learning with electronic health records10.1038/s41746-018-0029-1 ↗
21. Mozer, M., Kazakov, D., and Lindsey, R. (2017). Discrete event, continuous time RNNs. arXiv.
22. Chen, R.T.Q., Rubanova, Y., Bettencourt, J., and Duvenaud, D.K. (2018, January 3–8). Neural ordinary differential equations. Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montreal, QC, Canada.
23. Rubanova, Y., Chen, R.T.Q., and Duvenaud, D. (2019, January 8–14). Latent ordinary differential equations for irregularly-sampled time series. Proceedings of the 33rd Conference on Neural Information Processing Systems, Vancouver, BC, Canada.
24. De Brouwer, E., Simm, J., Arany, A., and Moreau, Y. (2019, January 8–14). GRU-ODE-Bayes: Continuous modeling of sporadically-observed time series. Proceedings of the 33rd International Conference on Neural Information Processing Systems, Vancouver, BC, Canada.
25. Huang, Z., Sun, Y., and Wang, W. (2020, January 6–12). Learning continuous system dynamics from irregularly-sampled partial observations. Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, BC, Canada.
26. Kidger, P., Morrill, J., Foster, J., and Lyons, T. (2020, January 6–12). Neural controlled differential equations for irregular time series. Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, BC, Canada.
27. Doersch, C. (2016). Tutorial on variational autoencoders. arXiv.
28. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., and Polosukhin, I. (2017, January 4–9). Attention is all you need. Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA.
29. Morrill, J., Kidger, P., Yang, L., and Lyons, T. (2021). Neural controlled differential equations for online prediction tasks. arXiv.
30. Kingma, D., and Welling, M. (2013). Auto-encoding variational Bayes. arXiv.
31. Lv, C. (2002). System Simulation and Modelling of Large Thermal Power Unit, Tsinghua University Press.
32. Zhao, Y., Cai, Y., and Jiang, H. (2023). Recurrent neural network-based hybrid modeling method for digital twin of boiler system in coal-fired power plant. Appl. Sci., 13.10.3390/app13084905 ↗
33. A generalized data assimilation architecture of digital twin for complex process industrial systems10.1088/1361-6501/ad27c6 ↗