Tensor Decomposition Through Neural Architectures
Source abstract
Abstract
Machine learning (ML) technologies are currently widely used in many domains of science and technology, to discover models that transform input data into output data. The main advantages of such a procedure are the generality and simplicity of the learning process, while their weaknesses remain the required amount of data needed to perform the training and the recurrent difficulties to explain the involved rationale. At present, a panoply of ML techniques exist, and the selection of a method or another depends, in general, on the type and amount of data being considered. This paper proposes a procedure which provides not a field or an image as an output, but its singular value decomposition (SVD), or an SVD-like decomposition, while injecting as input data scalars or the SVD decomposition of an input field. The result is a tensor-to-tensor decomposition, without the need for the full fields, or an input to an output SVD-like decomposition. The proposed method works for the non-hyper-parallepipedic domain, and for any space dimensionality. The results show the ability of the proposed architecture to link the input filed and output field, without requiring access to full space reconstruction.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Tensor decomposition and applications · Model Reduction and Neural Networks · Power System Optimization and Stability
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
Mechanical Engineering Department, University of North Florida, 1 UNF Drive, Jacksonville, FL 32224, USA
PIMM Research Laboratory, UMR 8006 CNRS-ENSAM-CNAM, Arts et Metiers Institute of Technology, 151 Boulevard de l’Hôpital, 75013 Paris, France
CNRS@CREATE Ltd., 1 Create Way, #08-01 CREATE Tower, Singapore 138602, Singapore
Funding
ESI Group, RTE, and SKF
References (48)
Show source-provided references
1. MacKay, D. (2003). Chapter 20—An Example Inference Task: Clustering. Information Theory, Inference and Learning Algorithms, Cambridge University Press.
2. MacQueen, J. (1967). Some Methods for classification and Analysis of Multivariate Observations. Proceedings of 5th Berkeley Symposium on Mathematical Statistics and Probability, University of California Press.
3. Cristianini, N., and Shawe-Taylor, J. (2000). An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods, Cambridge University Press.10.1017/CBO9780511801389 ↗
4. Kirkwood, C.W. (2025, January 28). Decision Tree Primer. Available online: https://www.public.asu.edu/~kirkwood/DAStuff/refs/decisiontrees/index.html.
5. Random Forests10.1023/A:1010933404324 ↗
6. Goodfellow, I., and Bengio, Y. (2016). Courville, A. Deep Learning, MIT Press.
7. Sancarlos, A., Champaney, V., Duval, J.L., Cueto, E., and Chinesta, F. (2021). PGD-based advanced nonlinear multiparametric regressions for constructing metamodels at the scarce-data limit. arXiv.
8. Chinesta, F., Cueto, E., Champaney, V., Ghnatios, C., Ammlar, A., Hascoet, N., Gonzalez, D., Alfaro, I., Di Lorenzo, D., and Pasquale, A. (2024). A Gentle Introduction on Data, Learning, Model Order Reduction and Twinning Methodologies and Techniques, Springer.
9. Venkatesan, R., and Li, B. (2017). Convolutional Neural Networks in Visual Computing: A Concise Guide, CRC Press.10.4324/9781315154282 ↗
10. Bronstein, M., Bruna, J., Cohen, T., and Velickovic, P. (2021). Geometric Deep Learning, Grids, Groups, Graphs, Geodesics and Gauges. arXiv.
11. Data driven governing equations approximation using deep neural networks10.1016/j.jcp.2019.06.042 ↗
12. Minimal gated unit for recurrent neural networks10.1007/s11633-016-1006-2 ↗
13. Chen, R.T.Q., Rubanova, Y., Bettencourt, J., and Duvenaud, D.K. (2018, January 3–8). Neural Ordinary Differential Equations. Proceedings of the Advances in Neural Information Processing Systems 31 (NeurIPS 2018), Montreal, QC, Canada.
14. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations10.1016/j.jcp.2018.10.045 ↗
15. On learning Hamiltonian systems from data10.1063/1.5128231 ↗
16. Thermodynamically consistent data-driven computational mechanics10.1007/s00161-018-0677-z ↗
17. Greydanus, S., Dzamba, M., and Yosinski, J. (2019). Hamiltonian Neural Networks. arXiv.
18. Deep learning of thermodynamics-aware reduced-order models from data
19. Learning non-Markovian physics from data10.1016/j.jcp.2020.109982 ↗
20. Lee, K., Trask, N.A., and Stinis, P. (2021). Machine learning structure preserving brackets for forecasting irreversible processes. arXiv.
21. Masi, F., Stefanou, I., Vannucci, P., and Maffi-Berthier, V. (2020). Thermodynamics-based Artificial Neural Networks for constitutive modeling. arXiv.10.26226/morressier.612f6736bc98103724100885 ↗
22. Zhang, Z., Shin, Y., and Karniadakis, G.E. (2021). GFINNs: GENERIC Formalism Informed Neural Networks for Deterministic and Stochastic Dynamical Systems. arXiv.10.1098/rsta.2021.0207 ↗
23. Ghnatios, C., Alfaro, I., González, D., Chinesta, F., and Cueto, E. (2019). Data-Driven GENERIC Modeling of Poroviscoelastic Materials. Entropy, 21.10.3390/e21121165 ↗
24. Ghnatios, C., Champaney, V., Pasquale, A., and Chinesta, F. (2022). A Regularized Real-Time Integrator for Data-Driven Control of Heating Channels. Computation, 10.10.3390/computation10100176 ↗
25. Lee, J.A., and Verleysen, M. (2007). Nonlinear Dimensionality Reduction, Springer.10.1007/978-0-387-39351-3 ↗
26. A short review in model order reduction based on Proper Generalized Decomposition10.1007/s11831-011-9064-7 ↗
27. Parametric PGD based computational vademecum for efficient design, optimization and control10.1007/s11831-013-9080-x ↗
28. Chinesta, F., Keunings, R., and Leygue, A. (2014). The Proper Generalized Decomposition for Advanced Numerical Simulations: A Primer, Springer.10.1007/978-3-319-02865-1 ↗
29. Stein, E., de Borst, R., and Hughes, T. (2015). Model Order Reduction. Encyclopedia of Computational Mechanics, John Wiley & Sons, Ltd.. [2nd ed.].
30. Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions10.1561/2200000059 ↗
31. Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 2 Applications and Future Perspectives10.1561/2200000067 ↗
32. Tensor Train Decomposition on TensorFlow (T3F)
33. Tensor-Train Decomposition10.1137/090752286 ↗
34. Tensor Decomposition for Signal Processing and Machine Learning10.1109/TSP.2017.2690524 ↗
35. Su, J., Byeon, W., Kossaifi, J., Huang, F., Kautz, J., and Anandkumar, A. (2020). Convolutional Tensor-Train LSTM for Spatio-temporal Learning. arXiv.
36. Xu, Y.L., Calvi, G.G., and Mandic, D.P. (2021, January 18–22). Tensor-Train Recurrent Neural Networks for Interpretable Multi-Way Financial Forecasting. Proceedings of the 2021 International Joint Conference on Neural Networks (IJCNN), Online.10.1109/IJCNN52387.2021.9534120 ↗
37. Yang, Y., Krompass, D., and Tresp, V. (2017, January 6–11). Tensor-train recurrent neural networks for video classification. Proceedings of the 34th International Conference on Machine Learning, ICML17, Sydney, Australia.
38. Ye, J., Wang, L., Li, G., Chen, D., Zhe, S., Chu, X., and Xu, Z. (2018, January 18–23). Learning Compact Recurrent Neural Networks with Block-Term Tensor Decomposition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.10.1109/CVPR.2018.00977 ↗
39. Bacciu, D., and Mandic, D.P. (2020). Tensor Decompositions in Deep Learning. arXiv.
40. Towards Compact Neural Networks via End-to-End Training: A Bayesian Tensor Approach with Automatic Rank Determination10.1137/21M1391444 ↗
41. Tensor Regression Networks
42. Novikov, A., Podoprikhin, D., Osokin, A., and Vetrov, D. (2015, January 7–12). Tensorizing neural networks. Proceedings of the 28th International Conference on Neural Information Processing Systems, NIPS’15, Montreal, QC, Canada.
43. Su, J., Li, J., Liu, X., Ranadive, T., Coley, C., Tuan, T.-C., and Huang, F. (2022). Compact Neural Architecture Designs by Tensor Representations. Front. Artif. Intell., 5.10.3389/frai.2022.728761 ↗
44. Tensor Decompositions and Applications10.1137/07070111X ↗
45. Efficient MATLAB Computations with Sparse and Factored Tensors10.1137/060676489 ↗
46. Ghnatios, C., and Chinesta, F. (2024). A Parsimonious Separated Representation Empowering PINN-PGD Based Solutions for Parametrized Partial Differential Equations. Mathematics, 12.10.3390/math12152365 ↗
47. Ghnatios, C. (2012). Simulation avancée des problèmes thermiques rencontrés lors de la mise en forme des composites. Matériaux et Structures en Mécanique, Ecole Centrale de Nantes (ECN).
48. Proper Generalized Decomposition based dynamic data-driven control of thermal processes10.1016/j.cma.2011.11.018 ↗