Solving Partial Differential Equations Using Deep Learning and Physical Constraints
Source abstract
Abstract
The various studies of partial differential equations (PDEs) are hot topics of mathematical research. Among them, solving PDEs is a very important and difficult task. Since many partial differential equations do not have analytical solutions, numerical methods are widely used to solve PDEs. Although numerical methods have been widely used with good performance, researchers are still searching for new methods for solving partial differential equations. In recent years, deep learning has achieved great success in many fields, such as image classification and natural language processing. Studies have shown that deep neural networks have powerful function-fitting capabilities and have great potential in the study of partial differential equations. In this paper, we introduce an improved Physics Informed Neural Network (PINN) for solving partial differential equations. PINN takes the physical information that is contained in partial differential equations as a regularization term, which improves the performance of neural networks. In this study, we use the method to study the wave equation, the KdV–Burgers equation, and the KdV equation. The experimental results show that PINN is effective in solving partial differential equations and deserves further research.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Model Reduction and Neural Networks · Hydrological Forecasting Using AI · Meteorological Phenomena and Simulations
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
Affiliations
College of Computer, National University of Defense Technology, Changsha 410073, China
College of Meteorology and Oceanography, National University of Defense Technology, Changsha 410073, China
Funding
National Natural Science Foundation of China · National Key R\&D Program of China
References (73)
Show source-provided references
1. Folland, G.B. (1995). Introduction to Partial Differential Equations, Princeton University Press.
2. Petrovsky, I.G. (2012). Lectures on Partial Differential Equations, Courier Corporation.
3. Courant, R., and Hilbert, D. (2008). Methods of Mathematical Physics: Partial Differential Equations, John Wiley & Sons.
4. Farlow, S.J. (1993). Partial Differential Equations for Scientists and Engineers, Courier Corporation.
5. Zauderer, E. (2011). Partial Differential Equations of Applied Mathematics, John Wiley & Sons.
6. A numerical study of the effects of atmospheric and wake turbulence on wind turbine dynamics10.1080/14685248.2012.668191 ↗
7. Massively parallel solvers for elliptic partial differential equations in numerical weather and climate prediction10.1002/qj.2327 ↗
8. Tröltzsch, F. (2010). Optimal Control of Partial Differential Equations: Theory, Methods, and Applications, American Mathematical Society.10.1090/gsm/112/07 ↗
9. Ames, W.F. (2014). Numerical Methods for Partial Differential Equations, Academic Press.
10. Quarteroni, A., and Valli, A. (2008). Numerical Approximation of Partial Differential Equations, Springer.
11. A review on basic data-driven approaches for industrial process monitoring10.1109/TIE.2014.2301773 ↗
12. Bai, Z., Brunton, S.L., Brunton, B.W., Kutz, J.N., Kaiser, E., Spohn, A., and Noack, B.R. (2017). Data-driven methods in fluid dynamics: Sparse classification from experimental data. Whither Turbulence and Big Data in the 21st Century?, Springer.10.1007/978-3-319-41217-7_17 ↗
13. Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press.
14. Deep learning10.1038/nature14539 ↗
15. Deep learning: Methods and applications10.1561/2000000039 ↗
16. Deep learning for hyperspectral image classification: An overview10.1109/TGRS.2019.2907932 ↗
17. A primer on neural network models for natural language processing10.1613/jair.4992 ↗
18. Deep Learning for fault detection in wind turbines10.1016/j.rser.2018.09.012 ↗
19. Lu, Y., and Lu, J. (2020). A Universal Approximation Theorem of Deep Neural Networks for Expressing Distributions. arXiv.
20. Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks10.1016/0893-6080(90)90005-6 ↗
21. Artificial neural networks for solving ordinary and partial differential equations10.1109/72.712178 ↗
22. Göküzüm, F.S., Nguyen, L.T.K., and Keip, M.A. (2019). An Artificial Neural Network Based Solution Scheme for Periodic Computational Homogenization of Electrostatic Problems. Math. Comput. Appl., 24.10.3390/mca24020040 ↗
23. A deep energy method for finite deformation hyperelasticity10.1016/j.euromechsol.2019.103874 ↗
24. Bar, L., and Sochen, N. (2019). Unsupervised deep learning algorithm for PDE-based forward and inverse problems. arXiv.
25. Freund, J.B., MacArt, J.F., and Sirignano, J. (2019). DPM: A deep learning PDE augmentation method (with application to large-eddy simulation). arXiv.
26. Data-driven deep learning of partial differential equations in modal space10.1016/j.jcp.2020.109307 ↗
27. Khoo, Y., Lu, J., and Ying, L. (2017). Solving parametric PDE problems with artificial neural networks. arXiv.
28. Deep neural networks for waves assisted by the Wiener–Hopf method10.1098/rspa.2019.0846 ↗
29. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations10.1016/j.jcp.2018.10.045 ↗
30. Raissi, M., Perdikaris, P., and Karniadakis, G.E. (2017). Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations. arXiv.
31. Raissi, M., Perdikaris, P., and Karniadakis, G.E. (2017). Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations. arXiv.
32. Chen, X., Duan, J., and Karniadakis, G.E. (2019). Learning and meta-learning of stochastic advection-diffusion-reaction systems from sparse measurements. arXiv.10.1017/S0956792520000169 ↗
33. Physics-informed neural networks for multiphysics data assimilation with application to subsurface transport10.1016/j.advwatres.2020.103610 ↗
34. Physics-Informed Deep Neural Networks for Learning Parameters and Constitutive Relationships in Subsurface Flow Problems10.1029/2019WR026731 ↗
35. Kadeethum, T., Jørgensen, T.M., and Nick, H.M. (2020). Physics-informed neural networks for solving nonlinear diffusivity and Biot’s equations. PLoS ONE, 15.10.1371/journal.pone.0232683 ↗
36. Machine learning in cardiovascular flows modeling: Predicting arterial blood pressure from non-invasive 4D flow MRI data using physics-informed neural networks10.1016/j.cma.2019.112623 ↗
37. Adaptive activation functions accelerate convergence in deep and physics-informed neural networks10.1016/j.jcp.2019.109136 ↗
38. Shanmuganathan, S. (2016). Artificial neural network modelling: An introduction. Artificial Neural Network Modelling, Springer.10.1007/978-3-319-28495-8 ↗
39. Nielsen, M.A. (2015). Neural Networks and Deep Learning, Determination Press.
40. Introduction to multi-layer feed-forward neural networks10.1016/S0169-7439(97)00061-0 ↗
41. Li, J., Cheng, J.H., Shi, J.Y., and Huang, F. (2012). Brief introduction of back propagation (BP) neural network algorithm and its improvement. Advances in Computer Science and Information Engineering, Springer.10.1007/978-3-642-30223-7_87 ↗
42. Learning in modal space: Solving time-dependent stochastic PDEs using physics-informed neural networks10.1137/19M1260141 ↗
43. Tipireddy, R., Perdikaris, P., Stinis, P., and Tartakovsky, A. (2019). A comparative study of physics-informed neural network models for learning unknown dynamics and constitutive relations. arXiv.
44. Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., and Isard, M. (2016, January 2–4). Tensorflow: A system for large-scale machine learning. Proceedings of the 12th {USENIX} symposium on operating systems design and implementation ({OSDI} 16), Savannah, GA, USA.
45. Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., and Lerer, A. (2017, January 9). Automatic differentiation in pytorch. Proceedings of the NIPS 2017 Autodiff Workshop, Long Beach, CA, USA.
46. Gradient-based optimization of hyperparameters10.1162/089976600300015187 ↗
47. Kylasa, S., Roosta, F., Mahoney, M.W., and Grama, A. (2019, January 2–4). GPU accelerated sub-sampled newton’s method for convex classification problems. Proceedings of the 2019 SIAM International Conference on Data Mining, Calgary, AB, Canada.10.1137/1.9781611975673.79 ↗
48. Richardson, A. (2018). Seismic full-waveform inversion using deep learning tools and techniques. arXiv.
49. Le, Q.V., Ngiam, J., Coates, A., Lahiri, A., Prochnow, B., and Ng, A.Y. (July, January 28). On optimization methods for deep learning. Proceedings of the 28th International Conference on Machine Learning, ICML, Bellevue, WA, USA.
50. Lu, L., Meng, X., Mao, Z., and Karniadakis, G.E. (2019). DeepXDE: A deep learning library for solving differential equations. arXiv.
51. Wave-equation dispersion inversion10.1093/gji/ggw465 ↗
52. Dynamic behaviors of interaction solutions of (3+ 1)-dimensional Shallow Water wave equation10.1016/j.camwa.2018.06.034 ↗
53. Kim, D. (2019). A Modified PML Acoustic Wave Equation. Symmetry, 11.10.3390/sym11020177 ↗
54. Raissi, M., Perdikaris, P., and Karniadakis, G.E. (2017). Numerical Gaussian processes for time-dependent and non-linear partial differential equations. arXiv.10.1137/17M1120762 ↗
55. Hanin, B., and Rolnick, D. (2018). How to start training: The effect of initialization and architecture. Advances in Neural Information Processing Systems, Curran Associates Inc.
56. Nonlinear waves in layered media: Solutions of the KdV–Burgers equation10.1016/j.geomphys.2018.03.016 ↗
57. Asymptotic Stability of Monotone Decreasing Kink Profile Solitary Wave Solutions for Generalized KdV-Burgers Equation10.1007/s10255-019-0825-7 ↗
58. On nonlinear superposition of the KdV–Burgers shock waves and the behavior of solitons in a layered medium10.1016/j.difgeo.2017.03.001 ↗
59. Numerical solution of Korteweg–de Vries-Burgers equation by the modified variational iteration algorithm-II arising in shallow water waves10.1088/1402-4896/ab6070 ↗
60. Numerical solution of Burgers’ equation with high order splitting methods10.1016/j.cam.2015.04.021 ↗
61. Travelling waves and conservation laws of a (2+ 1)-dimensional coupling system with Korteweg-de Vries equation10.21042/AMNS.2018.1.00018 ↗
62. 2D Burgers equation with large Reynolds number using POD/DEIM and calibration10.1002/fld.4249 ↗
63. Inverse scattering transform and soliton classification of the coupled modified Korteweg-de Vries equation10.1016/j.cnsns.2017.03.022 ↗
64. Soliton solutions to the fifth-order Korteweg–de Vries equation and their applications to surface and internal water waves10.1063/1.5009965 ↗
65. Driscoll, T.A., Hale, N., and Trefethen, L.N. (2014). Chebfun Guide, Pafnuty Publications.
66. Modified homogeneous balance method: Applications and new solutions10.1016/j.chaos.2015.01.017 ↗
67. Soliton solution of good Boussinesq equation10.1007/s10013-015-0157-8 ↗
68. Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data10.1016/j.jcp.2019.05.024 ↗
69. Murphy, K.P. (2012). Machine Learning: A Probabilistic Perspective, MIT Press.
70. A Short Review on Model Order Reduction Based on Proper Generalized Decomposition10.1007/s11831-011-9064-7 ↗
71. Ohayon, R., and Schotté, J.S. (2017). Fluid–Structure Interaction Problems. Encyclopedia of Computational Mechanics, American Cancer Society. [2nd ed.].10.1002/9781119176817.ecm2045 ↗
72. Nguyen-Thanh, V.M., Nguyen, L.T.K., Rabczuk, T., and Zhuang, X. (2020). A surrogate model for computational homogenization of elastostatics at finite strain using HDMR-based neural network. Int. J. Numer. Methods Eng.
73. Computational homogenization of nonlinear elastic materials using neural networks10.1002/nme.4953 ↗