TY - JOUR
T1 - Optimally weighted loss functions for solving PDEs with Neural Networks
AU - van der Meer, Remco
AU - Oosterlee, Cornelis W.
AU - Borovykh, Anastasia
N1 - Publisher Copyright:
© 2021 Elsevier B.V.
PY - 2022/5/15
Y1 - 2022/5/15
N2 - Recent works have shown that deep neural networks can be employed to solve partial differential equations, giving rise to the framework of physics informed neural networks (Raissi et al., 2007). We introduce a generalization for these methods that manifests as a scaling parameter which balances the relative importance of the different constraints imposed by partial differential equations. A mathematical motivation of these generalized methods is provided, which shows that for linear and well-posed partial differential equations, the functional form is convex. We then derive a choice for the scaling parameter that is optimal with respect to a measure of relative error. Because this optimal choice relies on having full knowledge of analytical solutions, we also propose a heuristic method to approximate this optimal choice. The proposed methods are compared numerically to the original methods on a variety of model partial differential equations, with the number of data points being updated adaptively. For several problems, including high-dimensional PDEs the proposed methods are shown to significantly enhance accuracy.
AB - Recent works have shown that deep neural networks can be employed to solve partial differential equations, giving rise to the framework of physics informed neural networks (Raissi et al., 2007). We introduce a generalization for these methods that manifests as a scaling parameter which balances the relative importance of the different constraints imposed by partial differential equations. A mathematical motivation of these generalized methods is provided, which shows that for linear and well-posed partial differential equations, the functional form is convex. We then derive a choice for the scaling parameter that is optimal with respect to a measure of relative error. Because this optimal choice relies on having full knowledge of analytical solutions, we also propose a heuristic method to approximate this optimal choice. The proposed methods are compared numerically to the original methods on a variety of model partial differential equations, with the number of data points being updated adaptively. For several problems, including high-dimensional PDEs the proposed methods are shown to significantly enhance accuracy.
KW - Convection–diffusion equation
KW - High-dimensional problems
KW - Loss functional
KW - Neural network
KW - Partial differential equation
KW - Poisson equation
UR - http://www.scopus.com/inward/record.url?scp=85120922086&partnerID=8YFLogxK
U2 - 10.1016/j.cam.2021.113887
DO - 10.1016/j.cam.2021.113887
M3 - Article
AN - SCOPUS:85120922086
SN - 0377-0427
VL - 405
SP - 1
EP - 18
JO - Journal of Computational and Applied Mathematics
JF - Journal of Computational and Applied Mathematics
M1 - 113887
ER -