Transforming physics-informed machine learning to convex optimization

Physics-Informed Machine Learning (PIML) offers a powerful paradigm of integrating data with physical laws to address important problems in engineering, such as parameter estimation, inferring hidden physics, equation discovery, and state prediction. However, PIML, such as Physics-Informed Neural Networks (PINNs), still lack the necessary accuracy, stability, and interpretability when applying in practical engineering due to many serious optimization challenges including the spectral bias, non-convex optimization, multi-objective optimization, and non-smooth optimization. In this study, we propose the Convex-PIML based on convex optimization and basis functions widely used in well-established numerical solvers to overcome all these limitations. The linear combination of B-splines is utilized to approximate the data, promoting the convexity of the loss function. By dividing variables into blocks and replacing the non-convex loss terms with convex approximations, the problem is further converted into a sequence of successively refined approximated convex optimization problems. This conversion known as Block Successive Convex Approximation (BSCA) allows the use of well-established convex optimization algorithms, obtaining solutions effectively and efficiently. Furthermore, an adaptive knot optimization method is introduced to mitigate the spectral bias issue of PIML, further improving the performance. The proposed fully adaptive framework by combining the adaptive knot optimization and BSCA is tested in scenarios with distinct types of physical prior. The results indicate that optimization problems are effectively solved in these scenarios, highlighting the potential of the framework for broad applications. Note that the Convex-PIML is also flexible since many other basis functions can also be incorporated to handle different systems.

Authors

• Letian Yi
• Siyuan Yang Nanyang Technological University
• Ying Cui
• Zhilu Lai

Keywords

• Physics-informed machine learning
• Convex optimization
• Parameter estimation
• Equation discovery
• State prediction