New Bridges Between Deep Learning and Partial Differential Equations

Abstract

Understanding the world through data and computation has always formed the core of scientific discovery. Amid many different approaches, two common paradigms have emerged. On the one hand, primarily data-driven approaches—such as deep neural networks (DNNs)—have proven extremely successful in recent years. Their success is based mainly on their ability to approximate complicated functions with generic models when trained using vast amounts of data and enormous computational resources. But despite their many triumphs, DNNs are difficult to analyze and thus remain mysterious. Most importantly, they lack the robustness, explainability, interpretability, and fairness required for high-stakes decision-making. On the other hand, increasingly realistic model-based approaches—typically derived from first principles and formulated as partial differential equations (PDEs)—are now available for various tasks. One can often calibrate these models—which enable detailed theoretical studies, analysis, and interpretation—with relatively few measurements, thus facilitating their accurate predictions of phenomena. However, computational methods for PDEs remains a vibrant research area whose open challenges include the efficient solution of highly nonlinear coupled systems and PDEs in high dimensions. In recent years, exciting work at the interface of data-driven and model-based approaches has blended both paradigms. For instance, PDE techniques and models have yielded better insight into deep learning algorithms, more robust networks, and more efficient training algorithms. As another example, consider the solution of high-dimensional PDEs, wherein DNNs have provided new avenues for tackling the curse of dimensionality. One must understand that the exchange between deep learning and PDEs is bidirectional and benefits both communities. I hope to offer a glimpse into a few of these activities and make a case for the creation of new bridges between applied mathematics and data science.

Publication
SIAM News