MATH Seminar

Title: Rethinking regularization in modern machine learning and computational imaging
Colloquium: Computational Mathematics
Speaker: Gregory Ongie of
Contact: James Nagy, jnagy@emory.edu
Date: 2020-01-13 at 4:00PM
Venue: MSC W303
Download Flyer
Abstract: Optimization is central to both supervised machine learning and inverse problems in computational imaging. These problems are often ill-posed and some form of regularization is necessary to obtain a useful solution. However, new paradigms in machine learning and computational imaging necessitate rethinking the role of regularization, as I will illustrate with two examples. First, in the context of supervised learning with shallow neural networks, I will show how a commonly used form of regularization has a surprising reinterpretation as a convex regularizer in function space. This yields novel insights into the role of overparameterization and depth in learning with neural networks having ReLU activations. Second, I will discuss a novel network architecture for solving linear inverse problems in computational imaging called a Neumann network. Rather than using a pre-specified regularizer, Neumann networks effectively learn a regularizer from training data, outperforming classical techniques. Beyond these two examples, I will show how many open problems in the mathematical foundations of deep learning and computational imaging relate to understanding regularization in its many forms.