Emory REU Computational Mathematics for Data Science

Synthetically Rebalancing Healthcare Datasets via Conditional DDPM

Fri, 01 Dec 2023 00:00:00 +0000

This blog post was written by Keira Behal, Jiayi Chen, Caleb Fikes, and Sophia Xiao and published with minor edits. Our work was guided by Dr. Yuanzhe Xi.In addition to this post, the team has also given a midterm presentation, created a poster blitz video, created a poster, created some code and wrote a paper.

Background

In recent years, machine learning algorithms have become increasingly important in healthcare for tasks like disease prediction, diagnostics, and treatment optimization. However, these algorithms can perpetuate harmful societal biases if the training data contains inherent biases or underrepresentation of certain groups, particularly minority communities. This bias issue is a significant concern in healthcare, where fair and equitable treatment is crucial. Even valuable data sources like Electronic Health Records (EHR), which provide a comprehensive overview of a patient’s health history, including diagnoses, treatments, and demographics, can suffer from underrepresentation of certain racial or ethnic minorities. This imbalance can lead to inequitable health outcomes, with minority groups receiving less accurate diagnoses or treatment recommendations due to their underrepresentation in the training data.

Our Approach

To address this challenge, we propose Minority Class Rebalancing through Augmented Data Generation (McRAGE), a novel approach to augment imbalanced medical datasets using samples generated by a deep generative model. The McRAGE process involves training a Conditional Denoising Diffusion Probabilistic Model (CDDPM) capable of generating high-quality synthetic EHR samples from underrepresented classes. We use this synthetic data to augment the imbalanced dataset, achieving a more balanced distribution across all classes. The dataset can be used to train an unbiased machine learning model. Our work intends to promote fair and accurate healthcare predictions, enhancing patient care and supporting equity through data science and artificial intelligence.

In our project, we focused on data-driven methods of synthetic sample generation through deep generative models because deep generative models aim to capture the underlying data distribution and generate samples that closely resemble the real data so that they can produce high-fidelity synthetic samples that retain the characteristics and variability of the original data. We chose a CDDPM to generate synthetic samples to augment our dataset for the following three reasons:

Stability of Training: DDPMs are easier to train than GANs (Generative Adversarial Networks), which are more susceptible to mode collapse and vanishing gradients.
High Fidelity Generation: diffusion models can produce higher quality samples than other generative models. Many commercial implementations of diffusion models such as Stability AI’s Stable Diffusion and Open AI’s DALL-E 2 have garnered significant funding and public interest.
Class-specific Generation: The CDDPM offers control over the generation of specific classes of data, whereas conventional DDPM is limited to generating samples that reflect the class distribution of its training set. By utilizing CDDPM, we can use the knowledge gained from the majority groups to improve the quality of generated data for the minority groups.

The MCRAGE process is both intuitive and theoretically justifiable. The algorithm results in a synthetically rebalanced training set where each “intersectional group,” or a unique combination of sensitive demographic factors, is equally represented. By generating an artificial stratified random sample, the process promotes statistical parity, meaning that classifiers trained on this data will have the same distribution of decisions for each group. This ensures that the classifier’s performance is nearly equivalent for each subgroup despite class imbalances in the training data. The MCRAGE process is proposed under the assumption that such a result holds. In that case, the process results in the nearest approximation to a stratified sample given all the information in the training set. This project aims to empirically test the performance of MCRAGE compared to previous state-of-the-art methods for mitigating dataset imbalance.

Our Contribution

Our approach to conditional generation in the diffusion model is called classifier-free guidance. It involves modifying the denoising update process by incorporating class information. Unlike the classifier guidance method, which relies on a separate classifier for conditional sample generation, our approach integrates the class information directly into the model. In our implementation, we added an extra class embedding to the time embedding in the conventional DDPM. This enables us to incorporate and utilize class information effectively.

Our Results

While developing our algorithm, we used a subset of MNIST to track our progress. Finally, we applied the algorithm to a tabular EHR dataset called Patient Treatment Classification with a random forest classifier. The dataset consists of Electronic Health Records collected from a private Hospital in Indonesia. The dataset contains eight laboratory test results for 3309 patients, used to determine patient treatment classification (inpatient care or outpatient care). In both cases, we compared the synthetically balanced datasets of our approach to the balanced dataset and those obtained from the SMOTE algorithm. We also compared the different approaches quantitatively using different fitness metrics and found that the classifier trained on data augmented by the DDPM performed the best.

These promising results motivate us to conduct the same procedure on a more extensive EHR data set with more features with multiple classes, possibly incorporating a multinomial diffusion.

Interested in Learning More?

Please see our poster and paper for more details.

A Tensor SVD-based Classification Algorithm Applied to fMRI Data

Tue, 14 Dec 2021 00:00:00 +0000

This post was written by Katy Keegan, Yihua Xu, Tanvi Vishwanath, and Vida Jon and published with minor edits. The team was advised by Dr. Elizabeth Newman. In addition to this post, the team has also given a midterm presentation , created a poster , published, code, and written a paper.

Overview: Can we look at a brain scan and know what the brain’s owner is thinking?

Our research attempts to use computers to correctly classify brain scans into 2 groups, depending on what they are doing during the scan.

Functional MRI

We use brain scans called functional MRIs that show us which parts of the brain are using more oxygen and are therefore most active. We have fMRIs of test subjects who, as they are being scanned, are also shown either a picture or a sentence. If computers can classify these study subjects into one of these two categories by only studying their scans, then in a sense we can read their minds.

fMRIs consist of three-dimensional pixels called voxels, and the data are numbers representing colors. Unlike static MRIs which take a scan at one point in time, fMRIs are repeated every few seconds creating a series of images for each trial. Here is an example of an fMRI of one brain during one trial. The different images are of different slices of the brain. The abbreviations on the right refer to brain regions, for example “SMA” stands for “Supplementary Motor Area” located at the top center of the head.

If we have three-dimensional fMRI brain voxel data for many patients, multiple scans in sequence, then we need to analyze a quantity of data unwieldy even for modern computers.

Image Classification

Image classification is using a computer to figure what what an image represents. Computers can’t see images, so they use features of the images that it can understand. For example, we can train a computer to match an image to a numerical digit. Computers learn by training on many images, for example using the MNIST database of handwritten images. MNIST contains a wide variety of images that can represent a 0 or a 1:

We can construct a “basis”, which can be thought of as a collection of the most relevant features shared by all of the images belonging to that class.

We see that the basis for Class 0 shows more curved features, while the basis for Class 1 contains traces of more straight and vertical features. To choose the basis the image more closely matches, we compute a “projection”, not unlike what you may have calculated with vectors. The larger the projection, the better the match to a particular basis.

The computer tries to classify this test image as either a zero or a one:

This image represents the projection of the test image onto the basis for numeral one:

The image projection onto the basis for zero, shows differences and inconsistencies:

The projection shows that the “distance” between our test image and our two classes is smaller for Class 1 than for Class 0, and so our method classifies our test image as a 1.

By learning from a training set of images a computer can examine the data in a new image and figure out which digit it most resembles. Similarly, we are training our computer to learn how to use the data in an fMRI to classify our study subjects into those who are shown an image and those who are shown a sentence.

Tensors and Singular Value Decomposition

Typically, large data sets like the fMRI voxels are stored in matrices, which have some powerful tools for extracting the most relevant components.

When we store fMRI data in a matrix we lose important relationships between the data points. For example a computer does not know that a particular voxel representing a part of the brain at one moment is that same part of the brain a few seconds later.

In our work, we study how to store our data in a tensor which is like matrix but with more than 2 dimensions. Our tensor of fMRIs have a total of 5 dimensions, shown in the figure below. The green slices consist of voxels of the brain in 3 spatial dimensions: x,y,z (yellow). For each trial multiple images are taken over several seconds (blue), and there are multiple trials (red).

<img width=“600” alt=“fmri_tensors”

When we store our voxel data in tensors instead of matrices, we retain all the information about the 3 dimensional location in the brain, the sequence of the image in time, and the trial. Now we want to decompose our tensor data using a method analogous to the way matrices can be decomposed, so we can use far less data and still extract an accurate prediction of what the subject is thinking. In matrices this is called Singular Value Decomposition; we call our approach tensor-SVD or tSVD.

Here is a diagram of matrix decomposition using SVD:

This what we imagine a tensor SVD would be:

You’ll notice that we use matrix multiplication in working with matrix SVDs. We are searching for the equivalent tensor multiplication.

Other Applications

Our research is not only useful for fMRIs; many datasets have multiple dimensions. Streaming entertaining companies have data on thousands of viewers and what movies they’ve watched. Hospitals track thousands of patients, each of whom has had multiple lab tests and other studies. If our research enables us to classify our fMRI subjects, then we may also be able to predict whether someone will want to watch Terminator, or whether a patient is likely to have cancer.

References

Tensor tensor products with invertible linear transforms

Tensor-tensor algebra for optimal representation and compression of multiway data

Tensor decompositions and applications

Image classification using local tensor singular value decompositions

Generative AI for Structure Discovery in Cardiovascular Digital Twins

Fri, 16 Jan 2026 00:00:00 +0000

Mentors: Dr. Marco Tezzele and Dr. Jimena Martin Tempestti

Overview

A digital twin (DT) is a virtual representation of a physical object that dynamically evolves with its real-world counterpart through sensed data, and provides value through optimal decision-making. Effective DTs rely on accurate mathematical representations of causal dependencies and temporal evolution, often modeled via Probabilistic Graphical Models (PGMs). Traditional approaches frequently assume a static, predefined graph topology (e.g., first-order Markov chains) or rely on simplified expert heuristics, which may fail to capture the rich, long-range dependencies inherent in physiological systems.

This project investigates how generative AI can automate the discovery of optimal mathematical structures for DTs. Students will develop a framework where a DT is represented as a dynamic PGM. Starting from a fully connected graph encompassing observations, latent digital states, quantities of interest, and actions, students will employ generative methods to prune and optimize the graph topology.

Research Challenges

The research will specifically focus on two “New Mathematics” challenges:

Topology Learning: Analyzing how the graph structure affects the identifiability and estimation of digital states
Temporal Memory: Investigating whether higher-order Markov chains generated by AI models yield better predictions than standard memoryless models

Validation

The methods will be validated on simulated cardiovascular datasets, aiming to predict patient outcomes and the optimal treatment plan for cardiovascular diseases.

AI-Assisted Exploration in Algebra and Number Theory

Fri, 16 Jan 2026 00:00:00 +0000

Mentor: Dr. Deependra Singh

Overview

Inspired by the recent work Mathematical exploration and discovery at scale by Georgiev, Gomez-Serrano, Tao, Wagner, and Google DeepMind, this project explores how LLM-based agents can be systematically integrated into research in algebra and number theory. The central idea is to combine three complementary modes of mathematical work into a single pipeline:

Computational exploration: Using evolutionary and code-based agents (in the spirit of AlphaEvolve) to perform large-scale numerical and symbolic experiments, with the aim of identifying patterns and generating plausible conjectures
Informal mathematical reasoning: Developing insights from experimentation into coherent, human-readable arguments, in the style of extended reasoning systems such as DeepThink
Formal verification: Translating successful arguments into fully rigorous, machine-checked proofs using proof assistants such as Lean, guided by tools similar to AlphaProof

Approach

As a first stage, we will apply this pipeline to a set of classical problems in number theory, including results on sums of squares, quadratic forms, and the $u$-invariant of local and global fields. These benchmarks serve both as a training ground for deploying the methodology and as a means of evaluating its strengths and limitations, since the underlying results are well understood.

Having calibrated the approach on these known cases, we will then turn to carefully selected open problems in less explored settings. In particular, we aim to investigate whether this framework can yield new insights for arithmetic questions over fields beyond the local and global fields, such as semi-global fields (for example, $\mathbb{C}((t))(x)$ and $\mathbb{Q}_p(x)$), where similar questions are less well-understood and systematic exploration may be especially informative.

Prerequisites

Fields, Galois Theory.

Joint Analysis of X-ray Ptychography and X-ray Fluorescence Reconstruction

Tue, 16 Dec 2025 00:00:00 +0000

This project was completed by Mingke Tian and Eric Zou under the mentorship of Yuanzhe Xi as part of the 2025 Emory REU program.

Figure 1: Reconstruction results from joint ptychography and fluorescence framework. Source: Adapted from D. J. Vine et al., Simultaneous X-ray fluorescence and ptychographic microscopy of Cyclotella meneghiniana, 2012.

Ptychographic Reconstruction Part

In the traditional way, we use separate methods for ptychographic and fluorescence reconstruction. We consider a ptychography experiment where the observed data $d_i$ is modeled as:

$$ d_i = \vert\mathcal{F}(\mathbf{P}_i \mathbf{z})\vert^2 + \epsilon_i $$

where:

$\mathcal{F}$: 2D discrete Fourier transform operator
$\mathbf{P}_i$: Probe matrix
$z_i$: the object itself which can be written as $z = x + y_i$

Reconstruction Problem

The reconstruction loss function is formulated as the following:

$$ \min_{\mathbf{P},\mathbf{z}} \Phi(\mathbf{P},\mathbf{z}) = \frac{1}{2} \sum_{j=1}^{N} \Vert \vert \mathcal{F}(\mathbf{P}_j \mathbf{z})\vert - \sqrt{d_j} \Vert_2^2 $$

X-ray Fluorescence Reconstruction

X-ray fluorescence reconstruct the real part of the image $x$ using a deconvolution method:

$$ \min_{\mathbf{w},\mathbf{P}} \sum_{e=1}^{N_e} \Vert \vert\mathbf{P}\vert^2 * \mathbf{w}_e - D_e \Vert_2^2 $$

where $\mathbf{w}$ represents elemental concentration maps. $D_e$ corresponds to the experimental flourescence map of element $e$.

Separate Optimization Framework

Separate optimization of ptychographic and fluorescence reconstruction may lead to the results:

$$ \min_{\mathbf{P},\mathbf{z}} \Phi(\mathbf{P},\mathbf{z}) + \min_{\mathbf{w},\mathbf{P}} \sum_{e=1}^{N_e} \Vert \vert\mathbf{P}\vert^2 * \mathbf{w}_e - D_e \Vert_2^2 $$

where $\alpha$ is a scaling parameter balancing between the two objectives.

Joint Optimization Framework

We propose a simultaneous optimization approach:

$$ \min_{\mathbf{w},\mathbf{P},\mathbf{z}} \sum_{e=1}^{N_e} \Vert \vert\mathbf{P}\vert^2 * \mathbf{w}_e - D_e \Vert_2^2 + \alpha \sum_{j=1}^{N} \Vert \vert \mathcal{F}(\mathbf{P}_j (\sum_e \mathbf{w}_e + i \beta))\vert - \sqrt{d_j} \Vert_2^2 $$

By using joint method, the loss function is consistently less than the original loss function from the separate method.

Absorption Coefficient Connection

The absorption coefficient relates to elemental concentrations via:

$$ \mathbf{z} = \delta + i \beta, \quad \delta = \sum_e \mathbf{w}_e \mu_e $$

Conclusion

This joint framework aims to enhance both ptychographic and fluorescence data processing, leveraging their complementarity for improved reconstruction.

Reference

[1] Deng, J., Vine, D.J., Chen, S. et al. X-ray ptychographic and fluorescence microscopy of frozen-hydrated cells using continuous scanning. Sci Rep 7, 445 (2017). https://doi.org/10.1038/s41598-017-00569-y

Hybrid Regularization for Random Feature Models

Fri, 01 Aug 2025 00:00:00 +0000

This blog post was written by Tony Maccagnano, Sarah Skaggs, Madeline Straus, and Advay Vyas and published with minor edits. The team was advised by Dr. Chelsea Drum. In addition to this website, the team has given a midterm presentation, filmed a poster blitz video, created a poster, published code, and worked on a manuscript.

Motivation and Background

Image classification is the process by which computer models learn to identify distinguishing features in images in order to recognize and categorize them effectively. As a result, it has many diverse applications in bioinformatics, 3D reconstruction, information extraction, and signal and image processing. In particular, the medical field has seen significant benefits from image classification, which has enabled earlier and more accurate diagnoses by detecting patterns that often elude the human eye and by helping to reduce the rate of misdiagnosis.

Two main problems within image classification are feature extraction and image labeling. For example, the image below represents photos of the widely studied MNIST dataset which contains handwritten digits 0-9. This dataset serves as a benchmark for training and evaluating classification models before applying them to real-world applications. A well-trained model should be able to efficiently and accurately classify each image as the correct digit before progressing to more complex and diverse classification problems.

Sample images from the MNIST dataset of handwritten digits.

Questions that we considered were:

How exactly do computer models learn to identify the contents of an image?
What happens when there is too much data for the computer system to handle?
Can we improve the efficiency of existing classification methods?

Our Problem

A common and effective approach in image classification is to represent it as a large-scale linear inverse problem of the form $$B \approx AX$$ Here, $B$ is the observed output data (image labels), $A$ is the input image matrix and $X$ is the unknown coefficient matrix we want to approximate. This means we need to determine the input parameters of a system based on the output data. However, many systems are noisy and small disturbances in the label matrix can lead to large differences in the input data, meaning that our problem may be ill-posed. Additionally, when working with large datasets, solving this problem explicitly becomes computationally challenging. One effective strategy to address this is the use of random feature models.

Random Feature Models

Random feature models transform image data into a higher-dimensional space using randomly generated features, enabling the use of linear methods for classification or regression. A common challenge with this approach is the double descent phenomenon, where test error dips, then spikes, and then falls again as the number of features increases. The spike often appears when the number of features is close to the number of training images, because the system is at the interpolation threshold and becomes ill-conditioned.

The figure below illustrates the double descent phenomenon when solving the problem directly without regularization. The test error spikes dramatically at $2^{10}$ features, which is also the number of images in this example.

Double descent phenomenon in unregularized random feature models.

Instead, we can regularize the problem to stabilize the solution and mitigate the effects of the double descent phenomenon.

We can either iteratively regularize the problem, meaning we solve it step by step on an increasing subspace, stopping when error is minimal, or we can do variational regularization, which involves solving a slightly different problem that incorporates the weights (complexity) of the solution. However, both of these approaches have their drawbacks. It can be difficult to know when to stop iterative regularization, while variational can take longer to converge and also requires us to provide a regularization parameter $\lambda$, and a bad one can negatively affect the accuracy of the model.

To get the benefits of these methods while mitigating their downsides, current research has combined the two into something called hybrid regularization. These methods use an iterative method to build the problem subspaces (e.g. Krylov) and apply variational regularization (e.g. Tikhonov) inside those subspaces.

Hybrid Regularization Methods

Previously, in [3], the hybrid LSQR method (available in MATLAB’s IR Tools package [2]) was successfully applied for the training of RFMs. Notably, the method:

Avoided the double descent phenomenon
Performed competitively against existing methods

However, this method uses computationally expensive inner products. Our proposal is to instead utilize a hybrid LSLU method, introduced in [1], which improves on hybrid LSQR by using a Hessenberg process that

Does not require inner products
Requires less storage and work

Through these improvements, this unique approach is advantageous for both mixed precision and high-performance computing scenarios.

Results and Conclusions

For our experiments, we tested various methods for the training of random feature models with the MNIST dataset. We specifically compared the hybrid LSLU method to hybrid LSQR and other standard regularized and unregularized methods.

We compared test errors against the number of features for different methods on the MNIST dataset, which consists of $70,000$ images ($60,000$ training images, $10,000$ test images) of digits $0, 1, \dots, 9$, where each image is composed of $28 \times 28$ pixels. For this data we can take $n_i = 28 \times 28 = 784$ input features (a gray-scale level for each pixel), and $n_o = 10$ for the number of unique labels (digits). For the following results, we only utilize $2^{10}$ training images. The unregularized method clearly exhibits double descent, while the two simple regularization methods, weight decay and gradient flow have a clear, smooth convergence. However, weight decay and gradient flow require computation of the SVD. Hybrid LSLU and hybrid LSQR converge in a similar manner without requiring SVD.

Comparison of test error vs. number of features for different methods.

In the following figures, we plot test error against number of features for hybrid LSQR and hybrid LSLU on the MNIST data set for different numbers of iterations. We see here that while the accuracy with hybrid LSQR improves with increased iterations, the accuracy of hybrid LSLU is fairly stagnant across most values of features. We do observe that if too many iterations are utilized for hybrid LSLU, then the double descent phenomenon occurs.

Test error for hybrid LSQR across different iterations.

Test error for hybrid LSLU across different iterations.

The following graphs show that hybrid LSLU converges much faster compared to hybrid LSQR in terms of number of iterations.

Error vs. iterations for 256 features.

Error vs. iterations for 1024 features.

Error vs. iterations for 4096 features.

Our experiments show that hybrid LSLU converges much faster compared to hybrid LSQR in terms of number of iterations. Thus, not only is hybrid LSLU cheaper (in terms of computational cost) per iteration than hybrid LSQR, but it also requires less iterations to achieve comparable accuracy. We plan to investigate how to optimally stop hybrid LSLU in order to ensure the avoidance of the double descent phenomenon. In our future research we also hope to apply the randomized sketch-and-solve version of hybrid LSLU, which has the potential to address some of the issues we currently see.

Our Activities

Weeks 1 and 2

We did literature review on previous and current research. Using IRTools, we replicated the results of the pre-existing papers on hybrid LSQR by implementing random feature model projections.

Week 3

We composed our midterm presentation and began experimenting with the hybrid LSLU function and how it compared to hybrid LSQR.

Week 4

We worked on the midterm presentation and then researched different ways to optimize the regularization parameter λ.

Week 5

We looked more at double descent and different types of GCV in order to optimize LSLU with RFMs. We also started on the webpage draft.

Week 6

We put together our poster blitz video and started work on the manuscript while also working on getting results for the poster and manuscript. We researched more theory in order to properly explain our ideas.

Week 7

We put together our poster for the poster presentation and finalized our results and research. We implemented LSLU on MNIST and CIFAR10 and compared the test error to unregularized, weight decay, gradient flow and LSQR. We started working on the manuscript in order to get a rough draft done quickly.

Week 8

We presented our poster, finished up the manuscript, and cleaned up our code.

More about the Team

Tony Maccagnano is a rising junior at Michigan Technological University, double majoring in Mathematics and Computer Science. He plans to go to graduate school for algebra or combinatorics.
Sarah Skaggs is a rising junior at Virginia Tech double majoring in Computational Modeling and Data Analytics and Mathematics. She plans to go to graduate school for applied math and her dream career is to use math and data science to advance medical technology. Outside of academics, she loves to play pickleball, adventure, play board games and spend time with friends and family.
Madeline Straus is a rising junior at Tufts University double majoring in Applied Mathematics and History, with plans to attend graduate school for math, and is particularly interested in numerical analysis and tomography. In her free time, she likes to read, compete at trivia, and play board games.
Advay Vyas is a rising sophomore at the University of Texas at Austin double majoring in Statistics & Data Science and Mathematics. He hopes to pursue a PhD in Statistics and then work on new ideas in machine learning and its applications. In his free time, he loves to read novels, bike, and watch sports.

References and Further Reading

Ariana N Brown et al. “Inner Product Free Krylov Methods for Large-Scale Inverse Problems”. In: arXiv preprint arXiv:2409.05239 (2024).
Silvia Gazzola, Per Christian Hansen, and James G Nagy. “IR Tools: a MATLAB package of iterative regularization methods and large-scale test problems”. In: Numerical Algorithms 81 (2019), pp. 773–811.
Kelvin Kan, James Nagy, and Lars Ruthotto. “Hybrid Regularization Methods Achieve Near-Optimal Regularization in Random Feature Models”. In: Submitted to Transactions on Machine Learning Research (2024).
Ali Rahimi and Benjamin Recht. “Random Features for Large-Scale Kernel Machines”. In: Advances in Neural Information Processing Systems. Vol. 20. 2007, pp. 1177–1184

Inverse problems and uncertainty quantification for the control of dynamical systems

Fri, 01 Aug 2025 00:00:00 +0000

By Nina Chafee, Micah Chandler, Benjamin Lebdaoui, and Meredith Orton

Advised by Dr. John Darges

How can we maximize our harvest without endangering the local wolf and bunny populations?

Introduction

We’re farmers, and we have a big problem! Every spring, the rabbit population skyrockets, decimating our crops, which means we don’t have quite enough food to feed everyone. We could douse the crops in rabbit-killing pesticides, but unfortunately, we actually care about the environment and its inhabitants—particularly the local population of wolves. Negatively impacting the rabbit population too much depletes their food source, and could drive both populations to extinction. So, as much as possible, how can we preserve the local ecosystem and reduce the bunny boom? You may find this video illuminating: Poster Blitz Video

Very Educated Guessing

To predict the impact of possible interventions, we first need to understand the dynamics of the system. We can do this by modeling the population dynamics between the bunnies and the wolves with the Lotka-Volterra Model:

$$\frac{dx}{dt} = \theta_1 x - \theta_2xy$$

$$\frac{dy}{dt} = -\theta_3 y + \theta_3 xy$$

$$x(0) = x_0, \quad y(0) = y_0,$$

where $\theta_1,\theta_2,\theta_3,\theta_4, x_0, y_0$ are the parameters that govern the dynamics of this system.

Of course, in the real world, we don’t know what these governing parameters are! All we have access to is noisy and often incomplete data. From that data, we need to perform parameter estimation to obtain the parameters that will make the model reproduce the data we’ve observed (psst this is an inverse problem*).

Deterministic Approach

We assume that our parameters are fixed unknowns. We want to minimize the distance between the output of our model and the data we are observing, measured with the Ordinary Least Squares (OLS) Estimator:

$$\theta_{OLS} = \text{argmin}{\theta}\sum{i= 1}^N(F(x_i;\theta)-d_i)^2,$$

where $\theta$ is parameter vector $[\theta_1,\theta_2,\theta_3,\theta_4,x_0,y_0]$, $F(x;\theta)$ is our model, and $d$ is our data.

We use data 20 years of local bunny and wolf populations. Note that we use synthetic data, so we know the true parameters for this experiment.

Bayesian Approach

We’ll assume our parameters are random variables, and we want to construct posterior distributions for our parameters; that is, what parameters are likely and how likely are they?

Using Bayes’ theorem, we combine prior knowledge with observed data:

$$p(\theta\mid data) = p(data\mid\theta) \cdot p(\theta)$$

Where:

$p(\theta\mid data)$ = posterior distribution (what we want)
$p(data\mid \theta)$ = likelihood (how well parameters explain data)
$p(\theta)$ = prior distribution (our initial beliefs about parameters)

This gives us:

Robustness to noise
Prior knowledge incorporation
Full uncertainty propagation

We do this through Markov Chain Monte Carlo (MCMC) Methods. Since we can’t directly calculate $p(\theta \mid data)$, MCMC creates a “chain” of parameter samples that eventually converges to the true posterior distribution.

Metropolis-Hastings Algorithm

Proposal distribution: $q(\theta_j , \theta_{j-1})$
Target distribution: $\Pi_{post}(\theta \mid d)$

Steps:

$\theta_j$ , sample $\theta^* \sim q(\theta_j)$
Evaluate quality of $\theta^*$ compared to $\theta_j$ using $\Pi_{post}$
Based on evaluation accept $\theta_{j+1} = \theta^*$ or reject $\theta_{j+1} = \theta_j$

Now we predict the future behavior of the bunny and wolf populations for the next 20 years to model future control strategies, and we can construct confidence and prediction intervals to quantify uncertainty from the noisy data:

Taking back the farm

Goal: limit the bunny boom in the spring without hurting the ecosystem! We can do this by driving the system of rabbits and wolves towards an equilibrium which will smooth population fluxuations without forcing either population to extinction.

Controls, denoted $\alpha(t)$, are actions that we take to change the system. The cost function measures how unfavorable controls are. The optimal control is the one that minimizes the cost function:

$$u(t) = \text{argmin} C_{x,t}(\alpha).$$

The cost function $C_{x,t}(\alpha)$ is often the sum of two parts: a measure of the control’s success $g(x(T))$ at the end state, and an integral measuring the control’s running cost.

$$C_{x,t}(\alpha) = g(x(T)) + \int_{t_0}^{T} L(x(t), \alpha(t), t)$$

While there are a few approaches to control Lotka-Volterra systems, we chose removal and addition of wolves to the system.

$$\frac{dx}{dt} = ax - bxy$$

$$\frac{dy}{dt} = -cy + dxy - u(t).$$

Cost function:

$$C(\alpha) = (p(T) - \frac{\theta_3}{\theta_4})^2 + (r(T) - \frac{\theta_1}{\theta_2})^2 + \int_0^T\alpha(t)^2dt$$

To propagate uncertainty, we run optimal control on each sample generated with MCMC, so we can create a posterior distribution of the optimal controls as well. We can then create confidence intervals with the standard deviation at each time point across some 5,000 samples.

Pseudo Spectral Method

We approximate our control and states using orthonormal Legendre polynomials, then substitute these approximations into our cost function and dynamical constraints to turn our ODE system into an algebraic one. Once this is done, we can use Gauss quadrature to optimize the new cost function, subject to the new dynamical constraints.

Pontryagin Maximal Principle

After constructing the Hamiltonian, we formulate the optimal control in terms of the costate variable, $\lambda$. We can then use the Shooting Method which utilizes an initial guess, $\lambda_0$, then uses root-finding to find the right value for $\lambda$ that satisfies the boundary conditions.

So what did we do?

We have created a full pipeline to take in noisy data, estimate the parameters of a model, derive and implement an optimal control for the system, and propagate uncertainty to quantify the confidence in our decision-making.

Poster: Download the Poster (PDF)

Slides: Download the Midterm Presentation (PDF)

Model Aware and Data Driven Inference

Tue, 01 Jul 2025 00:00:00 +0000

This blog post was written by Alexander DeLise, Kyle Loh, Krish Patel, and Meredith Teague and published with minor edits. The team was advised by Andrea Arnold and Matthias Chung. In addition to this post, the team has also created a poster and filmed a poster blitz video.

Introduction

Inverse problems arise naturally whenever we want to recover hidden information from indirect, often noisy measurements. Mathematically, an inverse problem usually takes the form:

$$ \mathbf{y} = \mathbf{F}(\mathbf{x}) + \varepsilon, $$

where:

$\mathbf{x}$ represents unknown parameters we aim to determine,
$\mathbf{F}$ is a known process (forward operator) that describes how these parameters produce observable data,
$\mathbf{y}$ are the observed measurements,
$\varepsilon$ is noise or error in the measurement.

Why Inverse Problems are Difficult

Inverse problems are critically important because their solutions drive decisions across science and engineering. However, solving inverse problems directly is notoriously challenging and often impossible due to several inherent issues:

Ill-posedness: Solutions may not exist, be unique, or depend continuously on the input data.
Noise Sensitivity: Small errors or noise in measurements can lead to significantly inaccurate solutions.
Computational Difficulty: Direct inversion of the operator $\mathbf{F}$ is frequently computationally infeasible or unstable.

Because of these difficulties, traditional methods often fail, motivating the search for more robust and insightful approaches.

Neural Networks as a Potential Solution and the “Black Box” Issue

Machine learning, particularly neural networks, has emerged as a promising solution. Neural networks can implicitly learn complex relationships from data, providing accurate approximations without needing explicit inversion.

However, a major drawback is their “black box” nature: they provide little to no insight into why and how they work. In fields where reliability and interpretability are crucial, this lack of understanding severely limits their practical adoption.

Our Approach: Linear Encoder-Decoder Networks

To address this interpretability gap, we investigate linear encoder-decoder (ED) networks. Linear ED networks simplify neural network architecture while maintaining the capacity to model inverse problems effectively:

$\mathbf{E}$ - Encoder: Compresses the input data into a lower-dimensional representation. This is our latent variable $\mathbf{z}$
$\mathbf{D}$ - Decoder: Attempts to reconstruct the original data from this reduced representation.

The linearity of these networks allows us to derive clear mathematical expressions and theoretical insights.

How Do We Decide What’s Best?

Choosing a way to measure success is crucial. We use Bayes risk minimization because it explicitly minimizes the expected reconstruction error:

$$ \min_{\operatorname{rank}(\mathbf{A}) \leq r} \ \mathbb{E} \left|\mathbf{A}Y - X \right|_2^2 , $$

where:

$\mathbf{A}$ is the mapping (network) we want to learn,
$Y$ are the observed measurements,
$X$ is the true unknown data.

Note that $X, Y$ are now random variables, and their realizations can be thought of as our datapoints. Bayes’ Risk Minimization helps us systematically find solutions that are expected to perform best.

Our Results in Action

Let’s take a look at two common scenarios in inverse problems and compare how our theoretical optimal mappings perform against the learned mappings by our ED networks.

Linear Denoising

The Theory

The Linear Denoising Problem is one of the simplest yet most informative cases of an inverse problem. Here, we aim to recover the original signal $X$ from noisy observations $Y$ that are direct perturbations of $X$, i.e., there is no intermediate transformation like a forward operator. Mathematically, we assume

$$ Y = X + \mathcal{E}, $$

where $\mathcal{E}$ is a random noise term.

Our goal is to find a low-rank linear map $\mathbf{A}$ that minimizes the expected squared reconstruction error between the predicted and true signals:

$$ \min_{\operatorname{rank}(\mathbf{A}) \leq r} \ \mathbb{E} \ \left| \mathbf{A} Y - X \right|_2^2. $$

This optimization is framed through the lens of Bayes’ Risk Minimization, which allows us to derive closed-form solutions for the optimal map. Just like in more complex inverse problems, we assume that $X$ and $\mathcal{E}$ are random variables with finite second moments, and we use symmetric matrix decompositions to analyze the structure of their variability.

To do this, we compute the second-moment matrices:

$\mathbf{\Gamma}_X = \mathbb{E}[XX^\top]$ for the clean data,
$\mathbf{\Gamma}_{Y} = \mathbb{E}[YY^\top]$ for the noise.

These are decomposed symmetrically, for instance via Cholesky decomposition or eigendecomposition, into:

$$ \mathbf{\Gamma}_{X} = \mathbf{L}_{X} \mathbf{L}_{X}^\top, \quad \mathbf{\Gamma}_{Y} = \mathbf{L}_{Y} \mathbf{L}_{Y}^\top, $$

where $\mathbf{L}_{X}$ and $\mathbf{L}_{Y}$ need not be full rank. The solution to the linear denoising optimization problem is given by

$$ \mathbf{A}_{\text{opt}}^r = \left( \mathbf{\Gamma}_X \mathbf{L}_Y^{\dagger , \top} \right)_r \mathbf{L}_Y^{\dagger}, $$

where $(\cdot)_r$ denotes the rank-$r$ truncated Singular Value Decomposition (SVD) of a matrix. This provides a clean analytic expression for the best low-rank denoiser under the assumed distributions.

The Experiment

In our experimental setup, we consider biomedical image data drawn from the MedMNIST dataset. Each image is first vectorized to form a column vector $\mathbf{x}_j \in \mathbb{R}^{784}$. To simulate the denoising problem, we add Gaussian white noise to each vector:

The noise is drawn independently from a zero-mean Gaussian distribution with standard deviation $\sigma = 0.05$.
This yields the observed measurement:

$$ \mathbf{y}_j = \mathbf{x}_j + \varepsilon_j, \quad \text{where } \varepsilon_j \sim \mathcal{N}\left( \mathbf{0}, \ 0.05^2 \cdot \mathbf{I} \right). $$

We then concatenate these entries into the matrices $\mathbf{X}, \mathbf{Y}$ for our data and observations, respectively. This setup allows us to test how well both learned and theoretical low-rank mappings can remove noise and recover the original image signals.

The Results

We compare our Bayes-optimal mappings $\mathbf{A}_{\text{opt}}^r$ against learned linear encoder-decoder mappings $\mathbf{A}_{\text{learn}}^r$ trained using gradient descent to minimize empirical reconstruction error. As expected, the theoretical mappings consistently outperform the learned ones, particularly at low ranks, where the model must compress the data most aggressively.

This demonstrates that even in a simplified setting, our analytic solutions are highly efficient at recovering the true signal from noisy observations using only a small number of latent features.

Inverse End-to-End Problem

The Theory

The Inverse End-to-End Problem is a more general formulation of the data denoising problem, and it refers to the task of recovering the original unknown parameters $X$ from indirect, noisy observations $Y$ that have been passed through a known forward process $\mathbf{F}$ and perturbed by some noise $\mathcal{E}$. Mathematically, we observe

$$ Y = \mathbf{F} X + \mathcal{E}, $$

and seek to find a low-rank linear map $\mathbf{A}$ that best approximates the inverse mapping, i.e., recovers $X$ from $Y$ by minimizing the expected reconstruction error:

$$ \min_{\operatorname{rank}(\mathbf{A}) \leq r} \ \mathbb{E} \ \left| \mathbf{A} Y - X \right|_2^2. $$

Our theoretically optimal mapping has the form

$$ \mathbf{A}_{\text{opt}}^r = \left( \mathbf{\Gamma}_X \mathbf{F}^\top \mathbf{L}_Y^{\dagger , \top} \right)_r \mathbf{L}_Y^{\dagger}, $$

where $\mathbf{\Gamma}_X$ is the second-moment matrix of the random variable $X$, $\mathbf{L}_Y$ comes from a symmetric decomposition of the second-moment matrix of the random variable $Y$ (i.e. $\mathbf{\Gamma}_Y = \mathbf{L}_Y \mathbf{L}_Y^\top$), and $(\cdot)_r$ denotes the rank-$r$ truncated SVD of a matrix, as mentioned before.

The Experiment

In this numerical experiment, we define the forward process $\mathbf{F}$ as a full rank Gaussian blur operator. Specifically:

The blur is implemented using a $5 \times 5$ Gaussian kernel with standard deviation $\sigma = 1.0$.
This is followed by the addition of Gaussian white noise with standard deviation $\sigma = 0.05$.

Again, our data comes from the MedMNIST dataset. Our observed data is thus generated by:

$$ \mathbf{y}_j = \mathbf{F}\mathbf{x}_j + \varepsilon_j, \quad \text{where } \varepsilon_j \sim \mathcal{N}\left( \mathbf{0}, \ 0.05^2 \cdot \mathbf{I} \right). $$

Again, we then concatenate these entries into the matrices $\mathbf{X}, \mathbf{Y}$ for our data and observations, respectively.

The Results

The animation below provides a clear demonstration of how the optimal rank-$r$ mapping $\mathbf{A}_{\text{opt}}^r$ consistently outperforms the learned $\mathbf{A}_{\text{learn}}^r$, especially at very low ranks.

Once again, this means that, even at very low ranks, our Bayes’ Risk-derived mappings recover the original parameters $X$ more accurately than the learned mappings, and hence do a better job of extracting what we truly care about from noisy, indirect measurements.

Acknowledgements

This work was conducted as part of the NSF REU Computational Mathematics for Data Science program at Emory University. The authors acknowledge NSF DMS-2349534 for support of this research.

Automatic Differentiation for Image Registration

Mon, 08 Jul 2024 00:00:00 +0000

This blog post was written by Cash Cherry, Warin Watson, and Rachelle Lang and published with minor edits. The team was advised by Lars Ruthotto. In addition to this post, the team has also given a midterm presentation, filmed a poster blitz video, created a poster and written a manuscript.

Image Registration

Image registration is the problem of finding a transformation which best “aligns” one image with another image. See FAIR for more detail on the problem. Some direct impacts of registration include

Medical Imaging: Accurate diagnosis, treatment planning, and monitoring of diseases.
Remote sensing: Analyze changes in the environment, such as deforestation, urbanization, and disaster assessment.
Computer Vision and Robotics: Object recognition, 3D reconstruction, and augmented reality applications.

Our Contributions

Over the course of the eight-week REU program, our team successfully implemented a multitude of image registration tools into working code. The variety of tools we implemented showcases how machine learning frameworks like PyTorch and JAX simplify code and saves programming time. Particularily, we highlight how auto-differentiation makes programs simpler. Below, we outline three different projects we pursued during the program.

Homotopy Optimization

The far left column shows the template $\mathcal{T}$, to be aligned with the reference $\mathcal{R}$. The middle column shows a non-ideal solution, a local minimizer. The far right column shows how with homotopy, a global minimizer can be obtained.

The main challenge in image registration is that the problem often lacks a clear or unique solution, and the approach to solving it is not straightforward. However, the problem becomes simpler when the images are blurred or smoothed. Computing the best transformation, or global minimizer, for a highly smoothed version of the image is more straightforward. Homotopy methods take advantage of this by first finding a global minimizer when the images are very smooth, and then tracing the path of the global minimizer through more and more detailed versions of the image back to the original problem. The figure above demonstrates that without homotopy optimization, we find a non-ideal transformation, but with homotopy omptimization, we can achieve a satisfactory one.

The difficulty with implementing homotopy optimization for image registration lies in having to compute a Hessian matrix, which is a laborious process to derive. However, this is made simpler with automatic-differentiation.

Super Resolution

Given a sequence of low resolution images of a subject in motion, the goal of super-resolution is to constructa high resolution of the first frame of the sequence. For example, a super resolution algorithm would be able to construct a high resolution image of a heart given only low resolution images taken at different points in time as the heart beats.

The super-resolution problem is like the image registration problem in that the goal is to transformations that align images. Particularily, we find a transformation for each low resolution image that aligns it with the high resolution image. But since the high-res image is unkown, we construct it simultaneously, optimizing for both the transformations and the high resolution image together.

Given low resolution images $\mathbf{d}$, these each have a high resolution counterpart labeled by the letter $\mathbf{f}$, which we find through our optimization.

We employ advanced techniques like variable projection to enhance the optimization process, improving the accuracy and efficiency of super-resolution.

Registration with Intensity Dynamics

In medical imaging, a contrast agent is often injected at the beginning of a sequence of images taken over time. This leads to observable dynamics - intensity changes over time - which contain medically relevant information. Correctly observing these dynamics requires tracking the same region through each image, which makes image registration important for this problem. However, to solve a registration problem, we have relied on the assumption that intensity is preserved- that is, that no such dynamics are present (There are some interesting math connections between intensity preservation and Neural ODEs).

The synthetic data in the above figure shows the data typical of this problem- the medulla and cortex of the kidneys experience different intensity changes over time after the injection of contrast. The original MRI is from Figure 4 of Registration of Dynamic Contrast Enhanced MRI with Local Rigidity Constraint.

This problem is of the same mathematical form as the Super Resolution problem. Previous work has used similar techniques as in super-resolution to estimate maps of parameters in various dynamical models. We are working on estimating the intensity changes directly.

Conclusion

In implementing three image-registration related problems with python machine learning tools, we have demonstrated the benefits of using these tools to further improve the state of image-registration. We have laid the groundwork for using homotopy methods for image registration, and have implemented variable projection with automatic differentiation. We hope that image registration researchers continue to take advantage of these tools in the future.

Efficient Processing of Image Sequences with Krylov Subspace Recycling

Mon, 08 Jul 2024 00:00:00 +0000

This blog post was written by Clara Armstrong, Olivia Kallay, and Srijon Sarkar and published with minor edits. The team was advised by Lucas Onisk. In addition to this post, the team has also given a midterm presentation, filmed a poster blitz video, created a poster and written a manuscript.

Background

The image deblurring problem can be framed as a least-squares problem. We let $A$ be our blurring matrix, the convolution of a Point-Spread Function (PSF) and a boundary condition. The true image is represented by $X$, and the blurred by $B$; we vectorize both to get our linear system, $Ax=b$. For example, we see below an image blurred using PSF Gauss with 2% noise:

Consider the solution $x$ written with respect to the singular value decomposition (SVD) assuming $A$ is invertible. The blurred imagine contains error such that $b=\hat{b}+e$; we observe that

$$ x = V\Sigma^{-1}U^Tb = \sum_{i=1}^{n}\frac{u_i^Tb}{\sigma_i}v_i = \sum_{i=1}^{n}\frac{u_i^T\hat{b}}{\sigma_i}v_i+ \sum_{i=1}^{n}\frac{u_i^Te}{\sigma_i}v_i $$

where the first sum is our true solution and the second is our inverted noise. For image deblurring problems, the $u_i^Te$ terms are constant in magnitude. $A$ is an ill-conditioned matrix, causing the solution to be dominated by the inverted noise, which becomes very big as the singular values $\sigma_i$ decay to numerical zero.

Our Approach

We will focus on two Krylov techniques: Arnoldi and Golub-Kahan bidiagonalization. Using Krylov methods allows us to work in a smaller subspace that captures enough information about the system to compute an approximate solution, making our problem easier to solve than solving directly.

A Krylov subspace is the span of repeated applications of a matrix $A$ to a vector $b$, or

$K_p(A, b) = \text{span}\{b, Ab, A^2b, ... , A^{p-1}b\}$

The Arnoldi relation is at the core of the Generalized Minimal Residual (GMRES) method for solving square nonsymmetric linear problems. After $p$ steps of the Arnoldi iteration, we develop the relation

$AV_p = V_{p+1}H_{p+1}$

where $A$ is the square operator from the linear system, $V_{p+1}$ has orthonormal columns that span the subspace $K_p(A, b)$, $V_p$ contains the first $p$ orthonormal columns of $V_{p+1}$, and $H_{p+1}$ is an upper Hessenberg matrix (an upper triangular matrix that contains an additional subdiagonal band) of the scalar coefficients from the orthonormalization process of Gram-Schmidt, used in Arnoldi.

Golub-Kahan bidiagonalization (GKB) is associated with the LSQR method, where $A$ could be non-square. GKB generates orthonormal vectors that span the following spaces:

$$K_p(A^TA, A^Tb) \text{ and } K_p(AA^T, b)$$

After $p$ steps of GKB, we have the following relationships

$$AV_p = U_{p+1}B_{p+1} \text{ and } A^TU_{p+1} = V_{p+1}\tilde B_{p+1}$$

where $V_{p+1}$ and $U_{p+1}$ contain orthonormal columns spanning $K_p (A^TA, A^Tb)$ and $K_p(AA^T, b)$, $B_{p+1}$ is the lower-bidiagonal, projected variant of $A$, and $\tilde B_{p+1}$ is square due to the inclusion of an additional column.

Our Observations

Since GMRES and LSQR require many matrix-vector products, we desire to gain efficiency through recycling strategies to circumvent such repetitive computation while solving sequences of closely related problems.

We observed for that $r^{(2)}=Ax_{approx}^{(1)}-b^{(2)}$, the residual of the second problem using our solution to the first (seed) problem, the Reconstructive Residual Norm (RRN) satisfies

$$DP \leq \frac{||r^{(2)}||}{||b||} < 1,$$

where DP represents a breakout parameter from the Discrepancy Principle, $\tau\delta$, in which $\tau$ is a safety parameter close to one and $\delta$ is an upper-bound on the erroneous right-hand sides.

Although the residual is above the breakout level given by the DP, it remains significantly lower than if we started from the zero vector (which would produce a relative residual of one). Thus, by recycling basis vectors from previous solutions to solve subsequent problems, we drastically reduce the RRN.

We note that with these types of methods, residuals decrease, but errors demonstrate semiconvergent behavior; we see the error decrease until a certain point at which it will begin growing exponentially.

As the basis from the seed problem contains useful information for non-arbitrarily chosen right-hand sides (image frames for our purposes), we incorporate this information by calculating

$$\text{res} = \frac{r^{(2)} - V_pV_p^Tr^{(2)}}{||r^{(2)} - V_pV_p^Tr^{(2)}||}$$

and appending it in $[V_p \quad res] = \tilde V_{\ell}$ thereby forming the flexible Arnoldi method as $A[\tilde V_{\ell}] = \tilde U_{\ell+1}J_{\ell+1}$ at step $\ell$. $U$ and $J$ come from completing the back half of Arnoldi, respectively representing a new basis and a projected variant of $A$.

Future Directions

Eventually, we will run out of memory as we continue iterating flexible Arnoldi and storing more basis vectors. To avoid this, we will utilize different compression strategies to decide which basis vectors should be retained for solving subsequent problems, and which may be discarded. The compression strategies we aim to use are

Truncated Singular Value Decomposition
Reduced Basis Decomposition
Solution-Oriented Compression

We aim to combine compression techniques with flexible algorithms, including extending our work to non-square operators to replicate flexible GKB for LSQR.

Further, as these iterative processes continue and our subspace grows, it might capture information from small singular values. So, we discern the need for penalized least squares using Tikhonov regularization for our projected problems as

$${\min_\textbf{x}} ||A\textbf{x}-\textbf{b}||_2^2 + \lambda||\textbf{x}||_2^2.$$

So far, we have only focused on sequences of images, such as those that may arise from a video. However, these techniques can also be useful for other sequences of linear inverse problems, such as those arising from medical imaging.

References

On the Use of Arnoldi and Golub-Kahan Bases to Solve Nonsymmetric Ill-Posed Inverse Problems, Brown, A. M., (2015)
GMRES: A Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems, Saad, Y. and Schultz, M.H., (1986)
Hybrid Projection Methods with Recycling for Inverse Problems, Jiang, J., Chung, J., and de Sturler, E., (2021)

Improving VAEs with Normalizing Flows

Mon, 01 Jul 2024 00:00:00 +0000

This post was written by Callihan Bertley, Claire Gan, Rishi Leburu, and Malia Walewski. The team was advised by Dr. Deepanshu Verma. In addition to this post, we have also created slides for a midterm presentation, a poster blitz video, and a poster.

Project Overview:

Variational Autoencoders (VAEs) have emerged as powerful deep generative models in recent years. Our project aims to improve the expressiveness of VAEs by incorporating different types of normalizing flows, specifically Inverse Autoregressive Flows (IAFs) and Partially Convex Potential Maps (PCP-Maps).

What are VAEs?

VAEs use neural network architectures to learn a compact representation of input data through an encoder and generate new data through a decoder. The encoder compresses the input into a latent vector, outputting a mean and standard deviation for each latent variable to approximate the true latent representation (posterior distribution). Samples from this distribution should resemble the input when passed through the decoder.

Figure 1: Architecture of Variational Autoencoder visualization

VAEs typically use a Gaussian approximation for the posterior distribution. However, since the true posterior distribution often deviates from normality, we aim to find a more expressive posterior.

How to improve VAEs?

To improve VAEs, we can apply normalizing flows (NFs) to the Gaussian approximated posterior distribution from the encoder. NFs apply a series of invertible transformations, parameterized by neural networks that can be optimized during VAE training.

The figure below demonstrate how normalizing flow are able to transform into a simple distribution to a more complex distribution.

Figure 2: Comparison between before (left) and after (right) of training using NFs.

Our goal is to show that models with normalizing flows can perform better than standard VAEs.

Methods

To address this challenge, we compare three different models with VAE using the MNIST handwriting dataset. In our experiment, we follow a similar implementation to L. Midgley’s standard VAE and IAF VAE. Then we implemented the PCP-Map.

Standard VAE The standard VAE uses a Gaussian approximation for both the encoder and latent distribution. For further details, see 1 and 4.
Inverse Autoregressive Flow (IAF) VAE The IAF VAE applies a series of invertible, autoregressive transformations to the Gaussian approximated distribution in the encoder. For each transformation, an autoregressive neural network takes in the input latent sample and a conditional vector dependent on the given inputs and outputs mean and standard deviation of the tranfromation.

The Jacobian matrices of the transformation are lower triangular, which makes computing the determinant, and hence the loss, not expensive.

Figure 3: Architecture of IAF VAE visualization
Partially Convex Potential Map The PCP-Map VAE is another way to transform the input, Gaussian distribution to a more complex distribution. This is achieved by parameterizing a transformation as the gradient of a scalar-valued partially input convex neural network. These maps are constructed to have a triangular Jacobian matrix. This structure allows the Jacobian to be computed as the Hessian of the PICNN.

Optimizing Evidence Lower Bound (ELBO) We use the KL-Divergence loss function to measure model performance, aiming to minimize the KL divergence between our approximate posterior and the true posterior by minimizing the negative Evidence Lower Bound (ELBO). The figure belwo shows how ELBO is measured.

Figure 4: Understanding of the KL-Divergence (Image Source: https://mbste.github.io/posts/vae/)

We compare the models using negative ELBO loss, reconstructed images, and contour plots.

Results and Conclusion

After training for approximately 2000 epochs, the IAF VAE produced slightly better reconstructed images than the standard VAE. The contour plot for the IAF VAE showed a non-Gaussian distribution more similar to the input data distribution, suggesting improved expressiveness.

References

[1] D. P. Kingma and M. Welling, “Auto-Encoding Variational Bayes,” 2014.

[2] D. P. Kingma, T. Salimans, R. Jozefowicz, X. Chen, I. Sutskever, and M. Welling, “Improving variational inference with inverse autoregressive flow,” 2017

[3] L. Midgley, “Improving variational inference with inverse autoregressive flow,” University of Cambridge, Tech. Rep. 2021. [Online]. Available: https://github.com/lollcat/Autoencoders-deep-dive/blob/Pytorch/Report.pdf

[4] L. Ruthotto and E. Haber, “An introduction to deep generative modeling,” GAMM-Mitt., vol. 44, no. 2, pp. Paper No.e202 100 008, 24, 2021. [Online]. Available: https://doi.org/10.1002/gamm.202100008

[5] Z. O. Wang, R. Baptista, Y. Marzouk, L. Ruthotto, and D. Verma, “Efficient neural network approaches for conditional optimal transport with applications in bayesian inference,” 2023. [Online]. Available: https://arxiv.org/abs/2310.16975

Optimal Experiment Design and Image Reconstruction using Generative Methods

Mon, 01 Jul 2024 00:00:00 +0000

This blog post was written by Spiros Manolas, Anish Mitagar, and Nela Riddle and published with minor edits. The team was advised by Dr. Nicole Yang.In addition to this post, the team has also given a midterm presentation, filmed a poster blitz video, created a poster and written a manuscript.

Reconstructing images from noisy and indirect observations is an ill-posed inverse problem critical for many medical and imaging applications However, obtaining such indirect observations is expensive time and cost wise. We wish to improve the measurement process by finding the best sub-sampled indirect measurements/observations to take, and reconstruct a quality image from the best sub-sampled set.

Example Training Process

Example Inference Process

We explore how Normalizing Flows, a generative method, and autoencoders can be leveraged to learn quality image generation from a sub-sampled set of indirect observations. We also explore introducing a differentiable or learnable mask function into our model as design parameter d, to potentially learn the best sub-sampled set of indirect observations to reconstruct from using Normalizing Flows.

Accurately Classifying Out-Of-Distribution Data in Facial Recognition

Fri, 01 Dec 2023 00:00:00 +0000

This blog post was written by Gianluca Barone, Aashirit Cunchala, and Rudy Nunez and published with minor edits. The team was advised by Nicole Yang. In addition to this post, the team gave a midterm presentation, made a poster blitz video, wrote a paper, and won the best poster award for their poster.

Introduction

As facial recognition technology is deployed at large scales, it is crucial to ensure its fairness. A fair model has approximately equal performance in all different classes. However, models are often unfair due to bias, which can occur for several reasons, such as having an unbalanced training set. One common assumption in machine learning is that the training and test images are from the same distribution, but in real-world applications, that is often not the case; for example, if a model is trained on an imbalanced training set, it may not do well on a balanced set. Due to this, the minority class can be overpowered by the majority class.

The core goal of this project is to increase a model’s fairness when encountering images outside of the training distribution. Such out-of-distribution (OOD) images belong to a distribution different from the one the model was trained on. We used two different face datasets to have different distributions: our training dataset was UTKFace, an imbalanced dataset of 20,000 images, and our testing dataset was FairFace, a balanced dataset of 100,000 images.

Outlier Exposure

One approach to making a model more familiar with out-of-distribution data is through outlier exposure. In outlier exposure, a model is exposed to both in-distribution and out-of-distribution data during training stages. When it encounters the OOD data during testing, it is more likely to classify it accurately. To do this, we attempted to collect the furthest OOD images from each dataset. One approach to finding these images was using Kullback–Leibler (KL) divergence based on pixels. KL divergence measures the distance between two different distributions. Using this, we found the images with the largest KL distance, which means they are the furthest from the distribution.

In outlier exposure, the loss function is modified to include an additional term. In this term, there is a parameter lambda that scales the significance of the other term, which we were able to train to find an optimal lambda. We used the KL divergence to relate the loss function to the distance between distributions.

Using the KL score, we have noticed that UTKFace and Fairface have quite similar distributions due to the low KL score. Due to this, outlier exposure may not be as effective since it does better when the distributions are more dissimilar. To rectify this, we increased the training set by combining UTKFace with 20% of outliers from FairFace. We found these outliers by collecting the 20% of images that are the furthest from the mean distribution of FairFace chosen through KL divergence.

Main Results

We observe that the accuracy and other metrics of the model can be improved by applying Outlier Exposure, incorporating a trainable weight parameter to increase the model’s emphasis on outlier images, and by re-weighting the importance of different class labels. Some items of future work include using feature norms of the images and other outputs of the CNN to sort the images and improve classification. Further, one may adapt the activation features code into the outlier exposure code to see if it is more accurate than using the KL divergence based on pixels.

Reflection

This project taught us much about neural networks and how to improve facial recognition models. We also learned more about writing manuscripts and doing research. We plan on continuing our work on this project and exploring the possibility of using image features and additional weight parameters to classify out-of-distribution data.

Interested in Learning More?

Please see our award-winning poster.

Fast & Fair: Efficient Training of Fair Neural Networks

Fri, 01 Dec 2023 00:00:00 +0000

This blog post was written by Allen Minch, Hung Anh Vu, and Annie Warren and published after some minor edits. The team was advised by Dr. Elizabeth Newman. More information about this project can be found in our paper, poster blitz video, and poster.

(Un)Fairness in Machine Learning

Our project focuses on a key problem in machine learning: unfairness. How do we define unfairness, and where does it come from?

Classifiers in machine learning, such as deep neural networks, can be very good at classifying complex data, such as images. However, machine learning classifiers trained through supervised learning are designed to minimize the average classification error in the data they are trained on. This means they are inherently designed to be very “faithful” to the data they are trained on and to reflect patterns and correlations in that data. This can be a problem when such correlations involve a sensitive attribute - for instance, a correlation of a person’s race with an attribute we believe should not be related to race.

Suppose the real-world data set that a machine learning classifier is trained on is biased with respect to some sensitive attribute, like race. In that case, the classifier will, in trying to maximize its accuracy on the data it is being trained on, be “faithful” to the bias that it is given and be trained to make predictions that reflect this bias. While it could be fairly stated that these biased predictions are not the fault of the machine learning algorithm itself but the data on which it is trained, that doesn’t change the fact that, as a society, we would generally view it as unfair to base real-world decisions on using a machine learning classifier that makes biased predictions. This is especially true when the stakes are high, where a biased classifier prediction could mean someone getting unfairly arrested, unfairly not being hired for a job, or unfairly being denied a loan. It is thus in our interests as a society to be careful about using machine learning classifiers that might be biased in their predictions and try to tweak how they are trained to reduce this unfairness.

Improving Fairness Through Adversarial Training

We develop efficient adversarial training techniques to reduce the unfairness of classifiers in machine learning in the context of a sensitive attribute like race. In our project, we are dealing with supervised machine learning, where a classifier is trained on a dataset with many samples of input features and a corresponding true label. Because the context is supervised, we have true labels as information to help us evaluate the classifier’s performance. To assess our success in improving the fairness of a classifier, we need to have fairness metrics by which to evaluate a classifier.

Why would we think that adversarial training would help improve fairness? There are a couple of lenses for understanding why we might expect this to improve fairness. One lens has to do with the idea of overfitting. Intuitively, a primary reason we expect a classifier to be unfair is that it is overfitting to the bias in the particular data it is trained on. Adversarial training, by nature, tries to combat overfitting to the training data with its efforts to make a classifier more robust by giving perturbations to the training data points. Thus, we might expect combating overfitting with adversarial training also to combat unfairness, making the classifier less sensitive to bias in the given data. A second lens is that intuitively, a situation we would tend to view as most flagrantly unfair is when two individuals are quite similar yet happen to be part of different sensitive groups and turn out to be classified differently by a classifier. Perhaps some unfairness we observe in a classifier manifests in this kind of scenario. If we can make our classifier more robust - so that two nearby points are more likely to be classified the same way - perhaps this kind of unfairness could be reduced, which could minimize the unfairness of our classifier overall. Since adversarial training improves robustness, this would be another lens to understand why we might expect adversarial training to improve fairness.

We develop a second-order optimization method for solving the adversarial training problem. When solving an optimization problem, first-order methods are used when we only need to compute the gradient. This means we rely solely on first-order information to optimize our function. In the unconstrained case, we can take a step as large as necessary in the direction of the gradient for maximal increase/decrease. However, for our approach, we must satisfy a constraint requiring us to project our solution onto the constraint set. Computing the gradient and projecting the solution can become very computationally expensive. Therefore, with some mathematical speculation, we could utilize second-order information (computing the Hessian) to solve the optimization problem more efficiently.

Evaluating Fairness

We used multiple metrics for evaluating fairness with respect to sensitive attributes. To describe these intuitively, imagine that we had a dataset of individuals whose race is either white or non-white and with true labels as to whether or not they have committed a crime in the past. Suppose we used this data to train a classifier designed to predict whether or not an individual will commit a crime in the future and then used a test dataset to see how the classifier performs.

Independence This means that the prediction of our classifier is uncorrelated with the sensitive attribute; a positive prediction is equally likely for all groups. In our example, independence being satisfied in a test dataset would mean that if 15% of white individuals in the dataset are predicted to commit a crime in the future, then 15% of non-white individuals are also. (Note: Y represents the true label, Y_hat the predicted label, and S the sensitive attribute in the graphic below. The graphic shows two probabilities that would have to be equal for independence to be satisfied).
Separation This means that the prediction of our classifier is conditionally independent of the sensitive attribute, given an individual’s true label.
Sufficiency: This means that our classifier’s predictive usefulness is equal across sensitive groups. If sufficiency is satisfied, then for each $(x, y)$, the probability that an individual’s true label is $x$ given the value of their predicted label is $y$ is the same across sensitive groups.

We implemented these fairness metrics in Python with PyTorch tensors. We worked with binary classification and sensitive attributes in our project, which meant that being given PyTorch tensors containing each individual’s sensitive attribute, true label, and predicted label respectively, we could obtain the proportions needed for evaluating our fairness metrics leveraging some boolean indexing tricks.

Coding Strategies and Main Findings

Our adversarial training problem consists of two optimization problems: an inner one and an outer one. The outer optimization problem is the typical machine learning problem of finding optimal classifier parameters. The inner optimization problem consists of, for each point in the training data set, finding a perturbation to that point within some radius that maximizes the value of the loss function. In practice, to computationally solve a two-layered optimization problem like this, the approach we take is to solve the inner optimization problem at each point, update parameters in the outer optimization problem based on these solutions, solve the inner optimization problem again, and continue going back and forth between the two problems.

As it turns out, robust training, because it requires solving the inner optimization problem at every training data point in every epoch of the outer optimization problem, can be pretty slow computationally compared to simply doing non-robust training. Thus, as the FastNFair team, we focused on solving the inner optimization problem as quickly and efficiently as possible.

Our numerical results on three datasets show that our method is more efficient than first-order methods. We also observe that adversarial training can improve fairness but potentially at the cost of accuracy. If robust training with a certain radius improves fairness, it appears to improve fairness by larger margins than random perturbation; solving the optimization problem well is worthwhile.

Interested in Learning More?

Please see our poster.

Investigating Impacts of Environmental and Socioeconomic Data

Fri, 01 Dec 2023 00:00:00 +0000

This blog post was written by Riley Chen, Mason Lu, Matilda Slosser, and Aneesh Srinivas and published with minor edits. The team was advised by Julianne Chung, Matthias Chung, Elizabeth Newman. In addition to this post, the team has also created a poster.

Introduction

Nitrogen Dioxide ($NO_2$) is a common gaseous pollutant. It is a by-product of combustion, making its local concentration levels directly tied to gas vehicle emissions and fossil fuel-sourced energy production. In the United States, cars are the largest source of $NO_2$, which links high concentrations to urban environments.

Tracking and predicting $NO_2$ concentrations is an issue of public health and social justice. Certain individuals, like children, older adults, or those with pre-existing lung conditions, are susceptible to poor health outcomes after exposure to $NO_2$. Marginalized people tend to be forced through economic and social structures into areas with relatively high $NO_2$ concentrations, resulting in disproportionate exposure.

Our data is generated from the Air Quality System (AQS) $NO_2$ monitoring networks over the contiguous United States of the Environmental Protection Agency (EPA). This data is from Jan 1st, 2000, to Dec 31st, 2016, with 6210 points. Our main goal is to create a model to predict average $NO_2$ concentration across the contiguous United States. We also aimed to analyze the relationship between $NO_2$ concentration and the Social Vulnerability Index (SVI) geographically.

Model-Driven Approach

We plotted the daily average concentration of $NO_2$ over the US from 2000 to 2016 and observed that the data has seasonal oscillations and a decaying trend.

Therefore, we incorporated this information to build the model shown below.

$ y_{\mathrm {model}}=p_1+p_2 e^{p_3(t-p4)}+p_5 \cos(2πp_6(t-p_7)). $

where

$p_1$ - our intercept (average $NO_2$ on December 31st, 1999).
$p_2$ - starting value of the exponential decay
$p_3$ - exponential decay scale factor
$p_4$ - the starting time. Since the data started in 2000, we expected $p_4 ≈2000$.
$p_5$ - the amplitude of the seasonal oscillation.
$p_6$ - the frequency of the cosine component.
$p_7$ - the cosine shift, which increases our model’s adaptability

To learn more about the parameters, we aimed to randomly generate samples, hoping that if the sample space was large enough, a good approximation for the distribution of parameters could be found. A Markov Chain Monte Carlo (MCMC) method was utilized to generate the samples, which implemented Bayes’ theorem. The specific algorithm we used is called Adaptive Metropolis. We fixed $p_4$ to be 2000 and $p_6$ to be 1 to ensure enough random walk for the algorithm. We generated 400,000 sets of values for the parameter set.

Plotting the samples component-wise, we observe that each graph only has one peak. That means that the samples agree and gives us some confidence that each parameter is within the range in its own graph. After investigating each parameter, we wanted to explore how each pair of parameters was related. Most were Gaussian if the outliers were ignored. However, $p_1$ and $p_3$ were highly correlated.

We also calculated the posterior function value of the new parameter set for each random set we generated. We then updated our current maximize-a-posterior (MAP) estimate for the parameter set. This estimate gives us the most probable value for the parameter sets given this data. The model with this parameter set is plotted with a 95 percent prediction interval, as shown below. The interval is wider at the peaks and troughs, making it harder to predict the exact value at the highest and lowest points in each oscillation cycle.

Main Findings

Our model-driven approach with appropriately selected parameters reasonably predicted the average daily $NO_2$ concentrations. Including a weight matrix in the objective function resulted in a better data fit. Our visual inspection of the posterior MCMC samples suggests high levels of agreement and demonstrate little uncertainty in their predictions. We also experimented with an LSTM model but could not achieve competitive results. Although weak for some years, we observe correlations between the SVI and NO2 concentration, most noticeable in 2010.

Interested in Learning More?

Please see our poster.

Low-Level Ozone Prediction

Fri, 01 Dec 2023 00:00:00 +0000

This blog post was written by Antonio Gamboa, Lucretia Gant, Timothy Gant, Andrea Glover, and Julianna Long and published with minor edits. The team was advised by Bree Ettinger.In addition to this post, the team has also given a midterm presentation, filmed a poster blitz video, created a poster and written a manuscript.

Motivation

Our team is identifying potential correlations between ground-level ozone pollution and its social and economic disparities, reducing environmental justice. We aim to utilize the results from this investigation to advocate for equitable policies and regulations.

Bringing social justice to communities impacted by unidentified factors depends on the guidance from scientific research that exposes and discovers inequalities. Communities with lower socioeconomic status often experience a disproportionate burden of air pollution, including higher levels of ground-level ozone, due to factors like proximity to industrial areas, transportation hubs, and poorer air quality monitoring. However, it is unclear how ground-level ozone pollution becomes an issue of social justice and equitable protection of public health for all communities.

What is Ozone, and how is it formed?

Ozone is a molecule composed of three oxygen atoms; it is highly reactive and mostly found as a colorless and odorless gas. At high altitudes in the stratosphere, it protects us from the damaging UV sun rays> However, at the ground level, in the troposphere, ozone is a secondary pollutant. As a secondary pollutant, ozone, it is formed through complex chemical reactions of molecules in the air.

Ground-level Ozone (O3) is formed when nitrogen oxides (NOx) and volatile organic compounds (VOCs) react in the presence of sunlight. NOx is primarily emitted from vehicle exhaust, power plants, and industrial processes, while VOCs come from sources such as gasoline vapors, solvents, and certain industrial activities. These emissions are known as ozone precursors.

When these precursors mix in the lower atmosphere and are exposed to sunlight, a series of chemical reactions occur, leading to the formation of ground-level ozone .

How Mathematics may be used to better monitor and deal with Ozone?

Mathematics is currently used to study air pollution. Mathematics provides us with equations and algorithms to solve a problem through computation. The mathematical algorithms act as an exact set of instructions to be followed by the hardware in the computer. The result of such a combination of mathematics and computer science provides us with important functional models that simulate and predict ozone concentrations.

Mathematical modeling can be done by taking large data sets for ozone collected daily over the years and finding a function that fits the data well with a small percent error. Furthermore, computation techniques such as machine learning can be applied to the data to predict future ozone levels. The mathematical prediction of low-level ozone can inform air quality forecasts and help people determine which outside activities are safe.

What type of Mathematical Applications may be used to better understand Ground Level Ozone?

Functional linear regression models have recently shown their ability to predict low ozone levels. Specifically, functional models based on bivariate splines over triangulation approximate the spatially distributed ozone measurements on a surface. This powerful computation allows the forecast and the quantifying of the uncertainty associated with the predictions of ozone levels for a given day. A mathematical functional regression model is a statistical technique combining functional data analysis and regression analysis principles. It is used to model the relationship between a dependent variable and one or more independent variables when both are functions or curves rather than simple numeric values. The data points are considered as functions or curves rather than individual observations. Functional principal component regression combines functional principal component analysis (PCA) and regression analysis. It uses the principal components of the functional independent variables to represent the variation in the data and estimates the coefficients for predicting the functional dependent variable. In this project, we have used PCA as a powerful tool for analyzing and modeling relationships between ground-level ozone and the SVI index for communities near the Atlanta area.

Our preliminary results suggest a potential correlation between ground-level ozone and socially vulnerable neighborhoods. Social Vulnerability Indexes (SVI) identify communities predisposed to harm if exposed to a natural disaster or harmful conditions. These communities are particularly at risk because of specific factors related to their socioeconomic status, household characteristics, racial and ethnic minority status, and housing type. An SVI index is helpful to governments and communities because it identifies and acknowledges which additional support and attention is needed to mitigate the harmful impacts of harmful environmental exposure. Our model approximates the spatially distributed ozone measurements on a surface, thereby predicting low-level ozone. In addition, by including the SVI and the Environmental Justice Index (EVI), our model calls for a better predictive tool. This tool can inform air quality and help determine which communities are most at risk. Our computational mathematical method also quantifies the uncertainty associated with our predictions and helps us identify the spatial distribution of low-level ozone among the most vulnerable communities.

Interested in Learning More?

Please see our manuscript or poster.

Comparing Reinforcement Learning to Optimal Control Methods on the Continuous Mountain Car Problem

Fri, 05 Aug 2022 00:00:00 +0000

What is the best way to get a car out of the bottom of a hill?

Introduction

This blog post was written by Jacob Mantooth,Dewan Chowdhury, Arjun Sethi-Olowin and published with minor edits.The team was advised by Dr.Lars Ruthotto.In addition to this post, the team has also given a midterm presentation, filmed a poster blitz video, created a poster, published code, and written a paper.

The word around town is that reinforcement learning is the top dog and has the answers to all our problems. We wanted to see if that really was the case, so this summer we took a trip to Emory University where we looked at the continuous mountain car problem to see if reinforcement learning really, was the best. The continuous mountain car problem is an example of an optimal control problem. In the image below you can see an example of what the continuous mountain car problem looks like.

You may be asking yourself what is an optimal control problem? An optimal control problem is problem that consists of controlling a dynamical system to minimize (or maximize) a given objective function. In our case the continuous mountain can be modeled as a ODE where $u$ is some controllable function. In the continuous mountain car problem our control,$u$, is whether the car accelerates left or right. In an optimal control problem, we seek to optimize some objective function, in our case we will minimize the objective function. Our two objective functions are the running cost, which penalizes the car for acceleration. While our other objective function is the terminal cost which penalizes the car for not reaching the goal, the top of the hill, in time.

Why This Problem?

You may be wondering why choose the Continuous Mountain Car Problem? Here are a couple of reasons why we picked this example,

Established benchmark for RL models
2-D state-space allows for good plots and visualizations
Both RL and optimal control problem
Finite horizon (time)
Continuous state and motion

The whole reason we are doing this is because we want to compare three different ways of solving the continuous mountain car problem and see which one really is the best. Our three approaches are

Local solution using numerical ODE solvers and nonlinear optimization (baseline)
Reinforcement learning with actor-critic algorithm (data-based approach)
Optimal control using both model and data

Our Three Approaches

A Local Method

Our first method that we looked at during this REU was the local method. We tried to find the optimal control $u_h$ by formulating an optimization problem.

We first discretize the control, state and the Lagrangian.

Setting $z_h^{(0)}=z_t$ and $\ell_h^{(0)}=0$, allows us to use a forward Euler scheme for some control $u$.

We then approximate our objective function which yields the optimization problem

To solve our optimization problem, we used gradient descent. By taking an initial guess for $u_h$ and repeatedly updating $u_h$ using the gradient of the objective function and step size $\alpha$

$(u_h)_0 = \vec0$

$\vdots$

$(u_h)_6 = (u_h)_5 - \alpha( \nabla J((u_h)_5))$

$\vdots$

$(u_h)_* = (u_h)_19 - \alpha( \nabla J((u_h)_19))$

Below is a nice visual example of what all this math means. When the tail reaches the dotted line, it means our car has reached the top of the hill.

The graph shows us the position vs velocity of the car. In the graph the black dot represents t and the tail of the plot, when x-position is .45 is time T. In the graph we see the color change from red-blue, in our plot the blue color is when the car is accelerating, and control is positive but red otherwise.

Our next goal was how do we create a nice visualization of what the actual solution looks like.

We see in this video what our optimal local solution looks like. A couple of things that should be noted is, if we move the car to a new position then this local solution may no longer work. The same can be said if we slowed down/speed up a bit then this solution may not even let the car get to the top of the mountain. Another downside of the local is that it is a non-linear and non-convex problem which makes this method slow. This local solution will serve as a baseline so we can compare other methods to something to see which one is really the best.

Global Methods

Now that we have established a baseline, we will discuss our other two methods. Our other two methods that we will be looking at are global methods, the first being reinforcement learning method and the other being optimal control method. You may be asking yourself what is the difference? Reinforcement learning is more of a data driven approach while the optimal control method is a hybrid approach, using both a model and data.

Reinforcement Learning

Our first stop in exploring global methods is reinforcement learning. We will be using reinforcement learning with actor-critic algorithm. This approach is completely data-based approach. In reinforcement learning it has no knowledge of the model, it only considers the objective function. In reinforcement learning we would like to maximize a reward, so in our case we will maximize negative cost. Reinforcement learning is stochastic in two ways with initial position and action space which allows for exploration. In Reinforcement learning we are trying to estimate an optimal control policy. One of the big things that we have yet to discuss is, what is actor-critic algorithm ? The actor-critic (AC) architecture for RL is well-suited for a continuous action-space as in the continuous mountain car problem 1 . In the actor-critic algorithm the critic must learn about and critique whatever policy is currently being followed by the actor. We worked in the OpenAI gym mountain car environment, so we were able to find preexisting code for our project. We also were able to adapted the TD advantage actor-critic algorithm adapted from here. The thing is our preexisting code was not the same as our problem, so we had to modify it some. After some modification to the code, the following video is the results that we were able to get after many training cycles.

In this video we see that RL gave us a sub optimal solution compared to the local solution. You may also notice that in the reinforcement learning method our car takes an extra swing backwards to get to the top of the hill. In our reinforcement learning method it took 1000’s episodes just to get the car to our goal. We saw that reinforcement learning is very fragile, a couple of changes saw our success rate go from 70% to barely making it all. The picture below is position vs velocity of the car.

as you can see compared to the local solution, we see that the RL solution is very sub optimal solution.

Optimal control method

Our last two methods were vastly different with reinforcement learning using a data driven approach and the local method using a model-based approach. We will now be looking at the optimal control method which combines both model and data driven approaches. In this approach, we aim to adhere to the method discussed here. We will estimate the corresponding value function with neural network approximators utilizing feedback from the Hamilton-Jacobi-Bellman equation and Hamiltonian.

Using OC method we were able to produce the following

Once again, we created a position vs velocity of the car graph. As you can see this graph is a sub optimal solution compared to the local method. Compared to the RL method, we see how much better OC was for our problem. We see through testing of the RL method that there are some draw backs to forgetting the model and just being purely driven by data.

Our Experiences

Week 1

In week one we decided to make a game plan for the following weeks. We would work on the local method for just a week since it was basically finished. For the other two methods we would spend two weeks on each method. Lastly, we would save the last week to wrap up all three methods and anything else that is left over. During the first week we wanted to look at the local method and explore it some.

Week 2 & 3

We decided to spend two weeks to look at reinforcement learning, during these two week we were able to produce a PowerPoint in beamer for our mid-week presentation. In week two and three we looked at our first global method. Dr.Ruthotto handed us some pre written code to play around with. A thing that should be noted is that this prewritten code only worked maybe 50% of the time. Looking deeper into the code we realized that we would have to mess around with the code to get it to match our problem. After we made these couple of changes in our code, we saw how fragile reinforcement learning is, instead of working 50% of the time our code barely worked at all. During this week we were also able to produce a rendering of both of our local and RL methods, adding a nice touch to our presentation that we gave.

Week 4 & 5

During week 4 and 5 we looked at optimal control method. During week 4 we were able to produce a rough draft of the website while also taking a deeper look into optimal control method. Week five we created a rough and final draft of our poster. While working on our poster we were able to produce a graph for OC so we can compare it to our other methods.

Week 6

We wrapped up any unfinished work including our paper, OC method and this website. After struggling with code for method three we finally found that our OC method had better results than RL. During week six we also gave a poster talk to the Emory staff and students. Our group ended up winning best poster and we each won some amazon gift cards. We will continue working on method three to try and get it to work for 150 steps, we also look to fine tune our code for method three and two.

More About Our Team

Reference

U. M. Ascher and C. Greif. “Chapter 9 (Optimization).” A First Course on Numerical Methods. SIAM. SIAM,2011

U. M. Ascher and C. Greif. “Chapter 14 (Numerical time integrators).” A First Course on Numerical Methods. SIAM. SIAM,2011

Undergraduate students and teacher develop segmentation methods for diagnosing Chiari Malformation

Tue, 14 Dec 2021 00:00:00 +0000

This post was written by Justin Smith, Elle Buser, Emma Hart, and Ben Hueneman and published with minor edits. The team was advised by Dr. Lars Ruthotto. In addition to this post, the team has also created slides for a midterm presentation, a poster blitz video, code, and a paper.

Collaboration Never Sleeps.

During our summer research at Emory University 2021 REU/RET program, our group focused on the algorithmic diagnosis of Chiari malformation from DENSE MRIs. We created an algorithm that can accurately and efficiently segment the cerebellum and brain stem from a magnitude image and use displacement data to classify whether or not a patient has the Chiari malformation. In doing so, we investigated two approaches; one that segments the given image by aligning and comparing the image to a known atlas and another that segments through deep learning.

Did Somebody Say Chiari Malformation?

Chiari malformation is a condition in which brain tissue extends into the spinal canal. While it can be difficult to diagnose Chari from anatomical images, a promising new direction for diagnosis is by looking at brain movement . Using an MRI technique called DENSE (shown below) that records how the brain moves, Dr. Oshinsky’s group (at Emory’s Dept. of Radiology) collected data about how Chiari patients have more brain movement in the cerebellum and brainstem than controls.

This method may be more accurate in diagnosing Chiari, however, the large number of manual processing steps may limit its use as a wide-spread screening tool. This project aims at exploring the use of machine learning algorithms to automize parts of the image processing pipeline, most critically the segmentation of the image into different brain regions. The teams worked with image data that has been collected and labeled by Dr. Oshinski’s group in a previous research study. The project is accessible to the team members since we can build upon recent progress and software made in image processing and computer vision and the image data is two-dimensional and of limited resolution, which enables fast experimentation. Despite this simplicity, the project allows us to investigate ML in a realistic setting and investigate the generalization properties and robustness of the approach.

Leave the SEGMENTATION to US!

We develop this project to solve the problem of identifying where the brain stem and cerebellum are in a given MRI. By finding or, in the language of the field, by segmenting the brain stem and cerebellum, we find the most relevant regions to look at brain movement. Using the DENSE MRI data, we can then average the movement over those regions to produce a biomarker that can help predict whether or not a patient has the Chiari Malformation. By producing these segmentations (examples above) automatically with the machine learning or atlas-based approaches, the diagnosis process could become much cheaper and more efficient.

Atlas Based Image Registration vs Machine Learning.

We first looked into atlas-based image registration as a way to produce automatic segmentations of the brain stem and cerebellum. Using the FAIR toolbox in MATLAB, the idea behind this method was to have a bank of MRI images with manually drawn segmentations that we could compare a new MR image to. Once we find a transformation (example below) between the known and new images, we can use the same transformation to produce a new segmentation from the know one.

We also looked into a machine learning approach. The goal here was to find a relationship between the DENSE images and their corresponding manual segmentations by training a model using convolutional neural networks (CNN). The network "learns" to identify images features, and, if successful, DENSE images can be used as inputs and the model will automatically segment the brain stem and cerebellum.

Our project implemented a CNN called U-Net, you can find out more about this network and the code we used here: U-Net: A PyTorch Implementation in 60 lines of Code

Overall, we found that the machine learning method produces better results, both in terms of segmentations of the brain stem and cerebellum, and in terms of how accurate the biomarkers calculated from those segmentations are. Results were very close to the manually produced target results, and we have ideas for further work that could make them even closer!

Additional information: to learn about atlas-based image registration and machine learning, check out these links! What is Image Registration? What is Machine Learning?

Time Management is Everything!

Here’s an outline of our process, as it evolved with time.

Week 1: During our first week, we created a working atlas-based image registration example, using FAIR: a MATLAB image registration toolset. We also began to look at the at a machine learning method, called U-Net, that we began setting up using PyTorch. We also set up GitHub, that we used throughout the project to collaborate on and publish codes.

Week 2: One of the first things we noticed when we began working with our data set the first week was the variability in the brightness and contrast of our images. In week two, we explored some different methods to help enhance the images. We decided to use a tool in MATLAB’s Image Processing Toolbox to normalize the images in a process called histogram normalization, which made our images much more consistent.

With this process complete, we began working on other atlas-based examples, and setting up the neural network with default parameters.

Week 3: We spent most of the third week preparing for our midterm presentation. It was helpful to practice presentation skills and get familiar using Beamer - a math presentation tool we weren’t yet familiar with that is the gold standard for mathematic presentations - and also to reflect on the progress we made, and talk through next steps with others after we presented. After our presentation, we started refining the machine learning method to use a new optimizer that automatically chooses the algorithm’s learning rate using a line search. This produced much better results!

Week 4: In the fourth week, we began working on our paper manuscript. Although there was still much of the research left to be done, coding in MATLAB and PyTorch, starting on this process of outlining our problem and explaining our methods helped us clarify next steps. At the end of the week, we turned back to our codes. On the atlas-based side, we explored ways to choose the images to compare to each other, because it can make a big difference in how good the outputted segmentation is. We also began devising a way to compare a new image we want to segment to more than one images we already have segmentations before, hoping that by averaging those results together, we could produce even better segmentations. On the machine learning side, we looked at ways to figure out the ideal number of iterations to use, so that we were neither overfitting nor underfitting our model to training data.

Week 5: Week five progressed very similarly to week four, as we continued working on our paper and developing better methods. We also began working on a poster to present our final results more broadly - and on this website!

Week 6: In the last week of the program, we had a lot left to do, finishing up our poster, website, and paper manuscript. We finalized every parameter in each method before finally looking at the testing set for the first time and analyzing our final results.

Poster Blitz Video

More About the Team

Elle Buser is a rising senior at the University of Wyoming majoring in mathematics and minoring in physics. At UWyo, she has worked as a tutor/supplemental instructor for Calc II and as a research assistant in the physics and astronomy department. Outside of school, she spends her time reading, doing arts & crafts, hiking, running, or just enjoying the great outdoors.
Emma Hart is a rising senior at Colgate University, double majoring in applied mathematics and educational studies. She enjoys combining these interests in math and education by volunteering as a math tutor and also by working for Colgate as a Writing Center peer consultant and computational mathematics grader. Outside of school, she enjoys crafting and spending time outside with friends and family.
Ben Huenemann is a rising junior at the University of Utah majoring in mathematics and computer science. He loves all things math, but is mainly interested in pure mathematics and plans to go on studying algebra or something similar in graduate school. Outside of school, he enjoys playing piano, evening bike rides, and board games with friends/family.
Justin Smith is an Early Childhood educator in Atlanta, Georgia, where he is in his 10th year of teaching. He is certified in Early Childhood Education and holds a Gifted Endorsement in the state of Georgia. He currently holds the Teacher of the Year award at his school. He is very active in his community and school, where started an organization called Obama Gentlemen with the primary duty to improve academics and address behavioral concerns of young men by instilling morals and values that will help transition young adolescent males into young adulthood. During his leisure time, he enjoys traveling, shopping, listening to music, and spending time with his family.

AI-Assisted Exploration of the Polygonal Faber-Krahn Inequality

Fri, 16 Jan 2026 00:00:00 +0000

Mentor: Dr. Levon Nurbekyan

Overview

This project focuses on the polygonal Faber-Krahn inequality, which conjectures that among all $n$-gons of fixed area, the regular $n$-gon minimizes the first Dirichlet eigenvalue of the Laplacian. The conjecture was introduced by Polya and Szego, who proved it for $n=3,4$ using Steiner symmetrization. For $n \ge 5$, however, these techniques break down, and the conjecture remains open.

The goal of the project is to use AI-assisted methods to explore this problem both computationally and conceptually, with the aim of either identifying potential counterexamples or producing empirically supported conjectures and strategies that could inform future theoretical work.

Computational Approach

For a broad range of values of $n$, we will numerically search for candidate optimal $n$-gons using a combination of gradient-based optimization and reinforcement learning, and subsequently leverage evolutionary coding agents in the spirit of AlphaEvolve to refine and extend these optimization strategies. The resulting large-scale experiments will either provide numerical evidence supporting the conjecture or identify configurations that warrant closer scrutiny.

Analysis

In the absence of apparent counterexamples, we will use models with advanced reasoning capabilities, in the spirit of DeepThink, to analyze the numerical results at a higher level, seeking recurring patterns or structural features in the evolution of optimizing shapes that may suggest new geometric or analytical approaches to the problem.

Prerequisites

A solid background in linear algebra, multivariable calculus, and basic numerical methods.

New algorithm helps make portable CT machine images clearer

Tue, 14 Dec 2021 00:00:00 +0000

This blog post was written by Mai Phuong Pham Huynh, Manuel Santana, and Ana Castillo and published with minor edits. The team was advised by Dr. James Nagy. In addition to this post, the team has also given a midterm presentation , created a poster, made a poster blitz video, published code, written a paper, and developed a lesson plan.

Research done at the 2021 REU/RET summer program at Emory University has devised a way to make medical diagnoses more effectively. The work behind this new algorithm relies heavily upon mathematical techniques that connect numerical linear algebra with optimization methods.

Some of the most important aspects of this work link branches of mathematics like calculus, statistics, algebra and geometry. The different contributions by mathematicians to these branches of mathematics were complementary and accelerated research in the area of image reconstruction. The results obtained from their work have offered other student mathematicians a different way to study these problems and shown that this is an effective way for radiology doctors to accurately diagnose illnesses and keep saving patients’ lives. The goal of the Point-of-Care Tomographic Imaging research group is to develop numerical methods that would estimate the geometry parameters of a portable CT scanning device and to reconstruct the image. CT images in portable devices are a lot clearer. This work has been one of the most interesting pieces of work ever accomplished.

COVID-19 and Imaging

The mathematical ideas involved in the Point-of-Care Tomographic Imaging project are of great importance. With these mathematical methods, imaging becomes a possibility with patients in all parts of the world, allowing them to live a healthy and happy life. To get better reconstructed images, the group considers a regularized linear least squares problem, to which the solution is approximated by using an alternating descent scheme known as block coordinate descent or (BCD). Each iteration of BCD has two steps. One BCD step solves a linear least square problem, and the next step solves the nonlinear problem.

"Coronavirus" by Yu. Samoilov is licensed under CC BY 2.0

By the 1900’s mathematicians had already found ways to use these iterative reconstruction techniques to reconstitute an image. In 1917, Johann Radon introduced the Radon Transform, followed by Stefan Kaczmarz in 1937. Scientists Allan McLeod Cormack and Godfrey Newbold Hounsfield developed the first scanning device. These innovations each contributed to modern medical scanning technology.

The medical field has greatly depended on these iterative reconstruction methods. In medical imaging, computed tomography (CT) techniques are becoming more and more popular for their ability to produce high quality images of the human body. CT methods use a combination of computer processes and mathematics to reconstruct images. The COVID-19 pandemic brought many challenges and setbacks to many doctors around the world and imaging played a vital role in helping diagnose the virus. Doctors have been able to use CT scans to diagnose and treat COVID-19 by examining the lungs of patients who are potentially infectious, or in recovery.

Point-of-Care Tomographic Imaging

Since the first CT scanner was developed, CT scanning technology has evolved significantly. Now, there are portable CT scanners that can be transported anywhere without trouble. The idea of transporting huge heavy CT scanners to remote locations was always a difficult task for many doctors. Point-of-care tomographic imaging has allowed radiologists to add portable CT scanners to their departments to increase patient satisfaction and improve medical outcomes. Portable CT scanners can be used to do scans of a patient without moving the patient out of bed.

A CT scanner is a device that is composed of a scanning gantry, x-ray generator, computer system, console panel and a physician’s viewing console. The scanning gantry is the part that will produce and detect x-rays. In a typical CT scan, a patient lays on a bed that moves through the gantry. An X-ray source rotates around the patient and shoots X-ray beams through the human body at different angles. These X-ray measurements are then processed on a computer using mathematical algorithms to create tomographic (cross-sectional) images of the tissues inside the body.

Limitations arise when using these portable CT scanners for medical procedures since these devices require extensive care, such as regular calibration for effective performance. This is where the point-of-care tomographic imaging problem begins.

The Beginning of the Problem

Two parameters that relate to the geometry of portable CT scanning devices are:

R - distance between source and detector and
θ - orientation of source to detector.

During the CT acquisition process, the x-ray source rotates around the patient. Each time the x-ray source moves to a new position, these parameters change. In point-of-care imaging, these parameters are essential during the data imaging process. Imprecise data (unknown parameters) result in the reconstruction of poor quality images, while precise data (known parameters) yield a well reconstructed image.

How were these geometry parameters estimated to obtain better reconstructed images?

The group considered solving a regularized linear least squares problem. The solution to this problem is approximated by using an alternating descent scheme known as block coordinate descent or (BCD). They used this alternating algorithm in which each iteration alternately solves a linear last squares problem (for the image) and a nonlinear least squares problem (for the geometry parameters). An alternate form of this algorithm was also studied as a fixed point iteration, with acceleration techniques used to improve convergence. Implementation and numerical examples were done using MATLAB to obtain results. To understand the background behind this problem, the connection between Beer’s Law and linear algebra needs to be studied.

Beer’s Law & Linear Algebra

The Beer-Lambert Law or Beer’s Law was first developed in 1729 by Pierre Bouguer, and in 1852, Beer added to the law. Beer’s law states that as an x-ray source emits an x-ray beam and it passes through an object, the beam loses energy exponentially. To create a 2-D image, an imaginary grid of pixels is placed atop the object, and an x-ray beam is then shot through the object. Beer’s Law is used to create a system of linear equations.

From Algebra, it is known that a system of equations has three types of solutions, infinitely many, exactly one, or no solution. When there is no solution, the system is called inconsistent, as opposed to consistent when there exists a solution. When there are more equations (x-rays) than unknowns (pixels), then the system is overdetermined. This is what happens in computed tomography. When there are more unknowns (pixels) than equations (x-rays), then the system is underdetermined. In each equation, every unknown represents a physical property, called attenuation, for each pixel value. Pixels that have the same amount of material have the same attenuation. An image is then created from the pixels. The images obtained based on the attenuation values allow doctors to make more accurate diagnoses. For example, bones have a different attenuation than lungs.

A system of linear equations can be solved in Precalculus by the Gauss-Jordan elimination method. In linear algebra, a matrix equation of the form Ax=b is solved to find x. Ideally, this can easily be done. Unfortunately, for this project, this might not be possible for several reasons: 1) b has noise in it, and 2) the exact matrix A may not be known because the parameters R and θ may be perturbed (for example, the CT machine may not be calibrated correctly). Matrix A may also be a huge matrix containing millions of rows and columns. Typical approaches learned in Pre-Calculus or a linear algebra class cannot be used to solve this problem. The tomographic imaging research group uses other methods to solve these large problems, referred to as ill-posed.

An ill-posed problem is often referred to as one that is not well-posed. In the 20th Century, French mathematician Jacques Hadamard defined a well-posed problem as one having three properties: 1) a solution exists, 2) the solution is unique, 3) the solution depends continously on initial values. An ill-posed problem needs to be approached differently, e.g. -regularization. Tikhonov regularization is a common form of regularization for ill-posed problems.

The Linear & NonLinear Least Squares Problem

Throughout the 1800s, mathematicians such as Legendre, Gauss, and Laplace contributed ideas to the method of least squares. For example, the method of least squares can be used to find the line that minimizes the sum of squared distances between teh line and the given data points. In computed tomography, a problem arises that involves least squares, a regularized linear least squares problem. This regularized linear least squares problem is composed of x, the image, A(p), a matrix that is constructed by the geometry parameters, R and θ, b, the measured data, and a parameter α, called the regularization parameter. The nonlinear least squares problem does not include the regularization parameter.

The solution to this problem is found by the block coordinate descent (BCD). The BCD algorithm is an optimization algorithm that solves this problem iteratively. BCD works by minimizing the parameters R and θ and x one at a time. The linear least squares problem is considered first. It takes the initial parameters R and θ, generate matrix A, to get x, the image. Here the parameters, R and θ are known and x is approximated. Then, the nonlinear least squares problem is considered. Once x is known, then x is used to approximate the parameters, R and θ. In this latter phase, x is known and the parameters R and θ are approximated. The pattern continues until a good image is produced.

Filter - Based Regularization

Since it is impossible to get data exactly from the detector due to numerous reasons, regularization is needed to lower the error the solution caused by noise in the data. Tikhonov regularization is one of the most common ways to handle these noise levels. The method is named after mathematician Andrey Tikhonov. Tikhonov worked in numerous topics and different fields of mathematics. His best contributions are in topology.

Singular value decomposition (SVD) plays an important role in understanding the basic idea behind Tikhonov Regularization. The singular values of the matrix A are significant. To achieve good reconstructed images with small error, it is necessary to have the balance of large and small singular values; the large singular values have good information about the solution, but the small singular values cause errors to blow up.

MATLAB Toolboxes & Implementation

MATLAB was used in the project, including the optimization, signal processing, and image processing toolboxes. The IR Tools, AIR Tools II, and Imfil packages were also used.

To simulate a problem, the IR tools package is used. IR tools also provide regularized linear least square solvers. Three of these linear least squares solvers are: 1) hybrid-LSQR algorithm, 2) IRN, 3) FISTA. To approximate the solution of the non-linear least squares problem two methods are used: 1)lsqnonlin, 2) imfil.

The implementation of BCD used the IR Tools package and naming conventions. In the base IRtools package, the function PRset is used to set up options for computed tomography problems, where PRset is updated to accept values related to the BCD (inital guess for parameters). To simulate a CT problem with unknown geometry parameters PRtomo_var is used with the image size being n×n. PRtomo_var generates all the data necessary to simulate the inverse problem. IRset updates to include parameters for BCD, such as which acceleration technique to use, and IRbcd implements the BCD algorithm.

The steps above permit the simulation and solving of a CT reconstruction experiment. With n=64 and perturbations added to the parameters R and θ, the test problem is generated. The image used is the Shepp-Logan image.

After solving the problem with the BCD algorithm, it can be seen that the solution with the BCD resembles that of the true parameters solution.

Acceleration Techniques: Tests & Results

Other mathematicians continue their work in these fields. Some of them introduced ideas on acceleration techniques to speed up the convergence rate of fixed point iteration problems. One of them is Donald G. Anderson, famous for his work on the Anderson Acceleration, also called Anderson mixing. In the tomographic imaging project, estimation of the geometry parameters can be viewed as a fixed point iteration technique. The research group used three fixed point acceleration schemes in their numerical experiments: 1) Irons-Tuck method, 2) Crossed - second method, 3) Anderson Acceleration. All these acceleration tests were performed and seemed to improve image resolution.

From tests performed, the acceleration techniques provided slightly better convergence, with the crossed secant method and Anderson Acceleration performing the best. The Irons-Tuck method converged much better in the angle parameters, but took much longer. The Irons-Tuck image seems to have the least background noise.

Poster Blitz Video

More About the Team

Mai Phuong Pham Huynh

Mai is a rising junior at Emory University, majoring in Applied Mathematics and Statistics. Her research focus is on numerical analysis and scientific computing. In her free time, she enjoys listening to violin concertos, doing giant Jigsaw puzzles, and searching for vintage vinyl records from the 50s-90s.

Manuel Santana

Manuel is a rising senior at Utah State University. His research interests are in all areas of computational and applied math. When not doing math he enjoys European handball, pickleball, raquetball, and cooking with his wife Emily.

Ana Castillo

Ana Castillo is in her 10th year teaching. She is certified to teach Spanish and Mathematics grades 6-12 in the states of Texas, Tennessee and Illinois. She is working for Proximity Learning, a company that has allowed her to teach virtually in different school districts in the United States. She has been a participant of the Park City Mathematics Institute Teacher Leadership Program (PCMI TLP) and part of a team that runs a Math Teachers’ Circle in the Rio Grande Valley Area. In her spare time, she enjoys taking long walks, listening to classical music (Vivaldi), spending time with her pets (dogs & donkeys), cooking, reading, writing and traveling.

Efficient Determinant Estimators for Potential Flow Generators

Tue, 14 Dec 2021 00:00:00 +0000

This post was written by Jonathan Valyou, Edward Shao, Lauren Proctor, and Stephen Stern. The project was advised by Dr. Yuanzhe Xi.

Introduction

Imagine that somehow, you’ve recently come to own a massive collection of paintings by famous painters, but none of these works are known to the world. You also happen to be a starving, yet incredibly talented artist, so you wonder about passing off some of your own paintings as part of the collection. You realize that you don’t quite have the skills, but you have a friend, a docent at the local museum, that’s willing to help you learn.

After some weeks of thinking the two of you come up with the following plan: each week you present your friend with a randomly chosen painting, either one that you’ve painted or one from the real collection. Then they do their best to authenticate it, telling you if they think it is real or a fake, and then you let them know whether or not they were correct.

Over the course of many weeks you both become better at your jobs because you both get feedback from each other. Each week you try something new, maybe a new technique, brushstroke, or type of paint. So, over time, your skills get better because you learn which combinations work well to trick your friend into thinking your painting is real. On the other hand, each time your friend makes a guess, they learn if they were right or not and learn from their mistakes and successes!

"Delaware Art Museum" by -Jeffrey- is licensed under CC BY-ND 2.0

Now, at this point you may be wondering why you’re reading some hypothetical story about a painter when this is supposed to be about mathematical research, so we’ll start getting to the point. The process above is actually the structure of a machine learning technique called Generative Adversarial Network, or GAN for short. In the story, you served the role as a “generator” by generating new paintings, and your friend serves the role as a “discriminator” discriminating between the real and fake. In practice GANs work just like the painting scenario, just over the course of what would be equivalent to thousands or millions of weeks. GANs turn out to be incredibly powerful tools which can be used in various ways like deep fakes that you may have seen in the news, or even theorizing new pharmaceutical drugs.

Background

Our work does not involve GANs, but it does lie within the the broader field of generative modeling; more specifically our work is in the category of Normalizing Flows. These are also generative models that do not require a discriminator like GANs, but do have some other interesting properties. One essential property is that they are invertible, which has a specific mathematical definition but can more easily be reframed back into our painting example. In this context, invertibility means that there is an exact correspondence between a painting you created and the choices that you made in painting it (e.g. the technique, brushstroke, type of paint, and millions of other small decisions that you made.) This correspondence means that just by looking at your painting I can write down a list of all of the decisions you made. Then if I give you that list I wrote down as a set of instructions you would create the exact same painting!

Paintings and List of Instructions

It turns out that this setup allows us a way to replace your friend’s method of determining if a painting is real or fake. In this context fake means something slightly different: now a fake painting means one that doesn’t belong in your collection. If we’re given a painting, it’s pretty difficult to know whether or not it belongs to your collection by just looking at it, but we can translate that painting into the list of choices that you would have made to paint it. Since you know your style choices pretty well, you can look at that list and tell us how likely it was that you’d make that combination of choices. If there’s a low probability that you would have made that combination, then the painting probably doesn’t belong! This new setup describes how Normalizing Flows work.

Our work extends a paper by Huang et al. by applying more computationally efficient algorithms for the learning and invertible transformation process they developed. If you are not familiar with neural networks or could use a brush up, we’d suggest checking out some or all of these videos by 3 blue 1 brown[link here]. They are a great resource for learning about and getting excited about mathematics and we highly recommend them as a resource. If you are comfortable with neural networks this would be great point to skim the paper that is central to our work, with specific emphasis on section 3.

Problem Formulation

Normalizing flows require an invertible function, like the correspondence between a painting and its list of choices. In general, constructing such a function is not a straightforward process. Huang et al. make use of a property of a special type of functions called convex functions. It turns out that a the gradient of a convex function is invertible, so they propose training an Input Convex Neural Network, which is a convex function, and using its gradient as our normalizing flow.

In order to transform probabilities from the latent space (our list of choices) and the learning space (paintings) we need to find the determinant of the Jacobian of our normalizing flow. If you are familiar with multivariate calculus, this comes from the change of variable formula. If you are familiar with single variable calculus, it is a higher dimensional analogue of changing dx to du when performing u-substitution. If you were keeping track, this is the Hessian of our convex neural network. In other words, we need the determinant of a matrix that unfortunately takes a lot of computer memory to store, and is impossibly expensive to compute.

To estimate the determinant without forming the matrix, Huang et al. utilize a tool called a Hutchison Trace Estimator and Stochastic Lanzcos Quadrature (SLQ). This works by deducing (spectral) information about the matrix by randomly sampling a bunch of vectors and measuring how much the matrix changes them through multiplication. This multiplication, called the Hessian vector product, can be performed without explicitly forming the matrix, but is still relatively expensive to compute. This process is required in the forward propagation through the network.

Another essential part of training neural networks is the loss function. In the first painting story about GANs, the loss function for you was whether or not you fooled your friend, and you learned each week by trying to minimize this function. Your friend learned each week by trying to maximize this function. (This is where the “adversarial” in the name GAN comes in, you have opposite goals!) The loss function for our work is a bit more complicated, but ultimately is a comparison of true probabilities, and the ones we estimated with the forward transform. This means that classically training our normalizing flow would require backpropogating with the gradient of the forward transform process. Unfortunately this turns out to be highly unstable. Instead, Huang et al. uses a surrogate loss function that combines some algebraic manipulation and another Hutchison Trace Estimator in tandem with an algorithm called the Conjugate Gradient method (CG). This process results in an estimate for the gradient of the loss function required for back propagation. Just like SLQ, CG requires computing a hessian vector product each iteration that it takes to converge on a solution.

Our Contribution

By using information we can deduce from the Stochastic Lanzcos Quadrature, we reduced the number of iterations that CG requires to converge. Specifically, CG approximately solves a linear system of equations, Ax = b. The number of iterations it takes for the algorithm to converge is related to the condition number of A, the smaller this number is the faster it converges. We apply a pre-conditioner, M, that instead solves the system MAx = Mb, which is designed to reduce the condition number of MA. Behind the scenes, we can find such a preconditioned because SLQ results in providing the smallest and largest eigenvalues and corresponding eigenvectors of A and the condition number of A is given by the ratio of the largest and smallest eigenvalues.

More About the Team

Jonathan Valyou

Jonathan Valyou is a senior at Emory University majoring in Applied Mathematics with a minor in Physics. His primary research areas of interest include scientific computing, data science, and optimization. Aside from this project, he has worked on neural network research. He is currently writing an honors thesis as part of the Emory Mathematics Honors Program. When not tinkering with computational models, he enjoys traveling, listening to classical and pop music, playing the euphonium, and trying new foods.

Lauren Proctor

Lauren Proctor is a junior at the University of Tennessee Majoring in Mathematics and Computer Science. She is a part of the Mathematics Honors program at her university, and has previously worked on natural language processing research. Outside of school, she enjoys hiking, baking, and spending time with friends.

Data assimilation for Glacier Modeling

Mon, 27 Jun 2022 11:36:49 -0400

This post was written by Emily Corcoran, Hannah Park-Kaufmann, and Logan Knudsen. The project was advised by Dr. Talea Mayo. Our team has also created a midterm presentation, blitz video, poster, paper, and has published their code.

Glaciers

Research has shown that climate change will likely impact storm surge inundation and make modeling this process more difficult. Sea-level rise caused by climate change plays a part in this impact. To better model sea-level rise, glaciers can be modeled. Marine-Terminating Glaciers have a natural flow towards the ocean, which contributes to sea level rise. By the year 2300, the Antarctic ice sheet is projected to cause up to 3 meters of sea level rise globally. Due to the severe impacts of glacial melting, modeling changes in ice sheets is an important task. There are challenges to modeling sea level rise, as ice sheet instability leads to significant sea-level rise uncertainty.

Image by W. Bulach, used under Creative Commons Attribution-Share Alike 4.0 International License

Modeling Glaciers

Our group is collaborating with Dr. Robel, a glaciologist, climate scientist, and applied mathematician from Georgia Tech, and working with the glacier model described in his 2018 paper. This ice sheet model aims to describe the changes in ice mass of marine-terminating glaciers, which may be impacted over time by climate change.

Image used with permission from Dr. Alexander Robel

A glacier can be represented with a simplified box model that has a length $L$, precipitation $P$, and height and flux at the grounding line $h_g$ and $Q_g$. This model is the best approximation for one variable and describes the dominant mode of the glacial system.

The two-stage model that our group is using incorporates a nested box into the system. This new box has a thickness, $H$, and an interior flux, $Q$. The change in length and height of the glacier can be described with these differential equations: $$\ \dfrac{dH}{dt}=P-\dfrac{Q_g}{L}-\dfrac{H}{h_gL}(Q-Q_g)$$ $$\ \dfrac{dL}{dt}=\dfrac{1}{h_g}(Q-Q_g)$$

Sensitivity Analysis

Sensitivity analyses study how various sources of uncertainty in a mathematical model contribute to the model’s overall uncertainty. This allows us to understand the model better.

Why do we do this? Why do we care about the uncertainty of a model? And where do the uncertainties even come from? In this ice sheet model, just like in any model, there are always going to be simplifications, and these lead to uncertainties. We need to have a good idea of which uncertainties matter the most, so that we better know the limits of where our model does a good job of simulating the real world.

The basic idea is this: We check sensitivity by using different distributions for the input parameters. If the outputs vary significantly, then the output is sensitive to the specification of the input distributions. Hence these should be defined with particular care. We can also look at the sensitivity of the model parameters to inform which parameter we’re going to work with in the data assimilation. We want to be working with the most sensitive parameter, because it has the most promise for things we vary later on to matter, in questions like: “if your data is from billions of years ago does that matter? Is it important to have your data from the last 60 years?” or “how much will noise impact the predictions?”

The uncertain model parameters we considered are: initial conditions, sill parameters, and SMB values. For consistency’s sake, we vary each parameter by +-10 percent of the nominal values originally given in our model code. Below you can see three graphs, one for each group of parameters varied, for each “time vs H(t)” (Height of the glacier at time) and “time vs L(t)” (Length of the glacier at time).

Looking at the distributions, we see that varying initial conditions (Leftmost) seems to produce the greatest spread, but the slopes of the lines there are all very similar. Varying the sill parameters (Middle) produces a lesser spread than varying initial conditions, however there is a greater variation in the slope of the lines. Finally, when varying the smb data (Rightmost) the result actually doesn’t change that much and is quite stable. Thus, according to our analysis, the model is the least sensitive to SMB parameters, and between initial and sill parameters judgement varies depending on what you care about more - spread or slope.

Data Assimilation

Data assimilation is a method to move models closer to reality using real world observations by readjusting the model state at specified times.

Image used with permission from Dr. Talea Mayo.

In this example we have used the ensemble Kalman filter method (ENKF) in order to perform our data assimilation. In basic terms, we initialize an ensemble( or a series of model runs with perturbed initial conditions) and performed data assimilation on each of the ensemble members, then to get our final analysis we took the mean of the ensemble.

The program used to model the glacier behavior and assimilate the data begins with choosing a set of initial conditions. Once the initial conditions are input to the model, which after taking a step using a Runge-Kutta 4th Order Method, is plugged into a Data Assimilation Method. Our main method is ENKF as previously mdentioned. Finally, the analyzed data from the assimilation is output and plugged back into the model. It should be noted that at sometimes the forecast output for the model is the same as the analyzed data.

Square Difference

The error measure we use in the best ensemble size and observation scheme is the square difference, $d^2$, which we define $$\ d_t^2 = \left(x_t - x^a_t\right)^2 $$ where $x_t$ is the true state from the truth simulation at time $t$ and $x^a_t$ is the analysis state at time $t$.

Ensemble Size

In the interest of lowering computational costs, we use the square difference in order to minimize ensemble size while also minimizing error. To do this, we choose an ensemble size, calculate the square difference at each $t$, and then calculated the mean of all these square differences. We ran this calculation for ensembles sizes from 2 up to 75, and found that ensembles of size 7-10 were ideal as they were at the point where the average square difference hovers around the same value.

Observation Scheme

We ran the model for various observation schemes to find the best observation scheme, i.e. the times frames and frequencies which can produce a sufficiently small average square difference over the course of the model run. We applied this process to our model and found that for before 1900 the best observation frequency, while still using small number of observations, would be every 19 years for a total of 100 observations. Similarly, for the time frame of 1950-2300 we found that yearly observations for a total of 350 observations is the best frequency.

Model Runs

Using the facts we established in the previous two sections, we ran the model using EnKF for the time frame of 0-2022 in order to project $H$ and $L$ into the future up to the year 2300. The following plots show the results of this experiment, which we will use to help calculate $Q$ and $Q_g$ over time, and in turn use it to calculate sea level rise.

Sea Level Rise

Using the formulas for $Q$ and $Q_g$ we can calculate the volume lost across the grounding line $$\ V_{gz} = W(Q-Q_g)t $$ We then used the to calculate the volume out at all times and add it up to get accumulated volume loss. We then assume that the width of the glacier is 50 km(at least in the case we show here). To convert this to sea level rise, note that 394.67 km$^3$ of ice is equivalent $1$ mm of sea level and get the following projection of sea level rise.

Next Steps

Using data assimilation can help to inform the glacier modelers and glaciologists who collect data about how to collect data in an efficient way. This can help researchers to more efficiently utilize funding and avoid unnecessarily expensive data collection that does not significantly improve glacier models. Data assimilation should be explored within more complicated glacier models, as the model used here is quite simplified. If more research is performed on this technique, it could greatly improve the practice of glacier modeling. Data assimilation can also be used for many geophysical modeling tasks, such as weather forcasting and hurricane storm surge modeling. Going forward, we plan to integrate the output of the glacier model into the ADCIRC hurricane storm surge model to predict the impact of glacier model on storm surge inundation.

References

The long future of Antarctic melting
Marine ice sheet instability amplifies and skews uncertainty in projections of future sea-level rise
Projected climate change impacts on hurricane storm surge inundation in the coastal United States
Response of marine-terminating glaciers to forcing: time scales, sensitivities, instabilities, and stochastic dynamics

About the Team

Emily Corcoran

Emily Corcoran is a junior at New Jersey Institute of Technology, majoring in Mathematical Sciences with a concentration in Applied Statistics and Data Analysis. Before this REU, she has worked as a research assistant in her school’s Visual Perception Lab. She is a student in the Albert Dorman Honors College and is an active member of NJIT’s school yearbook and Knit ’n Crochet club. When she is not in class, she can be found reading, listening to music, or attending a local play.

Logan Knudsen

Logan Knudsen is a senior at Texas A&M University majoring in Mathematics with minors in Oceanography and Meteorology. Before this REU, he has worked doing research on Data Analysis using Benford’s Law and as a Teaching Assistant. Logan is currently the President of Texas A&M’s Math Club and a member of student radio, KANM. When not in class, he can be found reading, playing the guitar or playing video games with his friends.

Hannah Park-Kaufmann

Hannah Park-Kaufmann is a junior at Bard College and Conservatory, majoring in Mathematics and Piano Performance. Before this REU, she conducted research on Numerical Semigroups and Polyhedra, and on Identifying Universal Traits in Healthy Pianistic Posture using Depth Data. She tutors math in the Bard Prison Initiative (BPI). When not doing math, she can be found playing piano, reading scores, reading literature and/or eating.

Fast Training of Implicit Networks with Applications in Inverse Problems

Mon, 27 Jun 2022 11:36:49 -0400

This post was written by Linghai Liu, Shuaicheng Tong, and Lisa Zhao and published with minor edits. The team was advised by Dr. Samy Wu Fung. In addition to this post, the team has also given a midterm presentation, filmed a poster blitz video, created a poster, published code, and written a paper.

What are Inverse Problems?

Inverse problems consist of recovering a signal $x^\ast$ (e.g. an image, a parameter of a PDE, etc.) from indirect, noisy measurements $d$. These problems arise in many applications such as medical imaging, computer vision, geophysical imaging, etc.

This measurement process is usually modeled as an operator $\mathcal{A}$, satisfying the following equation: $$ d = \mathcal{A} x^\ast + \boldsymbol{\varepsilon}, $$ where $\mathcal{A}$ is a mapping from signal space $\mathbb{R}^n$ of original images to measurement space $\mathbb{R}^m$. Since our project deals with image deblurring, we have the following variables:

$d \in \mathbb{R}^{n}$: blurred image with noise
$x^\ast \in \mathbb{R}^{n}$: original image
$\boldsymbol{\varepsilon} \in \mathbb{R}^{m}$: random unknown noise

Solving Inverse Problems from a Classical Approach

Using direct inverse we have: $$ d = \mathcal{A} x^\ast + \boldsymbol{\varepsilon} \Longrightarrow x^\ast = \mathcal{A}^{-1} d - \mathcal{A}^{-1} \boldsymbol{\varepsilon} $$ However, since $\boldsymbol{\varepsilon}$ is unknown, directly inverting may end up amplifying this noise factor⁠. Because of this noise corruption, the reconstructed image ends up being unrecognizable.

To better visulaize this, we have the following set of pictures:

Original Image

Blurred Noisy Image

Direct Inverse

In order to minimize the noise factor, we want to formulate a regularized optimization problem.

We essentially want to find the minimium distance between the reconstructed image and the observed blurred image, plus a regularizer $R(x)$.

This regularizer is chosen based on prior knowledge of the data; this can often lead to inaccuracies—meaning the reconstructed image will be a bit blurry.

For example, using a gradient descent scheme where we handpick a regularizer to help stabilize the reconstruction, we have the following set of pictures:

Original Image

Blurred Noisy Image

Gradient Descent

We see that the reconstructed image using gradient descent is a huge improvement from direct inverse; however, there are still blurry areas we can improve on. In search of a better method, we turn towards implicit learning.

Implicit Deep Learning

The issue with the classical approach is that the regularizer is chosen heristically. To combat this, our approach now is to utilize data to learn and train the regularizer.

To do this we mimic gradient descent, but replace the gradient of the regularizer, $\lambda \nabla_x R$, with a trainable network.

However, this creates some problems concerning memory cost and the number of layers, $K$, in our neural network. The memory grows linearly as $K$—chosen heuristically—increases.

With implicit deep learning we send $K \to \infty$ until we find a fixed point of a single layer $T_\Theta(\cdot)$.

Implicit Backpropagation

Suppose now we have found a fixed point $x^\ast$ for a single layer. Then, $$ x^\ast = T_\Theta (x^\ast) $$

Using implicit differentiation on the equation above we have,

$$\frac{d x^\ast}{d \Theta} = \left( I - \frac{d T_\Theta (x^\ast)}{d x^\ast}\right)^{-1} \frac{\partial T_\Theta (x^\ast)}{\partial \Theta}$$

However, solving this is very expensive because of the inverse term.

To circumvent this issue, we use a recently proposed method called Jacobian-Free Backpropagation.

Jacobian-Free Backpropagation (JFB)

The goal of JFB is to alleviate memory requirement and avoid high computational cost in implicit networks.

The key idea is to replace the problematic Jacobian $$\left( I - \frac{d T_\Theta (x^\ast)}{d x^\ast}\right)$$ with the identity matrix $I$.

For a comparison, if we were to calculate the true gradient using implicit networks we have the following equation:

$$\nabla_\Theta \ell = \frac{d \ell}{d x^\ast} \left( I - \frac{d T_\Theta (x^\ast)}{d x^\ast}\right) ^{-1} \frac{\partial T_\Theta (x^\ast)}{\partial \Theta}$$

Using JFB to approximate the gradient we only need to solve: $$p_\Theta = \frac{d \ell}{d x^\ast} \frac{\partial T_\Theta (x^\ast)}{\partial \Theta}$$ which is a descent direction for the loss $\ell$.

Utilizing JFB, we avoid computing the Jacobian term. As a result, implicit networks are trained faster and more easily implemented.

Note: the JFB approach relies on a set of conditions to be true:

$T_\Theta$ is contraction mapping with Lipschitz constant $\gamma$
$T_\Theta$ is continuously differentiable w.r.t. $\Theta$
$M := \frac{\partial T_\Theta}{\partial \Theta}$ has full column rank
$M$ is well-conditioned, i.e., $\kappa (M^T M) < \frac{1}{\gamma}$

Numerical Experiments

In our project we used the CelebA dataset, which consist of annotated celebrity faces. The images are categorized into various sections based on specific features that the celebrities have. For example, whether or not they have bangs, wear glasses, have a pointy nose, etc.

Results

The results are as follows

From the graph we see that the loss is decreasing as the number of epochs increases. An epoch is one complete pass of the entire dataset through our algorithm.

Note: Two metrics are commonly used for assessing the quality of reconstructed images: the peak-signal-to-noise ratio (PSNR, a positive number, best at $+\infty$) and the structural similarity index measure (SSIM, also positive, best at $1$).

Acknowledgements

We sincerely thank the guidance of our mentor, Dr. Samy Wu Fung, and other mentors at Emory University for the opportunity.

More About the Team

Linghai Liu is a rising senior at Brown University, double concentrating in applied mathematics - computer science and mathematics. His main interests lie at the intersection of statistical theory, machine learning, and optimization. Outside of work, he enjoys reading novels and watching animes.

Shuaicheng Tong is a rising junior at the University of California, Los Angeles, majoring in applied mathematics and minoring in statistics. He is interested in optimization and machine learning. He volunteers at the UCLA Statistics Club where he tutors mathematics and statistics. Outside of school, he enjoys working out, hiking, and watching Star Wars shows.

Lisa Zhao is a rising sophomore at the University of California, Berkeley, double majoring in statistics and economics. She is interested in learning about how statistics is used as a powerful tool in finance. Outside of work, she enjoys swimming, drawing, and watching TV shows.

Learning Ordinary Differential Equations from Data

Mon, 27 Jun 2022 11:36:49 -0400

This post was written by Emma Hayes, Mathias Heider, and Carrie Vanty and published with minor edits. The team was advised by Dr. Deepanshu Verma.

In addition to this post, the team has also created slides for a midterm presentation, a poster blitz video, and a poster.

Project Overview:

Imagine a spring mass system (Figure 1). What if you wanted to find the location of the mass at any given time point? In order to find this information, you must first understand the dynamics of the system. A spring mass system is an example of simple harmonic motion where total energy is conserved. This means that you can model the dynamics using a Hamiltonian Ordinary Differential Equation, which has the quality of energy conservation. To solve our problem, we use neural networks utilizing Hamiltonians in the forward propagation to predict our coordinates.

Figure 1 - Spring Mass by Oleg Alexandrov (public domain)

Our project aims to compute the value of the Hamiltonian for any given time and set of initial conditions using Hamiltonian Inspired neural networks. We will first introduce the mathematical background of our project and the novel technique we implemented for our forward propagation. Results will then be presented and analyzed. Lastly, we will discuss how our project expands upon both Ruthotto 3 and Greydanus 6 papers.

Background:

Often, neural networks are thought of as a black box, where the actual inner-workings are not the main focus. However, since we are mathematicians, we want to understand how the network functions in order to best optimize it. For this reason, we began by looking at why and how ODEs were first used in neural networks. Ordinary differential equations were first used in Residual Neural Networks due to the similarity between the forward propagation equation and discretization of an ordinary differential equation. The only difference is multiplication of the step size, which we denote as $\mathbf{h}$. $$Y_{j+1} = Y_j + \mathbf{h}\sigma(Y_j K_j + b_j)$$ In the context of our residual neural network, the ODE as forward propagation means that for each layer of the network, we will move one time step forward in the discretization of our network ODE. The weights and biases, $K$ and $b$, may change in between the layers depending on the given values in the network ODE. The output of our network is the Hamiltonian value at the given time, and from that we are able to approximate position and velocity values for the mass. When estimating coordinates of a Hamiltonian system, or the value of the Hamiltonian itself, the Hamiltonian relationships are important for forward propagation. Hamiltonians intrinsically conserve energy, meaning the network is better able to learn conservation laws and predict examples with energy conservation. Without considering these relationships, it would be much more difficult for the network to learn conservation, which can cause a buildup of error. In many studies (3, 6), they have found that by using the Hamiltonian equations in Hamiltonian data sets, error has decreased. We plan to investigate this further and find which discretization methods and algorithms will perform the best.

Learning Hamiltonians from Data

To learn about Hamiltonian dynamics from data, we use neural networks. Specifically a modified version of the Residual Neural Network (RNN), which we call a Hamiltonian Inspired Neural Network (HINN) drawn from 3. To create this HINN, we primarily used 2 packages - PyTorch and hessQuik. The difference between our HINN, and the traditional RNN and Ruthotto \textit{et al}’ HINN, is in our forward propagation method and how we input values into our MSE loss function. The forward propagation uses the autograd feature to calculate both $\frac{\partial H_{\theta}}{\partial \mathbf{p}}$ and $\frac{\partial H_{\theta}}{\partial \mathbf{q}}$, where $\theta$ are the network parameters we wish to optimize and $H_{\theta}$ is our network output. We then use these values in discretizing $\mathbf{p_{\theta}}$ and $\mathbf{q_{\theta}}$. $$\mathbf{p_{\theta+1}} = \mathbf{p_{\theta}} + h\frac{\partial H_{\theta}}{\partial \mathbf{q}} $$ $$\mathbf{q_{\theta+1}} = \mathbf{q_{\theta}} - h\frac{\partial H_{\theta}}{\partial \mathbf{p}}$$

The new values are then plugged into our MSE loss function. Using these techniques we created two HINNs for two different examples, those being the Simple Spring Mass System and the Two Body Problem.

Results:

Figure 2 - Spring Mass System 1. Training Loss Graph 2. Learned Position Values over True Position Values 3. Relationship of Learned p and q over ground truth

Figure 3 - Two Body Problem 1. Training Loss Graph 2. Learned Trajectories Graph over True Trajectories 3. Learned Energy over True Energy 4. Learned Position over True Position.

What’s Next

Currently our results look promising as our learned values closely match the ground truth values. Moving foward, we would like to add more complexity to our current examples: instead of fixed time steps, we would like to attempt variable time steps. We would also like to take on the Three Body Problem, which unlike our current examples has no analytical solution.

Information about Us

Mathias Heider is a rising senior at the University of Delaware, majoring in Computer Science and Mathematics and Economics. His interest are in machine learning and data science specifically when it relates to dataset with bioinformatics applications. Outside of class, Mathias likes to ski, hangout with friends, and play video games

Carrie Vanty is a rising senior at Middlebury College, majoring in mathematics. She loves working with ordinary differential equations in applied math. Outside of school, Carrie likes to ski, hike, and craft.

Emma Hayes is a rising junior at Carnegie Mellon University, majoring in Computational and Applied Mathematics. She enjoys learning about new topics in mathematics and incorporating computer science into her work. Outside of class, Emma likes hiking, baking, and making art.

References

[1] https://en.wikipedia.org/wiki/Effective_mass_(spring%E2%80%93mass_system)#/media/File:Simple_harmonic_oscillator.gif

[2] Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016.

[3] Eldad Haber and Lars Ruthotto. Stable architectures for deep neural networks. Inverse Problems, 34(1):014004, 22, 2018.

[4] Brian de Silva, Kathleen Champion, Markus Quade, Jean-Christophe Loiseau, J. Kutz, and Steven Brunton. Pysindy: A python package for the sparse identification of nonlinear dynamical systems from data. Journal of Open Source Software, 5(49):2104, 2020.

[5] Alan A. Kaptanoglu, Brian M. de Silva, Urban Fasel, Kadierdan Kaheman, Andy J. Goldschmidt, Jared Callaham, Charles B. Delahunt, Zachary G. Nicolaou, Kathleen Champion, Jean-Christophe Loiseau, J. Nathan Kutz, and Steven L. Brunton. Pysindy: A comprehensive python package for robust sparse system identification. Journal of Open Source Software, 7(69):3994, 2022.

[6] Sam Greydanus, Misko Dzamba, and Jason Yosinski. Hamiltonian neural networks. ArXiv, abs/1906.01563, 2019.

Low-Precision Algorithms for Image Processing

Mon, 27 Jun 2022 11:36:49 -0400

This post was written by Xiaoyun Gong, Yizhou Chen, and Xiang Ji and published with minor edits. The team was advised by Dr. James Nagy. In addition to this post, the team has also created slides for a midterm presentation, a poster blitz video, code, and a paper.

In One Sentence:

Our group works on experimenting with iterative methods for solving inverse problems at different precision levels.

Background: Why Low Precision?

What is the most important aspect for an excellent gaming experience? A lot of people would answer real-time! Everyone wants their games to be fast, and it is always a bummer that the screen freezes during a critical combat. This is why we are investigating low precision arithmetic: to decrease the computation time and speed things up.

Nowadays, most computer systems operate on double precision (64-bit) arithmetic. However, if we decrease the number of bits for each number to 16 bits or even lower, the processing time can be much significantly reduced, although the benefit comes at the cost of a loss of accuracy.

Using tensor cores for mixed-precision scientific computing. Oct-2021. url: https://developer.nvidia.com/blog/tensor-cores-mixed-precision-scientific-computing/

Simulating Low Precision

Matlab function chop

To simulate low precision arithmetic on our 64-bit computers, we have imported a MATLAB package called chop. The toolbox allows us to explore single precision, half precision, and other customized formats. Each input needs to be transformed, but the real work comes from chopping each operation. The code below is a toy example of how to calculate $x + y \times z$ in half precision with chop.

Blocking

When the number is being chopped from double precision to half precision, a lot of bits are dropped (from 64 bits to 16 bits). This would certainly cause a level of inaccuracy, so in order to reduce the errors, a method called blocking is used. Blocking is the same as breaking a large operation into smaller chunks, where each is computed independently and the result is then summed.

We compute the inner product for each precision and block size for 20 times and calculate the average. The errors are calculated as the differences between the result of using the chopped version of inner product function and the built-in function in matlab.

On the left-hand side of the graph where the size of the vector is 1000, the errors of half precision are the largest because it has the least bits. If we take a closer look at only half precision, we get the graph on the right with different vector sizes. The errors decrease sharply when the blocking method is introduced. However, larger block sizes do not necessarily mean lower errors, as the graph suggested: the errors increase again as the block size keeps growing. That is because when the block size is large, it's the same as doing no blocking at all. For example, for a size-500 vector, once the block size reaches 500, it just means putting the whole vector into the first block, the same as when blocking is not introduced. Therefore, the line becomes flat from 500. We use 256 as our default block size in our codes because the matrix dimension is rather large in our problem.

Inverse Problems and Iterative Methods

Inverse problems are problems where our goal is to find the internal or hidden information (inputs) from outside measurements (results). The internal data can be approximated by iterative methods, a repeating implementation of the same system of equations, with variables getting updated each round, hoping the value generated can be closer to the true value we desire each term.

Conjugate Gradient Method

The Conjugate Gradient algorithm (CG) aims to solve the linear system Ax = b where A is SPD (symmetric and positive definite),transforming the problem of finding solution to an optimization problem where we want to minimize $\phi(x)=\frac{1}{2}x^{T}Ax-x^{T}b$. This can be easily seen from $\nabla \phi (x) = 0$ -> $Ax-b=0$.

In each step, the method provides us with a search direction and a step-length so that the error of this iteration is A-orthogonal to the search direction of the previous iteration. Eventually, it will converge to the minimal point. The CGLS algorithm is the least-squares version of the CG method, applied to the normal equation A^TAx = A^Tb. However, CGLS requires computing inner products, which can overflow for large-scale problems in low precision.

Chebyshev Semi-Iterative Method

The Chebyshev Semi-Iterative (CS) Method requires no inner product computation, which is great because inner products can cause overflow easily in low precision. But there is always the trade-off! The CS method requires the user to have an idea of the range of the matrix A’s eigenvalues. The result given by CS is a linear combination of all solutions in each iteration, and the weights are obtained from the Chebyshev polynomial, which has the favorable property to ensure that the result obtained in each iteration of CS is smaller than an upper bound.

Experiment

IR Tools

We modify the CGLS method in the IRtool package in Matlab so that it can operate in lower precision, and we use two test problems in the same package to investigate how the method performs at lower precision, mainly half precision.

Image Deblurring Using CGLS

First, we use our modified version of CGLS without regularization to solve the image deblurring problem. In this application, we solve Ax = b, where b is an observed blurred image, A is a matrix that models the blurring operation, and x is the desired clean image. We didn’t add any noise to b in the problem of Ax = b at the beginning, and the graphs are demonstrated below.

We use our modified version of CGLS for single and half precision, and the graph in single precision is similar to the graph in double precision. However, for half precision, the background is not the same as that in the double-precision or single-precision graph; it contains more artifacts.

We also plot the error norms of the solution at each iteration using different precision levels.

From the graph, all three error norms overlap from the beginning until around the 20th iteration, where the half-precision errors begin to deviate from those in single and double precision. The difference is due to the round-off errors of half precision, which add up and take over. Besides, the error norms for half precision terminates at the 28th iteration because overflow of inner products causes NaNs (Not a Number) to be computed during the iteration.

After investigating the idealized situations where there is no noise in the observed image, we then apply our code to problems that contains additive random noise to see how it is likely to perform in real life. That is, we try to compute x from the observed image b = Ax + noise.

For half precision, with 0.1% noise, the picture looks almost the same as the one that contains no noise. However, if the noise level is increased to 1%, the background has substantially more artifacts, while the middle object is still identifiable. Noise has taken over the black background but not the satellite yet. Eventually, the whole image is flushed with the noisy artifacts with 10% noise; the picture no longer contains any meaningful information. Notice that the results below are generated using x from the best iteration, that is the iteration with the smallest error norms, not from the last iteration.

Now we turn our attention to the error norm, the difference between the original image and the one our algorithm generates at each iteration. When 0.1% noise is added, as the number of iterations goes up, the error norm reduces significantly across all three formats. Intriguingly, for images with 1% or 10% noise, the best reconstruction is not the last iteration but somewhere along the middle (it’s around the 50th iteration for 1% and 10th for 10%). The reason behind the phenomenon is that while we are transforming the output image, b, the blended noise also gets inverted along each iteration. Eventually, the random data accumulate and dominate the solution at some point. We are showing the results where the error norm is the smallest to see what is the best possible solution we can compute. However, in reality the true x is not known, meaning we don’t know the error norms, so we can only show results from the last iteration, not from the best iteration.

Image Deblurring Using CS

In order to prevent the occurrence of overflow, we experiment with the CS algorithm (where no inner products are needed) and use chop for lower precision. Tikhonov regularization is applied to CS after we find out that the algorithm performs poorly due to the close-to-zero singular values of A when it’s ill-conditioned. Now we are solving: $$\min_{x} {||Ax-b||_2^2+\lambda^2||x||_2^2}$$ where $\lambda$ is a parameter that needs to be chosen. Here we show experiments for the case with 10% noise, and we use $\lambda$ = 0.199.

From the graph below, it is clear that even with 10% noise, the half-precision image looks very similar to that in double precision, better than what we have using CGLS (results from the last iteration).

For the image deblurring problem, we further comfirm the similarity by plotting the error norms.

We can see that the error norms of the three precision levels overlap, illustrating that the result in half precision is close to that in double precision.

Image Deblurring Using CGLS with regularization

To fairly compare CGLS and CS, we add Tikhonov regularization to CGLS and run the test problem again. The diagrams are listed below.

The diagram for half precision looks much better than that produced by CGLS without regularization. However, difference still presents between half and double precision in the background. At the end of the iteration for half precision, the error norms still increase again. If we zoom in the graph of the error norms for CS and CGLS with regularization, we can see that the error norms at half precision converge for CS but increases rapidly for CGLS, suggesting that for half precision, CS is a better choice, especially when the noise level is high. When the noise level is close to zero, CS becomes susceptible because of the accumulation of round-off errors. However, for double precision, the CGLS method with regularization is clearly more stable. Therefore, CGLS with regularization is more suitable for double precision.

We performed similar experiments with an image reconstruction problem from tomography; see our paper for further details!

More about us

Yizhou Chen

Hi! My name is Yizhou, but I go by Riley as well. I'm a rising junior at Emory University, double majoring in Applied Math and Physics. My interest is in computational Math & computational Physics. Outside work I enjoy watching sitcoms and my favourite one is Frasier! I am an animal person and I have a toy poodle who's nine years old. I also love cycling and hiking.

Xiaoyun Gong

Hello I am Xiaoyun Gong. I am a rising senior majoring in Applied Mathematics and Statistics. I am interested in math and I also enjoy coding!! In my free time I like watching anime (most recent favorite is Made in Abyss) and drawing. I like sweet food and I am a cat person. 🐱

Xiang Ji

Hi, I am Xiang Ji, but you can also call me Zoe. I am a rising junior at Emory University who is double majoring in applied mathematics and statistics and art history. My research interest is in computational mathematics and image processing. I enjoy going to art museums and watching movies. It's quite fun doing research this summer!

Model-based approaches to neuronal network firing and its subsequent validation with a previously recorded in-vivo dataset

Mon, 27 Jun 2022 11:36:49 -0400

The research featured in this blog post was performed by Carly Ferrell, Qile Jiang, and Olivia Leu, and the team was advised by Dr. Michael Caiola. This blog post was written by Carly Ferrell and published with minor edits. In addition to this post, the team has also given a midterm presentation, made a poster blitz video, and created a poster. They are currently working on a paper with the aim to publish it in an academic journal.

Mathematical Modeling of Healthy and Parkinsonian Firing Patterns in the Primate Thalamocortical Motor Circuit

Parkinson’s disease (PD) is a slowly progressing neuro-degenerative disease featuring impaired motor symptoms such as bradykinesia, muscular rigidity, and resting tremors.¹ In industrialized countries, PD affects 0.3% of all people and 1% of people over age 60.⁶ The basal ganglia, motor thalamus, and motor cortex are three main components of the brain’s motor circuit and are responsible for movement planning and execution; movement disorders such as PD can develop when the typical activity of this circuit is disrupted.^2,3 Specifically, PD is associated with the loss of dopaminergic neurons and altered neuronal oscillations in the beta-band (13-30 Hz).⁴ Other projects, such as the 2019 paper by M. Caiola and M. Holmes, have investigated the changes in the basal ganglia neuronal activity from a mathematical modeling perspective, but little research has been done on the parkinsonism-associated changes in the areas of the thalamus and cortex which are involved in the motor circuit.⁵ We employ a mathematical model to investigate network connection changes within the thalamocortical motor ciruit to better understand the transition from healthy to parkinsonian states in the brain.

Firing Rate Model

We choose to use a firing rate model to describe our system. This approach can successfully represent networks, since each unit in the model can represent a population of neurons receiving input (average firing rates) from other neuron populations.

A simplified circuit diagram of the thalamocortical motor circuit network is shown below, and provides the neuroscience basis for our model. The rounded squares each represent a population of neurons, which are connected by either excitatory (arrow-tipped lines) or inhibitory (circle-tipped lines) synaptic weights. The green circle represents the interneuron population of the thalamus.

GPi (y₁)	globus pallidus internal
TC (y₂)	thalamocortical neurons
CT5 (y₃)	corticothalamic layer 5
CT6 (y₄)	corticothalamic layer 6
RTN (y₅)	thalamic reticular nucleus
IN (γ)	thalamic interneuron population

Treating the interneuron population as a “relay,” γ, we can establish the following system of equations:

τ₁y’₁ = −y₁ + f₁(β₁ + h)

τ₂y’₂ = −y₂ + f₂(w₁₂y₁ + w₃₂y₃ + w₄₂y₄ − w₅₂y₅ + γ + b₂)

τ₃y’₃ = −y₃ + f₃(w₂₃y₂ + w₄₃y₄ + b₃)

τ₄y’₄ = −y₄ + f₄(w₃₄y₃ + b₄)

τ₅y’₅ = −y₅ + f₅(w₄₅y₄ + b₅)

γ = −w₆₂(−w₁₆y₁ − w₅₆y₅ + w₄₆y₄ + b₆)

y_i	average neuronal population firing rate
w_jk	weight of the connection between populations j and k
h	constant basal ganglia input
τ_i	membrane time constant
f_i	activation function

Note that w₂₃ represents the difference between the excitatory and inhibitory inputs from TC to CT5. Note also that w_jk > 0 and τ_i > 0.

This can be represented with vectors and matrices as:

Ty’ = −y + F(x) ⟹ Ty’ = Ay + B

Activation Function Selection

Neurons traditionally respond to inputs sigmoidally.^7,8,9 However, this model creates a nonlinear system of equations for which it is impossible to solve for eigenvalues analytically. In order to attain eigenvalues and be able to comment on the behavior of the model as a whole, we must establish a simpler activation function that still manages to approximate experimental neuron discharge behavior.⁵ A piecewise linear (PWL) activation function is ideal in our case, as it allows us to break down a complex system into linear pieces which can be solved and manipulated:

We can break down this system into 3⁵ = 243 distinct linear regions in space, each with its own steady state (fixed point in space which the solution tends to as time increases). Out of these 243 regions, only the region in which each activation function is between 0 spikes/sec and its maximum firing rate contains a physiologically realistic steady state, further denoted as the middle region (outlined in green in the diagram below).

Data Matching

This semi-linear firing rate model has a number of constant values that we must locate in experimental data and incorporate, namely the baseline firing rates, maximum firing rates, and membrane time constants for each neuron population involved in our simplified motor circuit model. We were able to find values for these parameters through literature review, although some required that we make estimates informed by information from areas of the brain that behave similarly or data on these parameters from mice, rats, or cats. However, there does not seem to be data that documents the baseline firing rate for the thalamic interneuron population in the primate brain. Given our uncertainty about the true baseline firing rate value for the primate thalamic interneuron population, we decided to create two models, one with the low and one with the high baseline. The parameter values are shown in the table below:

Neuron Population	b_i	M_i	τ_i
GPi (y₁)	55 Hz^{5,10,11,12,13}	200 Hz¹⁴	8 ms^11,15,16
TC (y₂)	18.5 Hz¹⁷	300 Hz	25 ms
CT5 (y₃)	7.25 Hz¹⁸	200 Hz	20 ms
CT6 (y₄)	7.25 Hz¹⁸	200 Hz	15 ms
RTN (y₅)	25 Hz¹⁷	500 Hz¹⁷	16.51 ms
IN (γ)	Low: 6 Hz¹⁹ High: 22.7 Hz²⁰	N/A	N/A

Stability and Steady State Conditions

No matter the disease state of our model, the neurons should not be at a state of maximal firing or absent firing for an extended period of time. Additionally, in Parkinsonian solutions, we should expect oscillations of firing rates. Thus the following must hold:

Middle region contains its own steady state, and trajectories must not stabilize in another region.
Healthy: Middle region is stable ⟶ trajectories are thus forced to stabilize in the middle region, making the system globally asymptotically stable.
Parkinsonian: Middle region is unstable ⟶ trajectories are thus forced to oscillate around the middle region, forming a globally stable limit cycle.

To determine stability, the PWL activation function allows us to solve for the eigenvalues of each of the 243 regions explicitly. We found 3 possible cases:

The region is stable regardless of weights.
The region’s stability is conditional on weight values.
The region (including the middle region) has eigenvalues that cannot be solved for analytically. Therefore, we used the Routh-Hurwitz Stability Criterion (RH) to derive 3 stability conditions.

Weight Search

The current literature does not specify the baseline firing rate for the interneuron population, b₆, so we took two estimates: b₆ = 6 for the low estimate, and b₆ = 22.7 for the high estimate.

Comparing our data to the predicted values our model outputted, we were able to minimize the sum of squared error between the two and find a healthy solution for both the low and the high estimates of b₆. The outputs for the low b₆ are shown below:

Healthy Solution:

w₁₂ = 1.520384442	w₁₆ = 1.621278311	w₂₃ = 0.4962387866	w₃₂ = 1.117631687
w₃₄ = 0.1540248925	w₄₂ = 1.217895798	w₄₃ = 0.0672671083	w₄₅ = 1.542582263
w₄₆ = 9.049109867	w₅₂ = 4.5350845	w₅₆ = 0.3330689302	w₆₂ = 7.127373038

Below is shown the firing rate outputs using these weights.

Here, all firing rates tend toward a specifc value as time increases, so they are stable solutions.

Parkinsonian Solution:

w₁₂ = 1.520384442	w₁₆ = 1.621278311	w₂₃ = 0.8691494663	w₃₂ = 0.5043792396
w₃₄ = 0.1540248925	w₄₂ = 1.217895798	w₄₃ = 0.0672671083	w₄₅ = 1.542582263
w₄₆ = 9.049109867	w₅₂ = 4.5350845	w₅₆ = 0.3330689302	w₆₂ = 7.127373038

Below is shown the firing rate outputs using these weights.

Here, several firing rates oscillate, indicating a limit cycle solution.

Weight Space

We were interested in the role of the thalamus in parkinsonian dysfunction, so we explored the relationship between w₂₃ and w₃₂, which represent the excitatory and inhibitory connections between TC and CT5. Forcing all correlating weights to be equal in healthy and parkinsonian solutions except w₂₃ and w₃₂, we found a healthy solution that could be forced into a parkinsonian state by only altering w₂₃ and w₃₂. In a parkinsonian solution, at least one of the Routh-Hurwitz stability conditions must be broken. We examined which condition or combination of conditions is broken when the system moves from a healthy to a parkinsonian state for different values of w₂₃ and w₃₂. The region plot for the low b₆ is shown below.

Conclusions and Future Directions

Our model can represent the average firing rates of healthy and parkinsonian states in the thalamocortical motor circuit.
We established stability and steady state conditions for the system to be healthy or parkinsonian.
We have found multiple sets of weights that both satisfy the conditions and match the neuronal firing patterns in our in-vivo primate dataset.
We discovered that changing only the connection strength between TC and CT5 can force the system from a healthy to a parkinsonian state.
Next steps:

Examine the transition from both healthy to parkinsonian and parkinsonian to healthy.
Explore methods of biologically validating our model by pharmacologically manipulating the weights of motor circuit network connections.

More About the Team

Carly Ferrell is a rising senior at Mississippi State University majoring in mathematics and minoring in statistics and music with a concentration in voice. She is interested in utilizing her skills in applied mathematics and statistcs to research music, specifically music theory and sight singing. Outside class, she enjoys reading, dancing, singing, and composing music.

Qile Jiang is a rising junior at Brown University majoring in Applied Mathematics. His primary research area is in applied dynamical systems, but he also has a keen interest in pure math topics such as algebra. Outside of school, he spends his time training boxing, painting, and going to operas and classical concerts.

Margaret Olivia Leu is a junior at Pomona College double majoring in mathematics and politics. She is interested in working on ways to use mathematics as a tool in the fields of politics and social justice work, and hopes to pursue a career that combines these two interests. Outside academics, she enjoys crocheting, cooking, and listening to music.

References

Sveinbjornsdottir, S. (2016).The clinical symptoms of Parkinson’s disease. Journal of Neurochemistry, 139(1), 318-324. https://doi.org/10.1111/jnc.13691.
DeLong, M. R., & Wichmann, T. (2007). Circuits and circuit disorders of the basal ganglia. Archives of Neurology, 64(1), 20–24. https://doi.org/10.1001/archneur.64.1.20.
Alexander, G. E., DeLong, M.R., & Strick, P.L. (1986). Parallel Organization of functionally segregated circuits linking basal ganglia and cortex. Annual Review of Neuroscience, 9(1), 357-381. https://doi.org/10.1146/annurev.ne.09.030186.002041
Galvan, A., Devergnas, A., & Wichmann, T. (2015). Alterations in neuronal activity in basal ganglia-thalamocortical circuits in the parkinsonian state. Frontiers in Neuroanatomy, 9, 5. https://doi.org/10.3389/fnana.2015.00005.
Caiola, M., & Holmes, M. H. (2019). Model and analysis for the onset of parkinsonian firing patterns in a simplified basal ganglia. International Journal of Neural Systems, 29(1). https://doi.org/10.1142/S0129065718500211.
de Lau, L. M. L., & Breteler, M. M. B. (2006). Epidemiology of Parkinson’s disease. The Lancet Neurology, 5(6), 525-535. https://doi.org/10.1016/S1474-4422(06)70471-9.
Rall, W. (1955). Experimental monosynaptic input-output relations in the mammalian spinal cord. Journal of Cellular and Comparative Physiology, 46(3), 413–437. https://doi.org/10.1002/jcp.1030460303
Wilson, C. J., & Bevan, M. D. (2011). Intrinsic dynamics and synaptic inputs control the activity patterns of subthalamic nucleus neurons in health and in Parkinson’s disease. Neuroscience, 198, 54–68. https://doi.org/10.1016/j.neuroscience.2011.06.049
Nambu, A., & Llinaś, R. (1994). Electrophysiology of globus pallidus neurons in vitro. Journal of Neurophysiology, 72(3), 1127–1139. https://doi.org/10.1152/jn.1994.72.3.1127
Kita, H., Tachibana, Y., Nambu, A., & Chiken, S. (2005). Balance of Monosynaptic Excitatory and Disynaptic Inhibitory Responses of the Globus Pallidus Induced after Stimulation of the Subthalamic Nucleus in the Monkey. Journal of Neuroscience, 25(38), 8611–8619. https://doi.org/10.1523/JNEUROSCI.1719-05.2005
Hikosaka, O. (2007). GABAergic output of the basal ganglia. Progress in Brain Research, 160, 209–226. https://doi.org/10.1016/S0079-6123(06)60012-5
Wichmann, T., Bergman, H., Starr, P. A., Subramanian, T., Watts, R. L., & DeLong, M. R. (1999). Comparison of MPTP-induced changes in spontaneous neuronal discharge in the internal pallidal segment and in the substantia nigra pars reticulata in primates. Experimental Brain Research, 125(4), 397–409. https://doi.org/10.1007/s002210050696
Bergman, H., Wichmann, T., Karmon, B., & DeLong, M. R. (1994). The primate subthalamic nucleus. II. Neuronal activity in the MPTP model of parkinsonism. Journal of Neurophysiology, 72(2), 507–520. https://doi.org/10.1152/jn.1994.72.2.507
Hashimoto, T., Elder, C. M., Okun, M. S., Patrick, S. K., & Vitek, J. L. (2003). Stimulation of the Subthalamic Nucleus Changes the Firing Pattern of Pallidal Neurons. The Journal of Neuroscience, 23(5), 1916–1923. https://doi.org/10.1523/JNEUROSCI.23-05-01916.2003
Nakanishi, H., Tamura, A., Kawai, K., & Yamamoto, K. (1997). Electrophysiological studies of rat substantia nigra neurons in an in vitro slice preparation after middle cerebral artery occlusion. Neuroscience, 77(4), 1021–1028. https://doi.org/10.1016/s0306-4522(96)00555-6
Nambu, A. (2007). Globus pallidus internal segment. Progress in Brain Research, 160, 135–150. https://doi.org/10.1016/S0079-6123(06)60008-3
van Albada, S. J., & Robinson, P. A. (2009). Mean-field modeling of the basal ganglia-thalamocortical system. I Firing rates in healthy and parkinsonian states. Journal of Theoretical Biology, 257(4), 642–663. https://doi.org/10.1016/j.jtbi.2008.12.018
Opris, I., Hampson, R. E., Stanford, T. R., Gerhardt, G. A., & Deadwyler, S. A. (2011). Neural Activity in Frontal Cortical Cell Layers: Evidence for Columnar Sensorimotor Processing. Journal of Cognitive Neuroscience, 23(6), 1507–1521. https://doi.org/10.1162/jocn.2010.21534
Ison, M. J., Mormann, F., Cerf, M., Koch, C., Fried, I., & Quiroga, R. Q. (2011). Selectivity of pyramidal cells and interneurons in the human medial temporal lobe. Journal of Neurophysiology, 106(4), 1713–1721. https://doi.org/10.1152/jn.00576.2010
Putrino, D. F., Chen, Z., Ghosh, S., & Brown, E. N. (2011). Motor Cortical Networks for Skilled Movements Have Dynamic Properties That Are Related to Accurate Reaching. Neural Plasticity, 2011, 1–15. https://doi.org/10.1155/2011/413543

Shallow vs. Deep Brain Network Models for Mental Disorder Analysis

Mon, 27 Jun 2022 11:36:49 -0400

This post was written by Erica Choi, Sally Smith, and Ethan Young and published with minor edits. The team was advised by Professor Carl Yang. In addition to this post and the paper, the team has also created slides for a midterm presentation, a poster blitz video, and a poster. This work was accepted at the BrainNN workshop at IEEE BigData 2022 and slides are available here.

Comparing Shallow vs. Deep Brain Network Models

Our shallow models use graph kernels to compare structural similarity of brain network data. Plugging those kernels into support vector machines allows us to classify patients’ brain scans. These kernel methods are called “shallow” because they do not require many layers of computation, unlike their “deep” model counterparts. Our deep models are graph neural networks, machine learning (ML) models that can exploit the local information of nodes in graph data to perform classification. In this project, our models are classifying brain scans as either diseased or healthy. We are comparing the two types of models, shallow and deep, to further determine which might be more useful in analyzing neuroimaging data, as well as working on using the models in conjunction with one another to leverage the strengths of both.

For useful background and definitions refer to Preliminaries.

Datasets

We are working with 2 datasets, one documenting human immunodeficiency virus (HIV) patients and one documenting bipolar disorder (BP) patients. Each dataset consists of functional magnetic resonance imaging (fMRI) scans, diffusion tensor imaging (DTI) scans, and classification labels in the form of integers, where 1 indicates a healthy patient and -1 indicates an unhealthy patient. Both datasets have been processed for us, as detailed in Section 3 of the paper authored by Cui et al.

Problem Formulation

The DTI and fMRI brain scans of each patient $i$ are represented as weighted adjacency matrices $\mathbf{W}_i \in \mathbb{R}^{M \times M}$. The adjacency matrix is constructed from the brain scan and is a natural way of mathematically representing graph data. Nodes in the brain network represent regions of interest (ROIs), and edge links between nodes indicate the strength of the connection between the two regions. In general, fMRI scans are considered to be more robust than DTI scans; specifically, fMRI scans are less affected by noise caused by data collection. Thus, our experiments prioritize working with the fMRI scans.

Classification Task

The standard graph classification task considers the problem of classifying graphs into two or more categories. The goal is to learn a model that maps graphs in the set of graphs $G$ to a set of labels $Y$. In this project, our set of graphs $G$ is the set of brain scans from patients and our set of labels $Y$ consists of two labels: diseased and healthy. The goal of our models is to classify brain scans accurately and improve model interpretability.

Implementation

For implementation of support vector machines (SVM) with graph kernels, we utilized threshold rounding to remove edge weights and sparsify the adjacency matrices. This means that values in the adjacency matrices were rounded to make the matrices simpler. While this results in information loss, it preserves the overall structure of the adjacency matrices and makes them usable for this particular method. It also makes the computation less expensive. Further manipulation creates a list of graph objects that are compatible with the Python package GraKel.

For implementation of graph convolutional networks (GCNs), we followed BrainGB’s code to create a data type that can be used with the Python package PyG.

For implementation of kernel graph neural networks (i.e., KerGNN), we followed KerGNN’s code and implemented threshold rounding to run experiments. The motivation for threshold rounding is the same as for implementing SVM.

Methods

1. Graph Kernels

Fig.1 - Support Vector Machines with Kernels

We computed three kernels to plug into SVM: Weisfeiler-Lehman (WL), Weisfeiler-Lehman Optimal Assignment (WLOA), and propagation (Prop). The choice of these kernels is motivated by exploiting structural information (i.e., subgraphs) in the brain networks. We tested these graph kernels to find which ones were most effective. On average, the propagation kernel classified HIV best and the WLOA kernel classified bipolar disorder best.

2. Graph Convolutional Networks (GCNs)

Fig.2 - BrainGB Framework

The deep model that we experimented with in this project is graph convolutional networks (GCNs). This is a “deep” model because GCNs are under the umbrella of “deep learning”. GCNs are modern ML algorithms that pass information through several layers and do more extensive computations than shallow models. Note that our GCNs (and GNNs in general) are shallow in the sense that the models have few layers. Machine learning is still a very active field of research, and recent interest in graph data has led to major strides in graph-based ML.

We implement message passing GNNs (MPGNN)—a type of GCN—using the BrainGB Python package, which is built on the Pytorch and Pytorch Geometric libraries. MPGNNs involve what are called message passing schemes to aggregate information from a node’s neighbors. Figure 2, adapted from Cui et al., visualizes the MPGNN architecture.

3. Merging Graph Kernels and GNNs

To leverage the higher order structural information given by graph kernels and local information given by GCNs, we implement GNNs that incorporate various graph kernels (WL, WLOA, etc.) and benchmark their performance on our dataset. The frameworks of particular interest to us are:

the graph convolution layer (GKC) proposed by Cosmo et al., visualized in Figure 3, and
the kernel graph neural network (KerGNN) proposed by Feng et al., visualized in Figure 5.

Fig.3 - GKNN Framework

Fig.5 - KerGNN Framework

Results

Our most successful model was a graph attention network (GAT) model that used a node concatenation message passing scheme. This model was able to classify HIV patients as healthy or diseased with 81% accuracy on average. In general, our highest performing models were classifying HIV data, particularly using deep models.

Our highest performing model for bipolar disorder prediction used support vector classifiers (SVCs) with propagation WLOA kernels and had average accuracy of 63%. The differences in performance are minor; furthermore, all kernels’ mean performance had high standard deviation.

Our preliminary results from using a combination of kernel methods and GNNs are not outperforming our HIV-GAT(node) model, but we are seeing some improvements in classifying bipolar disorder with the hybrid model, particularly with KerGNN.

Discussion

In general, we found that our models were better able to classify HIV patients than BP patients. Cui et al. observes that HIV affects both the visual network (VN) and default mode network (DMN), while bipolar disorder mainly affects the bilateral limbic network (BLN). It is possible that HIV was easier to model because it significantly affected multiple networks in the brain, while BP was more elusive with only one major network significantly affected.

For more details and discussion of our results, see our manuscript (coming soon).

Limitations

Due to our datasets consisting of less than 100 patients each, our results may not generalize well beyond our specific dataset. If this study were to be replicated, a larger dataset would be ideal, but the expensive nature of brain imaging data and its processing requirements will pose some degree of limitation to any study that uses it. Additionally, brain imaging data is a highly protected data type due to the right to privacy of the patients whose brain scans are used in these experiments. This means that much of the information about the patients is kept private, so it can be challenging to find confounding variables or alternative explanations for statistical results from this data.

Another notable limitation is that of structure of the brain networks themselves. Specifically, it remains unclear what subgraphs and higher-order information are relevant in classifying brain scans as belonging to diseased or healthy individuals. GNNs also have limitations. For example, GNNs are prone to overfitting, especially with datasets as small as our own. This is an issue that could potentially be alleviated with access to a larger dataset.

Future Work

There are many avenues with which we may take future research in brain network classification. There are many ways of incorporating graph kernels into GNNs that improve the interpretability of the model, which in turn gives insights into the key underlying structures that help to classify brain networks.

Preliminaries

For technical details of implementing support vector classifiers (SVC) using the Python package sklearn, see this link.

For the mathematical theory underlying SVC, see this blog post.

For a survey of graph kernels, see this paper.

For an introduction to graph neural networks (GNNs), see this blog post.

References

BrainGB: A Benchmark for Brain Network Analysis with Graph Neural Networks

BrainNNExplainer: An Interpretable Graph Neural Network Framework for Brain Network based Disease Analysis

Deep Graph Kernels

Graph Kernel Neural Networks

KerGNNs: Interpretable Graph Neural Networks with Graph Kernels

From Images to Patient-Specific Models in Cardiology

Tue, 14 Dec 2021 00:00:00 +0000

This post was written by Kai Chang, Allison Dennis, Shannon Lee, Michele Perry, Minxing (Matt) Zhang, and Mohamad Hindawi and published with minor edits. The team was advised by Dr. Alessandro Veneziani. In addition to this post, the team has also created slides for a midterm presentation, a poster blitz video, and a poster.

Background: Mathematical Modeling from the Heart

The role of mathematical modeling in clinics is particularly evident in cardiology, as computational mechanics for many historical reasons is a mature field of applied mathematics; on the other hand, many important cardiovascular pathologies have a significant mechanical component, in terms of fluid, structure and their interactions. The clinical impact of mathematical models strongly relies on reconstructing patient geometries to customize and personalize numerical simulations. Advances in medical image processing made over the last two decades have enabled virtual patient-specific models. A key step of the processing pipeline in Cardiology is the extraction of complex vascular geometries like an aortic dissection from medical images (typically, Computed Tomographies, Magnetic Resonance, and Optical Coherence Tomography). Our team evaluated the relation between PDEs and image segmentation/reconstruction through the level set method, and compared this segmentation approach with deep learning ones based on Convolutional Neural Networks.

Heart

Project Overview: Comparison of Segmentation Techniques

Imaging has been revolutionizing medical research and clinical practices for decades. One of the components of medical image processing is “segmentation” which has a wide range of applications. Segmentation is the image processing step to identify a region of interest (an artery, a bone, etc.) in an image. We focused on medical images, specifically coronaries based on Optical coherence tomography (OCT). Using OCT images provided by the Emory School of Medicine, we analyzed two segmentation approaches: the Level-Set Method and Machine Learning based on CNN (Convolutional Neural Networks). Our project aimed at segmenting coronaries based on the OCT images using these two methods. We then compared the results of deterministic image-segmentation methods vs deep learning ones.

Picture1

Materials and Methods

Level Set Methods and VMTK

The Level Set Method utilizes implicit functions to identify the region of interest in an image, where the implicit function is the numerical solution of a Partial Differential Equation (PDE) that is defined on the image that is being segmented. The basic idea of the Level Set is to correlate the velocity to the gray level of the image in such a way that the gray level of the image is driving the evolution of the phi close to the boundary. The Level Set is a very powerful method that extracts the border of a region or image, and it can handle changing topology well. It is also a great tool because it utilizes physical concepts, such as velocity, mean curvature, and elastic energy for image segmentation problems. We used the image segmentation software, Vascular Modeling ToolKit (VMTK), that is based on the Level Set. VMTK is a collection of tools and libraries for image-based modeling of medical images. This segmentation method is model-driven, meaning that the technique is established on physical concepts of the problem.

Figure1

FEniCS and MATLAB

The FEniCS Project is a research and software project aimed at creating mathematical methods and software for automated computational mathematical modeling. As the implicit function, the numerical solution of a PDE, is involved in our project, we tried to use FEniCS in Python for solving PDEs using finite element methods. For MATLAB, our group used Image Segmenter App under Image Processing Toolbox and applied Thresholding, Active Contours, Graph Cut, Auto Cluster, etc. to segment 2-D images. For 3-D volumetric images, we used Volume Segmenter to create and refine binary or semantic segmentation masks to segment the images by means of automated, semi-automated, and manual techniques.

Figure2

Convolutional Neural Networks (CNNs)

Convolutional Neural Networks are a deep learning algorithm for image classification. The CNN’s convolutional layer parameters comprise of filters, where the values of the filters are learned during the training phase. The layers are for feature learning and classification, specifically for classifying the pixels in an image with respect to a background or vessel. We used the image processing software, PyTorch. The deep learning (DL) based method involves using training data from a database of images to train the algorithm in PyTorch.

Figure7

Our Activities & Experiences

Weeks 1 and 2

Using CNNs, we generated OCT segmentation maps in PyTorch on the original data provided by Dr. Molony. The same data were used in VMTK slice by slice.

Week 3

We added 15 new images to our OCT data set in PyTorch. With more training data, the loss value went down, and the accuracy increased. We worked to find the most accurate segmentation process in VMTK to extract the level set for the boundary of the coronary. We cleaned a sample image using GIMP. To create a 3D stack of images, each slice was replicated 30 times. After extracting the surface in VMTK, ParaView extracted the outline of the region. We were then in a good position to start comparing the contours from the two methods.

Research Week 4:

We explored different metrics for comparing the results visually and numerically. One option is the Jaccard Index, which quantifies the percent overlap between the segmented images. Another option was to extract the VMTK and PyTorch contour lines and boundary points in ParaView and then use Python or Matlab code to plot the x and y coordinates of the results as overlapping figures. Then, we could use integration to calculate the difference between the two curves (L^2 metrics) .

Illustration2

Comparison

Which method was faster?

The deep learning method was much faster than VMTK, but it still required a waiting period to complete the epochs. This might be because our training data and development data was relatively small. We came to recognize that a limitation of the deep learning approach was that labeled and large data sets are required for preventing CNN overfitting and increasing accuracy.

Which method required more human intervention?

VMTK segmentation required far more human intervention than the DL method. With VMTK, we had to initialize the image by selecting an initialization type, identifying a lower and upper threshold of pixel values, and placing seeds that identified the region we planned to segment. We also selected the numerical values for the level set conditions, including number of iterations, propagation, curvature, and advection. We actively supervised the segmentation and chose whether to accept or reject the results. On the other hand, the DL method required much less decision making. With the DL algorithm, we empirically set a few parameters associated with training the CNN, such as training rate and number of epochs. Then, the DL algorithm trained itself and segmented the OCT images without user participation.

Other advantages and limitations?

PyTorch is better at handling noise suppression than VMTK. On the other hand, VMTK handles the data normalization, contrast enhancement, and conversion of color images to grayscale better.

Illustration

Our Results & Conclusions

After evaluating the benefits and limitations of both methods, we have come to the conclusion that instead of preferring one over the other, we can combine the two for better results. We could train a neural network with an optimal parameter selection to use in VMTK segmentation for setting the parameters. We can have a CNN trained on the OCT data to tell us what the correct parameter set is. This could prove to be useful, especially since we encountered a set-back in determining the initialization type, thresholds, and parameters to set for the best segmentation result in VMTK. We should not have a competition over the two techniques, rather, we should combine the strengths and benefits of both model-driven and data-driven approaches.

Future works should focus on improved segmentation using unsupervised Deep Learning where the machine uses image-derived features, or supervised learning that requires Gold Standard (GS) segmentation to train it. The deep learning-based algorithm demonstrated high accuracy based on Jaccard Index. We should look into Edge-based deformable models, and approaches using blood vessel tracking algorithms and seeding points to find the minimum path according to image-derived metrics.

More About The Team

Allison Dennis is a rising junior at Texas A&M University majoring in Applied Mathematics with a minor in Cybersecurity. Her interests are scientific computing and linear algebra, and her dream career is to be a Calculus 1 and 2 professor. Besides doing math, she loves to read, hike, watch Gilmore Girls, and spend time with her family and friends.
Dr. Mohamad Hindawi teaches AP Calc in Tucker High school. He is certified to teach AP Physics, AP Chem, AP Stat, and AP Comp Sc. His interest is in Mathematical Modeling and Differential Eq specially Navier-Stokes Equation. Outside academia, He enjoys swimming long distances, deep sea water fishing in Alaska for Halibut and King Salmon.
Shannon Lee is a rising junior at Southern Methodist University majoring in accounting, applied mathematics, and statistics. Her interests are learning new computational math and statistical modeling techniques in all areas. Outside of school, she enjoys playing tennis, traveling, and spending time with friends and family.
Michele Perry is a rising senior at University of South Alabama majoring in Math/Statistics and minoring in music. Her dream career is to utilize math in astronomical research at NASA or a university. Outside of academics, she spends her time practicing for band, reading, traveling, and hanging out with her cat and dogs.
Minxing(Matt) Zhang
Kai Chang

References

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3785070/
Moccia, Sara, et al. “Blood vessel segmentation algorithms—review of methods, datasets and evaluation metrics.” Computer methods and programs in biomedicine, 158 (2018): 71-91.
https://fenicsproject.org

Iterative Methods at Lower Precision

Fri, 01 Aug 2031 00:00:00 +0000

Since numbers in the computer are represented with a fixed number of bits, loss of accuracy during calculation is unavoidable. At high precision where more bits (e.g. 64) are allocated to each number, round-off errors are typically small. On the other hand, calculating at lower precision, such as half (16 bits), has the advantage of being much faster. This research focuses on experimenting with arithmetic at different precision levels for large-scale inverse problems, which are represented by linear systems with ill-conditioned matrices. We modified the Conjugate Gradient Method for Least Squares (CGLS) and the Chebyshev Semi-Iterative Method (CS) with Tikhonov regularization to do arithmetic at lower precision using the MATLAB chop function, and we ran experiments on applications from image processing and compared their performance at different precision levels. We concluded that CGLS is a more stable algorithm, but overflows easily due to the computation of inner products, while CS is less likely to overflow but it has more erratic convergence behavior. When the noise level is high, CS outperforms CGLS by being able to run more iterations before overflow occurs; when the noise level is close to zero, CS appears to be more susceptible to accumulation of round-off errors.

Call for Summer 2026 Applications

Fri, 19 Dec 2025 00:00:00 +0000

We are pleased to announce that our applications website for REU applicants is now live. We will start reviewing applications on February 1 and all applications received by March 1 will receive full consideration.

Our theme will be Computational Science and AI for New Mathematics. This year’s projects span both pure and applied mathematics, exploring how modern AI can push the boundaries of mathematical discovery. For more information about the application process, see this page and for a list of projects see the Summer 2026 tab.

For questions about the program, please contact Dr. Levon Nurbekyan at levon.nurbekyan@emory.edu.

2026 Project Mentors

Dr. Levon Nurbekyan - AI-Assisted Exploration of the Polygonal Faber-Krahn Inequality
Dr. Marco Tezzele and Dr. Jimena Martin Tempestti - Generative AI for Structure Discovery in Cardiovascular Digital Twins
Dr. Deependra Singh - AI-Assisted Exploration in Algebra and Number Theory

Applications of Automatic Differentiation in Image Registration

Mon, 28 Jul 2025 00:00:00 +0000

We demonstrate that automatic differentiation, which has become commonly available in machine learning frameworks, is an efficient way to explore ideas that lead to algorithmic improvement in multi-scale affine image registration and affine super-resolution problems. In our first experiment on multi-scale registration, we implement an ODE predictor-corrector method involving a derivative with respect to the scale parameter and the Hessian of an image registration objective function, both of which would be difficult to compute without AD. Our findings indicate that exact Hessians are necessary for the method to provide any benefits over a traditional multi-scale method; a Gauss-Newton Hessian approximation fails to provide such benefits. In our second experiment, we implement a variable projected Gauss-Newton method for super-resolution and use AD to differentiate through the iteratively computed projection, a method previously unaddressed in the literature. We show that Jacobians obtained without differentiating through the projection are poor approximations to the true Jacobians of the variable projected forward map and explore the performance of some other approximations. By addressing these problems, this work contributes to the application of AD in image registration and sets a precedent for further use of machine learning tools in this field.

Call for Summer 2025 Applications

Thu, 19 Dec 2024 00:00:00 +0000

We are pleased to announce that our applications website for REU applicants is now live. As we did last year, we are using NSF’s new ETAP system for the first time, which should make it easy for applicants to apply. We will start reviewing applications on February 1 and all applications received by March 1 will receive full consideration.

Our theme will be Models meet Data. For more information about the application process, see this page and for a list of projects see the Summer 2025 tab.

Highlights from Our 2024 Summer REU Program

Thu, 07 Nov 2024 00:00:00 +0000

This summer, we welcomed 15 talented students from across the country on Emory’s beatiful campus. Under the umbrella theme Learning from Images, the students worked in five teams, each diving into projects aimed at advancing our understanding and ability to learn from imaging data. This program, held over eight weeks on campus, provided the young researchers with an opportunity to engage in cutting-edge computational mathematics and machine learning.

The Learning from Images theme tackled a timely challenge: how to make sense of the overwhelming amount of imaging data generated daily. Computational algorithms can unlock insights from this data, improving our ability to detect new patterns and enhance image quality, especially in areas critical to healthcare. Each project addressed a unique aspect of this challenge—from improving image distribution models and optimizing measurement designs to refining algorithms for reconstructing image sequences. You can read more about the projects here.

The students worked with their mentors to explore and develop advanced imaging techniques that leverage machine learning, numerical linear algebra, optimization, and differential equations. Weekly project meetings, a seminar series, and various social events fostered an engaging learning environment. Students had the chance to present their progress in midterm presentations, culminating in a highly anticipated final poster session.

The poster session marked the program’s grand finale, drawing a large crowd of faculty, students, and postdocs. Each team showcased their findings and shared insights from their experiments, highlighting the strengths and potential areas of improvement for their algorithms. By a narrow margin, this year’s poster award went to the team of Malia Walewski, Rishi Leburu, Claire Gan, and Callihan Bertley for their project on Improving VAEs with Normalizing Flows. The picture below shows Malia and Rishi presenting their work.

All teams created a poster blitz video and wrote blog posts about their work, which are now live on the 2024 tab. These additional resources capture the excitement and innovation of the program and give a closer look into the students’ experiences.

We’re proud of our 2024 REU cohort and look forward to following their careers in the years to come!

Training Implicit Networks for Image Deblurring using Jacobian-Free Backpropagation

Thu, 01 Feb 2024 00:00:00 +0000

Recent efforts in applying implicit networks to solve inverse problems in imaging have achieved competitive or even superior results when compared to feedforward networks. These implicit networks only require constant memory during backpropagation, regardless of the number of layers. However, they are not necessarily easy to train. Gradient calculations are computationally expensive because they require backpropagating through a fixed point. In particular, this process requires solving a large linear system whose size is determined by the number of features in the fixed point iteration. This paper explores a recently proposed method, Jacobian-free Backpropagation (JFB), a backpropagation scheme that circumvents such calculation, in the context of image deblurring problems. Our results show that JFB is comparable against fine-tuned optimization schemes, state-of-the-art (SOTA) feedforward networks, and existing implicit networks at a reduced computational cost.

Call for Summer 2024 Applications

Thu, 21 Dec 2023 00:00:00 +0000

We are pleased to announce that our applications website for REU applicants is now live. This year, we are using NSF’s new ETAP system for the first time, which should make it easier for applicants to apply. We will start reviewing applications on February 1 and all applications received by March 1 will receive full consideration.

For more information about the application process, see this page and stay tuned for a list of projects to be posted a the Summer 2024 tab.

Learning from Images

The amount of imaging data generated every day exceeds human imagination. With their ability to statistically analyze such large datasets, computational algorithms can enhance our ability to discover new patterns and improve imaging data quality for critical healthcare applications and beyond.

This theme’s projects provide new mathematical insights and algorithms enabling learning from image data. The goals of the individual projects include improving algorithms for learning the distribution of image data, optimizing the measurement design to improve image quality, developing efficient algorithms for reconstructing image sequences, and generalizing machine learning techniques to learn transformations between images.

The projects build upon and advance state-of-the-art techniques from machine learning, numerical linear algebra, and differential equations. Students will learn about these techniques and be trained to combine them in new ways to build effective algorithms. Their analysis and experiments will provide new insights into the strengths and weaknesses of their approaches.

MCRAGE: Synthetic Healthcare Data for Fairness

Fri, 27 Oct 2023 00:00:00 +0000

In the field of healthcare, electronic health records (EHR) serve as crucial training data for developing machine learning models for diagnosis, treatment, and the management of healthcare resources. However, medical datasets are often imbalanced in terms of sensitive attributes such as race/ethnicity, gender, and age. Machine learning models trained on class-imbalanced EHR datasets perform significantly worse in deployment for individuals of the minority classes compared to samples from majority classes, which may lead to inequitable healthcare outcomes for minority groups. To address this challenge, we propose Minority Class Rebalancing through Augmentation by Generative modeling (MCRAGE), a novel approach to augment imbalanced datasets using samples generated by a deep generative model. The MCRAGE process involves training a Conditional Denoising Diffusion Probabilistic Model (CDDPM) capable of generating high-quality synthetic EHR samples from underrepresented classes. We use this synthetic data to augment the existing imbalanced dataset, thereby achieving a more balanced distribution across all classes, which can be used to train an unbiased machine learning model. We measure the performance of MCRAGE versus alternative approaches using Accuracy, F1 score and AUROC. We provide theoretical justification for our method in terms of recent convergence results for DDPMs with minimal assumptions.

Emory Welcomes 2023 REU/RET Participants on Campus

Tue, 13 Jun 2023 00:00:00 +0000

This is an exciting week for us as we finally entered the core phase of our 2023 REU/RET. We are delighted to have a diverse and talented group of participants joining us from different states and institutions. They will spend the next six weeks on Emory’s campus and will have access to state-of-the-art facilities, resources, and mentorship. We look forward to getting to know them and supporting them throughout this journey.

The theme of this year’s program is data for social justice and our five teams will explore various topics related to this theme from a mathematical perspective. Our projects span topics motivated by health disparities, environmental justice, and fairness in deep learning. The participants will study and develop mathematical algorithms and computational tools to analyze and solve open challenges in these areas. They will have plenty of opportunities to meet with their mentors and peers who will provide feedback and guidance.

In the next six weeks, our participants will not only work on their research projects, but also engage in a variety of activities that will enhance their skills and knowledge in data science. Our weekly seminar and ad-hoc lectures will train participants in most aspects of mathematical research in this area and give advice on career development.

We are excited to kick off this program and look forward to a productive and fun summer with our participants!

Ensemble Kalman Filtering for Glacier Modeling

Thu, 01 Dec 2022 00:00:00 +0000

Working with a two-stage ice sheet model, we explore how statistical data assimilation methods can be used to improve predictions of glacier melt and relatedly, sea level rise. We find that the EnKF improves model runs initialized using incorrect initial conditions or parameters, providing us with better models of future glacier melt. We explore the necessary number of observations needed to produce an accurate model run. Further, we determine that the deviations from the truth in output that stem from having few data points in the pre-satellite era can be corrected with modern observation data. Finally, using data derived from our improved model we calculate sea level rise and model storm surges to understand the affect caused by sea level rise.

Participants present their work at the REU/RET Poster Session

Sun, 26 Jun 2022 00:00:00 +0000

In their last week on campus, our REU/RET teams presented their work in our annual poster session. The event was held in our beautiful atrium and was a magnet for many students, postdocs, and faculty from the Departments of Mathematics and Computer Science.

In preparation for the event and to advertise their work, the teams made two-minute Poster Blitz videos. Since the videos were full of creativity, we played them during the event using our smart screen. The videos are now hosted at:

During the event, more than a dozen judges had the difficult task of scoring the quality of the posters, how well the teams presented and explained their work, and the results of the research itself, to determine the winning team. All teams scored very well on and with the smallest of margins, the poster on reinforcement learning vs. optimal control won the award. The team consists of Arjun Sethi-Olowin (a rising senior at Rice University), Dewan Chowdhury (a rising junior at Rutgers University), and Jacob Mantooth (a rising senior at East Central University). They each received a $50 gift card from a major online retailer.

But no one left empty-handed! Our official REU/RET t-shirts, designed by our outreach committee, arrived just in time. Also, we held a raffle with Emory and SIAM merchandise. Here, Jim Nagy presents one of the main prizes, a pair of SIAM socks, going to Logan Knudsen:

We thank the Department of Mathematics for making the poster session a huge hit and providing the prizes.

Emory Welcomes 2022 REU/RET Participants on Campus

Mon, 13 Jun 2022 00:00:00 +0000

Today marks the first day of our core REU/RET phase 2022, and we are delighted to welcome our participants from across the country to our beautiful campus. This year on our site, 22 undergraduate researchers and four teachers will work on eight projects under the theme Models meet Data. All projects will combine tools from computational mathematics (modeling, differential equations, optimization, linear algebra) with statistical and data science techniques (deep learning, reinforcement learning, data assimilation) to tackle challenging research questions.

Over the next six weeks, the groups will have several milestones which give opportunities to practice most modes of scientific communication. In their research contracts, the participants will organize their collaboration and develop a timeline for the project. Their midterm presentations give an excellent opportunity to expose synergies and ideas across the teams. The participants will also create a blog post about their work, and there will be a poster session toward the end of the program. Throughout the six weeks, the teams will work on a manuscript that describes their project.

In addition to working on this packed research schedule, we will have many events to foster learning (e.g., ad-hoc lectures, a weekly seminar), networking (e.g., excursions and other social events organized by the participants), and professional development (e.g., scientific writing workshops, career panels).

Besides the NSF support, we are also grateful for the support from Emory’s Department of Mathematics, which provides the top-notch facilities to run this program smoothly and funds to host events such as this morning’s breakfast. In addition to one co-working area, a computer pool, and two seminar rooms dedicated to our site, we’ll also try out our department’s computing facilities and new digital whiteboard.

REU Participants Receive National Fellowships

Fri, 20 May 2022 00:00:00 +0000

We are excited to congratulate Manuel Santana and Katie Keegan, who participated in our program in the summer of 2021, on winning nationally-competitive scholarships that will accelerate enhance their graduate school career.

Manuel Santana, a 2021 Goldwater Scholar, who graduated with a BA in Computational Mathematics from Utah State University was awarded a prestigious scholarship through the NSF’s Graduate Research Fellowship Program (GRFP). Besides his research on computational methods that accelerate the reconstruction of computed tomography on our site, he also worked on combinatorics, and shared many tutorials for solving algebra problems. He will join Cal Tech for his PhD.

Katie Keegan was offered not only an NSF GRFP but also a [Computational Science Graduate Fellowship by the US Department of Energy]0(https://www.krellinst.org/csgf/), which she accepted. Katie completed her BS in Applied Mathematics at Mary Baldwin College this spring and participated in two REUs. In the summer of 2020, she was part of ICERM’s REU and published a paper on a modified watermarking scheme based on the SVD, which was later also featured in SIAM News. On our site she developed a tensor SVD-based classification algorithm for fMRI data. We are extremely delighted that Katie will join our Computational Mathematics PhD program starting this fall.

2022 Projects and Mentors Announced

Thu, 24 Feb 2022 00:00:00 +0000

With the application deadline for the summer 2022 program approaching, we’re delighted to announce the project topics and mentors. This year, we will have six projects centered on the theme models meet data. Applications include storm surge modeling and neuroscience and mathematical techniques cover a variety of topics ranging from data assimilation to deep neural networks and reinforcement learning. The teams will be mentored by faculty from Emory’s department of Mathematics and Computer Science and two former faculty and students who will join us as guest mentors. A full list of topics can be found on the Summer 2022 page. Applications for the REU and RET are due on March 1.

Comparing Shallow and Deep Graph Models for Brain Network Analysis

Tue, 04 Jan 2022 00:00:00 +0000

It is well-established that graph neural networks (GNNs) can effectively model networked data in a variety of fields. However, whether GNNs can outperform traditional shallow graph classification models such as graph kernels for brain network analysis remains unclear. To this end, we analyze different approaches for modeling brain networks, including graph kernel based SVM, basic GNNs and kernelized GNNs. These models are designed to aid in the analysis of diseases and mental disorders such as bipolar disorder, human immunodeficiency virus (HIV), post-traumatic stress disorder (PTSD), and depression. In particular, we conduct experiments with three methods: kernelized support vector machines (SVM), message passing graph neural networks (MPGNNs), and kernel graph neural networks (KerGNN). We conclude that 1) deep models (GNNs) generally outperform shallow models (SVM) and 2) models considering specific graph motifs do not seem to significantly improve performance. We also identify other graph kernels and GNN frameworks that show promise in motivating further research in brain network analysis.

A Tensor SVD-based Classification Algorithm Applied to fMRI Data

Sun, 31 Oct 2021 00:00:00 +0000

To analyze the abundance of multidimensional data, tensor-based frameworks have been developed. Traditionally, the matrix singular value decomposition (SVD) is used to extract the most dominant features from a matrix containing the vectorized data. While the SVD is highly useful for data that can be appropriately represented as a matrix, this step of vectorization causes us to lose the high-dimensional relationships intrinsic to the data. To facilitate efficient multidimensional feature extraction, we utilize a projection-based classification algorithm using the t-SVDM, a tensor analog of the matrix SVD. Our work extends the t-SVDM framework and the classification algorithm, both initially proposed for tensors of order 3, to any number of dimensions. We then apply this algorithm to a classification task using the StarPlus fMRI dataset. Our numerical experiments demonstrate that there exists a superior tensor-based approach to fMRI classification than the best possible equivalent matrix-based approach. Our results illustrate the advantages of our chosen tensor framework, provide insight into beneficial choices of parameters, and could be further developed for classification of more complex imaging data. We provide our Python implementation in this repository.

Comparison of atlas-based and neural-network-based semantic segmentation for DENSE MRI images

Wed, 29 Sep 2021 00:00:00 +0000

Two segmentation methods, one atlas-based and one neural-network-based, were compared to see how well they can each automatically segment the brain stem and cerebellum in Displacement Encoding with Stimulated Echoes Magnetic Resonance Imaging (DENSE-MRI) data. The segmentation is a pre-requisite for estimating the average displacements in these regions, which have recently been proposed as biomarkers in the diagnosis of Chiari Malformation type I (CMI). In numerical experiments, the segmentations of both methods were similar to manual segmentations provided by trained experts. It was found that, overall, the neural-network-based method alone produced more accurate segmentations than the atlas-based method did alone, but that a combination of the two methods – in which the atlas-based method is used for the segmentation of the brain stem and the neural-network is used for the segmentation of the cerebellum – may be the most successful.

Alternating Minimization for Computed Tomography with Unknown Geometry Parameters

Wed, 18 Aug 2021 00:00:00 +0000

Due to the COVID-19 pandemic, there is an increasing demand for portable CT machines worldwide in order to diagnose patients in a variety of settings [16]. This has lead to a need for CT image reconstruction algorithms that can produce high-quality images in the case when multiple types of geometry parameters have been perturbed. In this paper, we present an alternating descent algorithm to address this issue, where one step minimizes a regularized linear least squares problem, and the other minimizes a bounded non-linear least-square problem. Additionally, we survey existing methods to accelerate the convergence algorithm and discuss implementation details through the use of MATLAB packages such as IRtools and imfil. Finally, numerical experiments are conducted to show the effectiveness of our algorithm.

Mon, 01 Jan 0001 00:00:00 +0000

Information for Applicants

Our site provides several funded summer research opportunities for undergraduate students. For summer 2026, applications can be submitted via ETAP. We will begin reviewing applications on February 1 and all applications received by March 1 will receive full consideration. Until we finalize the roster (usually by early April) we will not decline qualified applicants.

Eligibility

Students must be enrolled in an undergraduate program at a US institution during the summer and must not graduate before the end of the program.

Participants must be US citizens or permanent residents due to NSF funding requirements.

We expect participants to be fully available in-person during our REU phase and on campus for eight weeks during the summer. This year’s program will be held from June 15 until August 7.

Expectations and Pre-Requisites

The research projects are designed to be accessible to participants with a strong working knowledge in Linear Algebra, Vector Calculus, Differential Equations and elementary programming experience. This can either be from past courses, courses students are enrolled in during the Spring term, or from independent studies. To help participants learn other project-specific materials, our site includes a comprehensive research training plan and our mentors are experienced and accessible to explain more advanced materials.

Our activities include professional development, a weekly lunch seminar, and social dinners. Past participants have also often organized trips in the Atlanta area.

Stipend Information

Supported students will receive a stipend of 5,600 USD, travel support of up to 800 USD to and from Emory, and free on-campus accommodations.

Housing

Participants will be housed on Emory’s beautiful Clairmont Campus. You can find a full description of the facilities here. We share an application link in April so everyone can register on time. Ideally, participants should arrive on the Sunday before the start of the program and move out on the Saturday after the program.

Participants will generally be housed in 4-bedroom apartments. Every participant will have their own bedroom and bathrooms, kitchen, and living room will be shared. The SIHP will take care of room assignments. As much as possible, you will live with other students from this site but there may be one or two apartments that are mixed with summer students from other programs.

Mon, 01 Jan 0001 00:00:00 +0000

About our REU site

The Emory Research Experience for Undergraduates site focuses on computational mathematics and its applications in data science, which impacts nearly every field of science, industry, and society. Despite its growing importance, the number of academic training opportunities in data science has not kept pace with the rapid growth in demand from private and public entities.

We have been running this program since the summer of 2021. Between 2021 and 2023, the site was held in conjunction with the Emory Research Experience for Teacher site and we hosted about a dozen in-service high school teachers.

Our site emphasizes developing research and professional skills that enable participants to understand, conduct, and effectively communicate research in this booming area. Each summer, our site trains at least twelve undergraduates for eight weeks. The participants are mentored by faculty members from the Department of Mathematics and the Department of Computer Science.

We recruit students nationwide.

How does it work?

Our REU site introduces participants to the mathematical theory and computational tools used in applications. Our projects include various topics ranging from data assimilation to machine learning. Guided by faculty mentors, undergraduate students work toward creating new mathematical insights and designing practical solutions.

Our site’s activities will be centered around a common theme that differs each year:

2024: Learning from Images
2025: Combining Models and Data
2026: Discovering new Mathematics with Computational and Machine Learning Methods

Within each theme, faculty mentors will pose at least four research problems tackled by one student team.

New insights of relevance to the broader scientific community will be created and disseminated in several ways, for example,

oral presentations
poster presentations
student/teacher-authored publications
open source software
blog posts / project websites
course materials for classroom-use

What pre-requisites are needed?

By definition, the research projects will take participants beyond standard coursework. To fill the gap, our site’s educational component introduces the participants to a range of mathematical techniques, including machine learning, deep neural networks, numerical linear algebra, optimization, partial differential equations, and statistics. The faculty mentors also provide their mentees with professional and computational skills, including scientific writing, oral and poster presentations, and cloud computing. The weekly seminar will feature group activities and faculty-led presentations on data and ethics, algorithmic bias, public scholarship.

Information for Applicants

Interested in joining? Learn more about about requirements, deadlines, and application materials on the information for applicants page.

Mon, 01 Jan 0001 00:00:00 +0000

Contents: Emory CMDS REU/RET Website

This directory and its subdirectories contain the content that appears on our website. Each subdirectory corresponds to a page on the website. Below we give a quick overview over the folders that you might want to edit.

home

This folder contains the elements of our main website. It mostly draws content from other places (such as featured projects) but also contains some text that can be edited in welcome.md

authors

The subdirectories in this folder contain information about the people involved on our site. If you see a folder with your name, you can maintain your picture and bio here. If you are new or do not have a folder already, you can copy one existing folder that corresponds to your role (e.g, mentor, student, teacher, …)

projects

This folder contains the website for the individual projects. The participants will have to edit their folder and can add images, videos, PDFs, etc.

People

Mon, 01 Jan 0001 00:00:00 +0000

Summer 2021

Mon, 01 Jan 0001 00:00:00 +0000

Summer 2022

Mon, 01 Jan 0001 00:00:00 +0000

Summer 2023

Mon, 01 Jan 0001 00:00:00 +0000

Summer 2024

Mon, 01 Jan 0001 00:00:00 +0000

Summer 2025

Mon, 01 Jan 0001 00:00:00 +0000

Summer 2026

Mon, 01 Jan 0001 00:00:00 +0000