In recent years, reinforcement learning (RL) methods have shown success in solving optimal control problems involving ordinary differential equations. RL approaches differ drastically from traditional optimal control methods. Where prior methods have relied on the ODE models, RL tries to learn the optimal control simply by observing actions and rewards. There is tremendous activity in the sciences and engineering utilizing both methods. However, RL and optimal control theory is rarely compared on the same problem. This REU project fills that gap and tries to investigate the trade-off between data-driven RL methods and model-driven optimal control methods on the continuous mountain car problem. We found that RL is not always the best, in our case we found that the OC method gave us a better result on the Continuous Mountain Car Problem.

Numerical models of coastal hydrodynamics play a vital role in understanding hurricane storm surges, particularly as the climate changes. However, uncertainties in model parameters and their representations, e.g., bottom stress and surface wind stress, limit the confidence of model results. Data is becoming increasingly available but also contains uncertainties. In this project we will explore methods of data assimilation, which utilize information about the uncertainties in both modeled and observed data to improve estimates of the coastal system state.

Implicit networks are a special type of architecture whose outputs are defined by a fixed point (or optimality) condition. To evaluate these networks, one performs an iterative process, where each iteration is considered a layer of the network. The depth of these networks often vary depending on the complexity of the input data; for instance, in natural language processing, it might take 3 iterations (or layers) to output the sentiment of a simple sentence, but it might take 100 layers for the network to output the sentiment of a complicated sentence. Unfortunately, training implicit networks efficiently typically comes at additional computational cost. This project explores fast and efficient algorithms for training implicit networks, with emphasis on their applications to inverse problems.

Assume you are given observations of a dynamical system at a few time points and want to learn its underlying ODE. Problems like this are abundant in scientific applications and there are several machine learning approaches to this problem. One approach with soaring popularity is called Neural ODEs, that is, ODEs whose dynamics are trainable Neural Networks, which makes them very flexible. There are a few different Neural ODE methods already developed, and our main goal is to extend two specific methods, taking elements from each and adding our own approaches to develop a new method.

Since numbers in the computer are represented with a fixed number of bits, loss of accuracy during calculation is unavoidable. At high precision where more bits (e.g. 64) are allocated to each number, round-off errors are typically small. On the other hand, calculating at lower precision, such as half (16 bits), has the advantage of being much faster. This research focuses on experimenting with arithmetic at different precision levels for large-scale inverse problems, which are represented by linear systems with ill-conditioned matrices. We modified the Conjugate Gradient Method for Least Squares (CGLS) and the Chebyshev Semi-Iterative Method (CS) with Tikhonov regularization to do arithmetic at lower precision using the MATLAB \textbf{chop} function, and we ran experiments on applications from image processing and compared their performance at different precision levels. We concluded that CGLS is a more stable algorithm, but overflows easily due to the computation of inner products, while CS is less likely to overflow but it has more erratic convergence behavior. When the noise level is high, CS outperforms CGLS by being able to run more iterations before overflow occurs; when the noise level is close to zero, CS appears to be more susceptible to accumulation of round-off errors.

The human brain sends signals using around 100 billion neurons connected in a complex and dynamic fashion. The sheer number of neurons makes the task of modeling, particularly at the network level, difficult. Instead of modeling each individual neuron, one can instead approximate the system using a series of coupled ordinary or delay differential equations. Using bifurcation theory, this approximate system can then be manipulated to mimic neuronal level changes that lead to neuronal dysfunction, disease states, and, ultimately, possible trajectories in parameter space that lead back to a healthy neuronal state. These types of models help add to the understanding of neuronal dysfunction but are pretty useless without data to validate the model. Fortunately, a rich dataset of in-vivo primate neuronal data recorded throughout various movement-related brain areas is available to help validate and corroborate findings for this project.

Human brains are complex organs with structures, functions, and mechanisms that are still largely unknown to us. Modern neuroscience research aims to help us better understand them. Some recent studies have agreed that interactions among brain regions are related to neural development and mental disorders, and modeling these interactions is a way to gain further insight into how brain regions, neural activity, and disease interact with each other. It is unclear what kind of mathematical models will be most useful for the task of modeling neural activity, so mathematicians with interests in this field are building and testing mathematical models that could progress neuroscience research further. In this project, we explore and analyze different approaches for modeling brain networks, ranging from traditional shallow graph models to modern deep graph neural networks. The goal of these models is to aid in the analysis of mental disorders and diseases such as post-traumatic stress disorder (PTSD), bipolar disorder, depression, and human immunodeficiency virus (HIV). We aim to harness modern computational methods to improve the accuracy of pre-existing models, especially ones that aim to predict whether a patient is diseased or healthy based on brain scan data. We adapt different graph mining techniques for brain networks, statistically and visually analyze the results, and evaluate each model’s classification performance.