|Title: Regression with Tropical Rational Functions and Application to Neural Network Initialization|
|Speaker: Alex Dunbar of Emory University|
|Contact: Lars Ruthotto, firstname.lastname@example.org|
|Date: 2022-11-03 at 10:00AM|
|Venue: MSC W301|
The tropical semiring and its associated polynomial and rational functions provide an algebraic framework for understanding continuous piecewise linear functions. We propose an alternating minimization heuristic for regression over the space of tropical rational functions. The method alternates between fitting the numerator and denominator via tropical polynomial regression, which is known to admit a closed-form solution. Our work is motivated by applications to ReLU neural network training. ReLU neural networks are a popular class of network architectures in the machine learning community which have recently been connected to tropical rational functions. We present experiments demonstrating the behavior of the alternating minimization method. Additionally, we present preliminary experiments which leverage tropical rational regression to initialize weights in ReLU neural networks and discuss geometric aspects of the network initialization problem.
See All Seminars