MATH Seminar

Title: Decentralized consensus optimization on networks with delayed and stochastic gradients
Seminar: Numerical Analysis and Scientific Computing
Speaker: Xiaojing Ye of Georgia State University
Contact: Lars Ruthotto, lruthotto@emory.edu
Date: 2018-11-02 at 2:00PM
Venue: MSC N302
Download Flyer
Abstract: Decentralized consensus optimization has extensive applications in many emerging big data, machine learning, and sensor network problems. In decentralized computing, nodes in a network privately hold parts of the objective function and need to collaboratively solve for the consensual optimal solution of the total objective, while they can only communicate with their immediate neighbors during updates. In real-world networks, it is often difficult and sometimes impossible to synchronize these nodes, and as a result they have to use stale and stochastic gradient information which may steer their iterates away from the optimal solution. In this talk, we focus on a decentralized consensus algorithm by taking the delays of gradients into consideration. We show that, as long as the random delays are bounded in expectation and a proper diminishing step size policy is employed, the iterates generated by this algorithm still converge to a consensual optimal solution. Convergence rates of both objective and consensus are derived. Numerical results on some synthetic optimization problems and on real seismic tomography will also be presented.