Probability Seminars——A random perturbation approach to some stochastic approximation algorithms in optimization
主 题: Probability Seminars——A random perturbation approach to some stochastic approximation algorithms in optimization
报告人: Professor Wenqing Hu (Missouri University of Science and Technology)
时 间: 2018-06-11 15:00-16:00
地 点: Room 1418, Sciences Building No. 1
Abstract: Many large-scale learning problems in modern statistics and machine learning can be reduced to solving stochastic optimization problems, i.e., the search for (local) minimum points of the expectation of an objective random function (loss function). These optimization problems are usually solved by certain stochastic approximation algorithms, which are recursive update rules with random inputs in each iteration. In this talk, we will be considering various types of such stochastic approximation algorithms, including the stochastic gradient descent, the stochastic composite gradient descent, as well as the stochastic heavy-ball method. By introducing approximating diffusion processes to the discrete recursive schemes, we will analyze the convergence of the diffusion limits to these algorithms via delicate techniques in stochastic analysis and asymptotic methods, in particular via random perturbations of dynamical systems. This talk is based on a series of joint works with Chris Junchi Li (Princeton), Weijie Su (UPenn) and Haoyi Xiong (formerly at Missouri S&T, now at Baidu).
