Optimization in the space of probability distributions with applications in statistics
Yao, Rentian
This item is only available for download by members of the University of Illinois community. Students, faculty, and staff at the U of I may log in with your NetID and password to view the item. If you are trying to access an Illinois-restricted dissertation or thesis, you can request a copy through your library's Inter-Library Loan office or purchase a copy directly from ProQuest.
Permalink
https://hdl.handle.net/2142/125655
Description
Title
Optimization in the space of probability distributions with applications in statistics
Author(s)
Yao, Rentian
Issue Date
2024-05-16
Director of Research (if dissertation) or Advisor (if thesis)
Yang, Yun
Chen, Xiaohui
Doctoral Committee Chair(s)
Yang, Yun
Committee Member(s)
Shao, Xiaofeng
Liu, Jingbo
Department of Study
Statistics
Discipline
Statistics
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
Ph.D.
Degree Level
Dissertation
Keyword(s)
Optimal Transport
Bayesian Statistics
Interacting Particle Systems
Optimization
Language
eng
Abstract
Many problems in statistics can be formulated as minimizing a functional over the space of all probability distributions on a (parameter) space. Examples include approximate Bayesian computation, non-parametric estimation, and single-cell analysis in mathematical biology. This thesis aims to address the following statistical problems by developing computationally tractable algorithms with theoretical foundations: 1. In the first part, we introduce a general computational framework to implement mean-field (MF) variational inference (VI) for Bayesian models using the Wasserstein gradient flow (WGF), a modern mathematical technique for realizing a gradient flow over the space of probability distributions. We prove the statistical guarantee of MFVI and the algorithmic convergence of using WGF under mild conditions. To implement the algorithm, we propose a new constraint-free function approximation method using neural networks. 2. In the second part, we investigate the nonparametric estimation problem of estimating the distribution-state dependent drift parameter of a Mckean--Vlasov equation with a constant diffusion parameter. This equation can be treated as the stochastic differential equation of which the density evolution equation is the gradient flow to minimize a specific energy functional. 3. In the third part, we concern the optimization of a displacement convex functional over multiple distributions. We derive an exponential algorithmic convergence rate under the quadratic growth (QG) condition on the objective functional, and a slower polynomial rate in the absence of the QG condition. By applying our theory to MFVI, the dependence of the algorithmic convergence rate on the number of blocks is better compared with the result in the first project. 4. In the last part, we explore the problem of convex optimization over the space of all probability distributions. We introduce an implicit scheme for discretizing a continuous-time gradient flow relative to the Kullback--Leibler (KL) divergence. We derive an explicit algorithmic convergence rate and apply our algorithm to compute non-parametric likelihood estimators and Bayesian posterior distributions.
Use this login method if you
don't
have an
@illinois.edu
email address.
(Oops, I do have one)
IDEALS migrated to a new platform on June 23, 2022. If you created
your account prior to this date, you will have to reset your password
using the forgot-password link below.
