Variational Formulations and Distributed Convex Optimization Methods for Modern Data Science Applications

The need to reason about uncertainty in large, complex, and multi-modal datasets has become increasingly common. The ability to transform samples from one distribution P to another distribution Q enables the solution to many problems in data science and machine learning. Performing provably good transformations, in general, still comprises computational difficulties, especially in high dimensions. Here, we consider the problem of computing such "measure transport maps" efficiently and at scale. Under the mild assumptions that P need not be known but can be sampled from, that the density of Q is known up to a proportionality constant, and that Q is log-concave, we provide a convex optimization formulation.  This approach, involving sequentially solving Wasserstein regularized relative entropy minimization problems, is inspired by variational formulations in non-equilibrium thermodynamics and allows for sequential construction of transport maps with exponential convergence in relative entropy.   We provide an empirical risk minimization formulation using maps parametrized by polynomial chaos along with a distributed and convergent algorithm that has been efficiently implemented in GPUs and Amazon Web Services architectures.   We provide examples of this framework, within the context of Bayesian inference, active learning for brain-machine interfaces, reinforcement learning, and generative modeling.