Topics in approximate inference

What is approximate inference? more

Why I should care about it? more

What this list is about? more

(Posts on specific topics coming 🙂 )

Front matter

Theme I: algorithms for fitting approximate distributions:

  1. variational inference (VI);
  2. Monte Carlo VI and gradient estimators;
  3. lower-bounds and upper-bounds;
  4. message passing, belief/expectation propagation;
  5. optimisation meets VI (proximal gradient, trust-region method, etc);
  6. wild approximate inference (with implicit q distributions).

Theme II: approximate distribution design:

  1. invertible transform;
  2. non-parametric approximations (e.g. empirical distributions in sampling);
  3. latent variable models as approximate distributions;
  4. perturbations/corrections after fitting a simple approximation;
  5. bridging VI and MCMC.

Theme III: applications:

  1. learning generative models/latent variable models;
  2. Bayesian neural networks;
  3. Gaussian processes;
  4. probabilistic graphical models.