Topics in approximate inference

What is approximate inference? more

Why I should care about it? more

What this list is about? more

(Posts on specific topics coming 🙂 )

Theme I: algorithms for fitting approximate distributions:

  1. variational inference (VI);
  2. message passing, belief/expectation propagation;
  3. black-box/Monte Carlo approximation techniques;
  4. reparameterisation trick and control variates;
  5. lower-bounds and upper-bounds;
  6. optimisation meets VI (proximal gradient, trust-region method, etc);
  7. new methods using density ratio estimation, integral probability metrics, etc.

Theme II: approximate distribution design:

  1. invertible transform;
  2. non-parametric approximations (e.g. empirical distributions in sampling);
  3. latent variable models as approximate distributions;
  4. perturbations/corrections after fitting a simple approximation;
  5. bridging VI and MCMC.

Theme III: applications:

  1. learning generative models/latent variable models;
  2. Bayesian neural networks;
  3. Gaussian processes;
  4. probabilistic graphical models.