Nick Tawn - Improving the Efficiency of the Parallel Tempering Algorithm
Bayesian inference typically requires MCMC methods to evaluate samples from the posterior, however it is important that the MCMC procedure employed samples ‘correctly’ from the distribution for the sample estimates to be valid. For instance, if the posterior distribution was multi-modal then by running an MCMC procedure for only a finite number of runs it is possible that the chain can become trapped and not explore the entire state space. Well known algorithms to aid mixing in multimodal settings are the Parallel and Simulated tempering algorithms. I will introduce these and demonstrate their powers and weaknesses for sampling in such situations, and then describe the way in which these algorithms can be set up to achieve optimal efficiency when sampling. The key feature of these algorithms is the ability to share information from the mixing in the hotter states to aid the mixing of the chain in the colder states. I will also present a new approach based on reparameterisation that could potentially enhance the algorithms’ efficiency. Empirical evidence is illustrated to show that this new algorithm appears to vastly enhance the trade of mixing information between temperature levels when targeting certain posterior distributions.