Fast sampling at phase transitions in statistical physics

Sampling algorithms are commonplace in statistics and machine learning – in particular, in Bayesian computation – and have been used for decades to enable inference, prediction and model comparison in many different settings. They are also widely used in statistical physics, where many popular sampling algorithms first originated [1, 2]. At a high level, the goals within each discipline are the same – to sample from and approximate statistical expectations with respect to some probability distribution – but the motivations, nomenclature and methods of explanation differ significantly. This has led to challenges in communicating between the fields, and indeed the fundamental goals of one field are often misunderstood in the other. We begin this talk with the basic framework of statistical physics, comparing and contrasting with Bayesian computation — in particular, emphasising that probability models are studied as functions of thermodynamic (hyper)parameters such as the temperature. This is particularly useful for characterising phase transitions, ie, boundaries in thermodynamic-parameter space between distinct thermodynamic phases.

We then move on to sampling algorithms, with a particular focus on the behaviour of the Metropolis algorithm [1] when simulating the 2D Ising and 2DXY models of magnetism. Metropolis dynamics are metastable in the low-temperature phase of each model, displaying asymptotically slow (ie, with dimensionality) mixing between states of equal probability density. Moreover, the Metropolis algorithm also suffers from the closely related phenomenon of critical slowing down at phase transitions. These correlated dynamics are characterised by asymptotically long integrated autocorrelation times, due to a flattening of the target density that essentially results from the system trying to exist simultaneously in both thermodynamic phases. Indeed, these key aspects of statistical physics have led to innovations in sampling algorithms that inform the Bayesian world. In particular, we present the Swendsen—Wang [3], Wolff [4] and event-chain Monte Carlo [5-7] algorithms. The first two simulate the 2D Ising model and were developed in response to the metastability and critical slowing down of the Metropolis algorithm. They circumvent both phenomena to mix with low autocorrelation and independent of dimensionality. We then show that event-chain Monte Carlo similarly circumvents the low-temperature Metropolis metastability of the 2DXY model [7] and discuss its potential utility in bypassing an hypothesised critical slowing down at the phase transition. This talk is based on a recent review paper on the subject [8].

[1] Metropolis et al., J. Chem. Phys. 21 1087 (1953)

[2] Alder & Wainwright, J. Chem. Phys. 27 1208 (1957)

[3] Swendsen & Wang, Phys. Rev. Lett. 58 86 (1987)

[4] Wolff, Phys. Rev. Lett. 62 361 (1989)

[5] Bernard, Krauth & Wilson, Phys. Rev. E 80 056704 (2009)

[6] Michel, Mayer & Krauth, EPL (Europhys. Lett.) 112 20003 (2015)

[7] Faulkner, arXiv:2209.03699 (2022)

[8] Faulkner & Livingstone, arXiv:2209.03699 (2022)

Michael Faulkner is an Assistant Professor at the Warwick Centre for Predictive Modelling. He’s primarily a computational statistical physicist, specialising in:

• Emergent electrostatics, metastability and correlated dynamics in systems that experience the Berezinskii-Kosterlitz-Thouless phase transition, eg, certain planar magnets, superfluids and superconductors.
• Molecular simulation in soft-matter physics, with a focus on electrostatics, high precision and numerical stability.
• Monte Carlo sampling algorithms in statistical physics and Bayesian computational statistics, with a particular interest in piecewise deterministic Markov processes such as event-chain Monte Carlo.

His academic career started as a PhD student at University College London and Ecole normale supérieure de Lyon from 2011 to 2015, under the co-supervision of Steve Bramwell and Peter Holdsworth. After a short postdoc and teaching position at Bristol Mathematics, he then moved to Bristol Physics in August 2017 after winning an EPSRC postdoctoral research fellowship. He was also a visiting scientist at Ecole normale supérieure from September 2017 to October 2018, and won a Max Planck Institute fellowship to visit the Max Planck Institute for the Physics of Complex Systems in Dresden in April 2018.