Proximal MCMC is a recently proposed class of MCMC methods which uses proximity maps instead of gradients to build proposal mechanisms which can be employed for both differentiable and non-differentiable targets.

These methods have been shown to be stable for a wide class of targets, making them a valuable alternative to Metropolis-adjusted Langevin algorithms (MALA); and have found wide application in imaging contexts.

The wider stability properties are obtained by building the Moreau-Yoshida envelope for the target of interest, which depends on a parameter .

In this work we investigate the optimal scaling problem for proximal MCMC, show that MALA is a special case of this class of algorithms and provide practical guidelines for the selection of the scale parameter and the parameter .

Richard EverittLink opens in a new window

Paul JenkinsLink opens in a new window

Gareth RobertsLink opens in a new window

Massimiliano TamborrinoLink opens in a new window

Jeremie HoussineauLink opens in a new window

Giovanni MontanaLink opens in a new window

Simon SpencerLink opens in a new window

Nicholas TawnLink opens in a new window

In a modern observational study based on healthcare databases, the number of observations and of predictors typically range in the order of 10^5 ~ 10^6 and of 10^4 ~ 10^5. Despite the large sample size, data rarely provide sufficient information to reliably estimate such a large number of parameters. Sparse regression techniques provide potential solutions, one notable approach being the Bayesian methods based on shrinkage priors. In the "large n & large p" setting, however, posterior computation encounters a major bottleneck at repeated sampling from a high-dimensional Gaussian distribution, whose precision matrix $\Phi$ is expensive to compute and factorize. In this talk, I will present a novel algorithm to speed up this bottleneck based on the following observation: we can cheaply generate a random vector $b$ such that the solution to the linear system $\Phi \beta = b$ has the desired Gaussian distribution. We can then solve the linear system by the conjugate gradient (CG) algorithm through matrix-vector multiplications by $\Phi$; this involves no explicit factorization or calculation of $\Phi$ itself. Rapid convergence of CG in this context is guaranteed by the theory of prior-preconditioning we develop. We apply our algorithm to a clinically relevant large-scale observational study with n = 72,489 patients and p = 22,175 clinical covariates, designed to assess the relative risk of adverse events from two alternative blood anti-coagulants. Our algorithm demonstrates an order of magnitude speed-up in posterior inference, in our case cutting the computation time from two weeks to less than a day.

Jon ForsterLink opens in a new window

Adam JohansenLink opens in a new window

Martyn PlummerLink opens in a new window

Mark SteelLink opens in a new window

Ioannis KosmidisLink opens in a new window

Over the past two decades, a wave of Bayesian explanations has swept through cognitive science, explaining behaviour in domains from intuitive physics and causal learning, to perception, motor control and language. Yet people produce stunningly incorrect answers in response to even the simplest questions about probabilities. How can a supposedly Bayesian brain paradoxically reason so poorly with probabilities?

We propose that the brain approximates Bayesian inference through local sampling, approximating the solutions to inferential problems in similar ways to how MCMC algorithms operate. In a recent experiment, we explicitly asked people to generate a sequence of random samples, and found that human sequences are different to iid sequences in very predictable ways. We compared human performance to that of multiple MCMC algorithms, and found that people are most similar to an MCMC algorithm with autocorrelated proposals.

A universal kernel whose sections approximate any causal and time-invariant filter in the fading memory category with inputs and outputs in a finite-dimensional Euclidean space is constructed. This kernel is built using the reservoir functional associated with a state-space representation of the Volterra series expansion available for any analytic fading memory filter. We call it the Volterra reservoir kernel. Even though the state-space representation and the corresponding reservoir feature map are defined on an infinite dimensional tensor algebra space, the kernel map is characterized by explicit recursions that are readily computable for specific datasets when employed in estimation problems using the representer theorem. We showcase the application of the reservoir kernel in two empirical exercises, namely, Mackey-Glass time series forecasting and bitcoin price prediction. We compare the performance of the reservoir kernel to the sigkernel in Salvi et al. and other kernel benchmarks therein.

Consider a stochastic process evolving on a directed acyclic graph. Observations are at the leaf vertices and interest lies in smoothing and parameter estimation.
I will discuss the {\it Backward Filtering Forward Guiding} (BFFG) algorithm. In this algorithm one first computes the Backward Information Filter (BIF) to a simplified version of the forward dynamics. The output of the BIF can subsequently be used to adjust the dynamics of the forward process such that it ends up in the observations at the leaf-vertices. This adjustment is an instance of Doob's $h$-transform. We call the adjusted forward process a {\it guided process}, as it guides the process to the observations. One can deduce closed-form expressions for the likelihood ratio between the true conditioned process and the guided process. This in turn can be exploited for designing Bayesian computational methods. In particular I will show that likelihood based inference can be done in a unified way for seemingly different models by defining two key operations and their composition.

This concerns joint work with Moritz Schauer (Chalmers University of Technology - University of Gothenburg). The BFFG-algorithm for diffusions is discussed in the paperrMider, Schauer & Van der Meulen (2021), while generalisations to graphical models and other stochastic processes are treated in Van derMeulen & Schauer (2022). An accessible introduction to that paper is given in Van der Meulen (2022).

M. Mider, M. Schauer and F.H. van der Meulen (2021), Continuous-discrete smoothing of diffusions, Electronic Journal of Statistics 15, 4295-4342. F.H. van der Meulen and M. Schauer (2021), Automatic Backward Filtering Forward Guiding for Markov processes and graphical models, arXiv:2010.03509 (submitted). F.H. van der Meulen (2022), Introduction to Automatic Backward Filtering Forward Guiding, arXiv:2203.04155.