Statistical Learning & Inference Seminars
The seminars will take place every Tuesdays 11am.
Term 1, 24-25
Date, Time and Room |
Speaker |
Title |
|||||||||||
01/10, 11am, MB2.22 |
Dr. Cornelius Fritz (Trinity College Dublin) | A Regression Framework for Studying Relationships among Attributes under Network Interference | |||||||||||
Abstract: |
To understand how the interconnected and interdependent world of the twenty-first century operates and make model-based predictions, joint probability models for networks and interdependent outcomes are needed. We propose a comprehensive regression framework for networks and interdependent outcomes with multiple advantages, including interpretability, scalability, and provable theoretical guarantees. The regression framework can be used for studying relationships among attributes of connected units and captures complex dependencies among connections and attributes, while retaining the virtues of linear regression, logistic regression, and other regression models by being interpretable and widely applicable. On the computational side, we show that the regression framework is amenable to scalable statistical computing based on convex optimization of pseudo-likelihoods using minorization-maximization methods. On the theoretical side, we establish convergence rates for pseudo-likelihood estimators based on a single observation of dependent connections and attributes. We demonstrate the regression framework using simulations and an application to hate speech on the social media platform X in the six months preceding the insurrection at the U.S. Capitol on January 6, 2021. | ||||||||||||
08/10, 11am, MB2.22 |
Low-rank models for dynamic multiplex graphs and multivariate time series |
||||||||||||
Abstract: |
This talk discusses low-rank models for two different types of data structures: dynamic multiplex graphs and panels of multivariate time series. The first part of the talk will present a doubly unfolded adjacency spectral embedding (DUASE) method for networks evolving over time, with different edge types, commonly known as multiplex networks. Statistical properties of DUASE will be discussed, and links with commonly used statistical models for clustering graphs will be presented. The second part of the talk will cover the case of a panel of multivariate time series where there is co-movement between the panel components, modelled via a vector autoregressive process. A Network Informed Restricted Vector Auto-Regressive (NIRVAR) process is proposed, with an algorithm that gives a low dimensional latent embedding of each component of the panel. Clustering in this latent space is then used to recover the non-zero entries of the VAR coefficient matrix. The proposed model outperforms alternative approaches in terms of prediction and inference in simulation studies and real-data examples in applications in finance and transportation systems. |
||||||||||||
15/10, 11am, MB2.22 |
Dr. Tom Rainforth (Oxford) | Modern Bayesian Experimental Design | |||||||||||
Abstract: |
Bayesian experimental design (BED) provides a powerful and general framework for optimizing the design of experiments. However, its deployment often poses substantial computational challenges that can undermine its practical use. In this talk, I will outline how recent advances have transformed our ability to overcome these challenges and thus utilize BED effectively, before discussing some key areas for future development in the field. Related review paper: https://arxiv.org/abs/2302.14545 | ||||||||||||
22/10, 11am, MB2.22 |
Dr. Robin Mitra (UCL) | Using saturated count models for user-friendly synthesis of categorical data | |||||||||||
Abstract: |
Synthetic data methods are being increasingly used to protect data confidentiality. Large sparse categorical data sets pose some significant challenges for synthesis which makes many traditional methods unsuitable. We explore using saturated count models for synthesis. These are appealing as they allow large categorical data sets to be synthesized quickly and conveniently, as well as permitting risk and utility metrics to be satisfied a priori, that is, prior to synthetic data generation. Most well-known count models for synthesizing categorical data at the tabular level tend to utilise either Poisson or Poisson-mixture distributions. However, the latter are always over-dispersed, with a variance that is an increasing function of the mean. As a result, relatively more noise is applied to larger counts than smaller counts. But this is contrary to the objective of data synthesis, where larger counts are typically lower risk than smaller counts, and therefore require less perturbation. We thus additionally explore the benefits of using the discretized gamma family distribution (DGAF) for synthesis within the saturated model framework. The DGAF provides the synthesizer with control of the variance-mean relationship, allowing smaller counts to be over-dispersed and larger counts to be under-dispersed, which in turn produces synthetic data with greater utility. The benefits of the DGAF are illustrated empirically using a database which can be viewed as a good substitute to the English School Census. | ||||||||||||
29/10, 11am, MB2.22 |
Dr. Song Liu (Bristol) | High-Dimensional Differential Parameter Inference in Exponential Family using Time Score Matching | |||||||||||
Abstract: |
This paper addresses differential inference in time-varying parametric probabilistic models, like graphical models with changing structures. Instead of estimating a high-dimensional model at each time and inferring changes later, we directly learn the differential parameter, i.e., the time derivative of the parameter. The main idea is treating the time score function of an exponential family model as a linear model of the differential parameter for direct estimation. We use time score matching to estimate parameter derivatives. We prove the consistency of a regularized score matching objective and demonstrate the finite-sample normality of a debiased estimator in high-dimensional settings. Our methodology effectively infers differential structures in high-dimensional graphical models, verified on simulated and real-world datasets. | ||||||||||||
05/11, 11am, MB2.22 |
Detecting Changes in Production Frontier | ||||||||||||
Abstract: |
In this talk, we first give a brief review of the nonparametric estimation problem of production frontier function, which concerns the maximum possible output given input levels and the efficiency of the firms. We then look at how (potentially) multiple changes over time in the production frontier can be detected. By assuming that the frontier always shifts upwards over time, which is plausible thanks to the advance in technologies, we can detect changes in the frontier at the near-optimal rate under regularity conditions, irrelevant of the dimensionality of the input. This can be achieved by modifying and utilising the well-known Free Disposal Hull (FDH) or Data Envelopment Analysis (DEA) algorithm in different ways, depending on whether the shift is global or local. Finally, we also discuss how the confidence intervals can be constructed in this setup. |
||||||||||||
12/11, 11am, MB2.22 |
Dr. Ed Cohen (Imperial College London)Link opens in a new window | Analysing spatial point patterns on the surface of 3D shapes | |||||||||||
Abstract: |
Statistical methodology for analysing spatial point patterns has traditionally focused on Euclidean data and planar surfaces. However, with recent advances in 3D biological imaging technologies targeting protein molecules on a cellâ€™s plasma membrane, spatial point patterns are now being observed on complex shapes and manifolds whose geometry must be respected for principled inference. Consequently, there is now a demand for tools that can analyse these data for important scientific studies in cellular and micro-biology. Motivated by studying the spatial distribution of LPS proteins on the surface of E-Coli, we develop the fundamental functional summary statistics for the analysis of point patterns to general convex bounded shapes and demonstrate how they can be used to test for complete spatial randomness. We then develop their multi-type extensions, together with a test for independence of the component marginal processes. To support these methods, we introduce a plug-in estimator for the intensity of a spatial point process on a manifold. We conclude with a discussion on how these methods can readily be extended to a class of non-convex shapes.
References: S. Ward, E.A.K. Cohen, N. M. Adams. Testing for complete spatial randomness on 3-dimensional bounded convex shapes. Spatial Statistics, Vol. 41, 2021. S. Ward, H. S. Battey and E. A. K. Cohen. Nonparametric estimation of the intensity function of a spatial point process on a Riemannian manifold. Biometrika, Vol. 110, 2023. S. Ward, E. A. K. Cohen and N. M. Adams. Functional summary statistics and testing for independence in marked point patterns on the surface of three-dimensional convex shapes. arXiv:2410.01063, 2024. |
||||||||||||
19/11, 11am, MB2.22 |
Prof. Thomas Nichols (Oxford) | ||||||||||||
Abstract: |
|||||||||||||
26/11, 11am, MB2.22 |
Prof. Qiwei Yao (LSE) | ||||||||||||
Abstract: |
|||||||||||||
03/12, 11am, MB2.22 |
Prof. Karthik Bharath (Nottingham) | ||||||||||||
Abstract: |
Term 3, 23-24
Date, Time and Room |
Speaker |
Title | |
30/04, 11am, MS.02 |
Oliver Feng (Bath) |
||
Abstract: |
In the context of linear regression, we construct a data-driven convex loss function with respect to which empirical risk minimisation yields optimal asymptotic variance in the downstream estimation of the regression coefficients. Our semiparametric approach targets the best decreasing approximation of the derivative of the log-density of the noise distribution. At the population level, this fitting process is a nonparametric extension of score matching, corresponding to a log-concave projection of the noise distribution with respect to the Fisher divergence. The procedure is computationally efficient, and we prove that our procedure attains the minimal asymptotic covariance among all convex M-estimators. As an example of a non-log-concave setting, for Cauchy errors, the optimal convex loss function is Huber-like, and our procedure yields an asymptotic efficiency greater than 0.87 relative to the oracle maximum likelihood estimator of the regression coefficients that uses knowledge of this error distribution; in this sense, we obtain robustness without sacrificing much efficiency. Numerical experiments confirm the practical merits of our proposal. This is joint work with Yu-Chun Kao, Min Xu and Richard Samworth. |
||
14/05, 11am, MS.01 |
Gilles Stupfler (University of Angers) |
Some new perspectives on extremal regression | |
Abstract: |
The objective of extremal regression is to estimate and infer quantities describing the tail of a conditional distribution. Examples of such quantities include quantiles and expectiles, and the regression version of the Expected Shortfall. Traditional regression estimators at the tails typically suffer from instability and inconsistency due to data sparseness, especially when the underlying conditional distributions are heavy-tailed. Existing approaches to extremal regression in the heavy-tailed case fall into two main categories: linear quantile regression approaches and, at the opposite, nonparametric approaches. They are also typically restricted to i.i.d. data-generating processes. I will here give an overview of a recent series of papers that discuss extremal regression methods in location-scale regression models (containing linear regression quantile models) and nonparametric regression models. Some key novel results include a general toolbox for extreme value estimation in the presence of random errors and joint asymptotic normality results for nonparametric extreme conditional quantile estimators constructed upon strongly mixing data. Joint work with A. Daouia, S. Girard, M. Oesting and A. Usseglio-Carleve. | ||
04/06, 11am, MB0.07 |
Rebecca Lewis (Oxford) |
High-dimensional logistic regression with separated data | |
Abstract: |
In a logistic regression model with separated data, the log-likelihood function asymptotes and the maximum likelihood estimator does not exist. We show that an exact analysis for each regression coefficient always produces half-infinite confidence sets for some parameters when the data are separable. Such conclusions are not vacuous, but an honest portrayal of the limitations of the data. Finite confidence sets are only achievable when additional, perhaps implicit, assumptions are made. In a high-dimensional regime, we consider the implications of enforcing a natural constraint on the vector of logistic-transformed probabilities. We derive a consistent estimator of the unknown logistic regression parameter that exists even when the data are separable. | ||
18/06, 1pm, MB0.07 |
Jenny Wadsworth (Lancaster) |
||
Abstract: |
A geometric representation for multivariate extremes, based on the shapes of scaled sample clouds in light-tailed margins and their so-called limit sets, has recently been shown to connect several existing extremal dependence concepts. However, these results are purely probabilistic, and the geometric approach itself has not been fully exploited for statistical inference. We outline a method for parametric estimation of the limit set shape, which includes a useful non-/semi-parametric estimate as a pre-processing step. More fundamentally, our approach provides a new class of asymptotically motivated statistical models for the tails of multivariate distributions, and such models can accommodate any combination of simultaneous or non-simultaneous extremes through appropriate parametric forms for the limit set shape. In this talk we will also present ongoing work moving towards semiparametric methodology for greater flexibility. Extrapolation further into the tail of the distribution is possible via simulation from the fitted model, and probability estimates are possible in regions where other frameworks struggle. Joint work with Ryan Campbell. | ||
25/06, 11am, MB0.07 | Nicola Gnecco (UCL) | Extremal Random Forests | |
Abstract: |
Classical methods for quantile regression fail in cases where the quantile of interest is extreme and only few or no training data points exceed it. Asymptotic results from extreme value theory can be used to extrapolate beyond the range of the data, and several approaches exist that use linear regression, kernel methods or generalized additive models. Most of these methods break down if the predictor space has more than a few dimensions or if the regression function of extreme quantiles is complex. We propose a method for extreme quantile regression that combines the flexibility of random forests with the theory of extrapolation. Our extremal random forest (ERF) estimates the parameters of a generalized Pareto distribution, conditional on the predictor vector, by maximizing a local likelihood with weights extracted from a quantile random forest. We penalize the shape parameter in this likelihood to regularize its variability in the predictor space. Under general domain of attraction conditions, we show consistency of the estimated parameters in both the unpenalized and penalized case. Simulation studies show that our ERF outperforms both classical quantile regression methods and existing regression approaches from extreme value theory. We apply our methodology to extreme quantile prediction for U.S. wage data. |