Title: Parameter Inference in Nonlinear Dynamical Systems using Gradient Matching
Inference in mechanistic models of non-linear differential equations is a challenging problem in current computational statistics. Due to the high computational costs of numerically solving the differential equations in every step of an iterative parameter adaptation scheme, approximate methods based on gradient matching have become popular. However, these methods critically hinge on the smoothing scheme for function interpolation. The present work adapts an idea from manifold learning and demonstrates that a time warping approach aiming to homogenize intrinsic length scales can lead to a significant improvement in parameter estimation accuracy. We demonstrate the effectiveness of this scheme on noisy data from two dynamical systems with periodic limit cycle, a biopathway, and an application from soft-tissue mechanics. Our study also provides a comparative evaluation on a wide range of signal-to-noise ratios.
Title: Score matching estimators for directional distributions
Hyvarinen's method of ''score matching estimation'' is an alternative to maximum likelihood estimation for exponential families. One major advantage is that it can eliminate the need to compute awkward normalizing constants. However, the price is a (usually modest) loss of efficiency and a restriction to densities on a Riemannian manifold which are differentiable with respect to the state variable in addition to the parameters. Important applications include von Mises-Fisher, Bingham and joint models on the sphere and related spaces. Several examples will be given, both analytic and numerical, to demonstrate its good performance.
Title: Measuring Sample Quality with Diffusions - how explicit PDE regularity result can excite machine learners
Yves van Gennip.
Title: PDE techniques for network problems
In recent years, ideas from the world of partial differential equations (PDEs) have found their way into the arena of graph and network problems. In this talk I will discuss how techniques based on nonlinear PDE models, such as the Allen-Cahn equation and the Merriman-Bence-Osher threshold dynamics scheme can be used to (approximately) detect particular structures in graphs, such as densely connected subgraphs (clustering and classification, minimum cuts) and bipartite subgraphs (maximum cuts). Such techniques not only often lead to fast algorithms that can be applied to large networks, but also pose interesting theoretical questions about the relationships between the graph models and their continuum counterparts, and about connections between the different graph models.
Title: Bayesian Uncertainty Quantification in the Classification of High Dimensional Data
In this talk, we present a Bayesian framework for semi-supervised binary classification on graphs. We develop several Bayesian models through the construction of a Gaussian prior from the graph Laplacian. Connections to the Ginzburg-Landau model are also made through the notion of a push-forward of the Gaussian prior under the double-well thresholding. We introduce efficient MCMC methods designed for large data sets to effectively sample from the posterior distribution for large scale problems. Through a variety of numerical experiments, we demonstrate the ability to perform uncertainty quantification by sampling from the posterior distribution. In particular, we observe empirically that the posterior mean and variance aligns well with certain external notions of uncertainty.
This is joint work with Andrea Bertozzi (UCLA), Michaeal Luo (UCLA) and Andrew Stuart (Caltech)
Title: Large Data Limits for Probabilistic Graphical Models
In this talk I will discuss the large data limits for a selection of graphical modelling techniques based on the graph Laplacian. In particular, I will focus on the Ginzburg-Landau model, the Probit model, the Kriging model, and the level set approach. Each of these models is based on minimizing an energy. Therefore, one can recast into a probabilistic framework by defining a Boltzmann distribution. In particular this allows for uncertainty quantification. I will give results concerning the asymptotics both in terms of minimizers of the energy and the Boltzmann distribution. Establishing the large data limit involves a passage from discrete to continuum, one which we can make using an optimal transport based topology that allows comparisons between discrete and continuum objects.
This is joint work with Matt Dunlop (Caltech), Dejan Slepcev (CMU), Andrew Stuart (Caltech) and Florian Theil (Warwick).
Title: Pattern formation of a nonlocal, anisotropic interaction model
We consider a class of large interacting particle systems with anisotropic repulsive-attractive interaction forces whose orientations depend on an underlying tensor field. An example of this class of models is the so-called Kücken-Champod model describing the formation of fingerprint patterns. This class of models can be regarded as a generalization of a gradient flow of a nonlocal interaction potential which has a local repulsion and a long-range attraction structure. In contrast to isotropic interaction models the anisotropic forces in our class of models cannot be derived from a potential. The underlying tensor field introduces an anisotropy leading to complex patterns which do not occur in isotropic models. We study the stationary states for the transition between the isotropic and the anisotropic model analytically by considering the associated mean-field PDEs and investigate the pattern formation numerically. Based on these theoretical and numerical results we adapt the forces in the Kücken-Champod model in such a way that we can model fingerprint patterns (and more general any desired pattern) as stationary solutions. This is joint work with M. Burger, B. Düring, P. Markowich and C.-B. Schönlieb.
Title: A Max-Cut Approximation Using A Graph Based MBO Scheme
The Max-Cut problem is a well known combinatorial optimization problem. In this talk we describe a fast approximation method. Given a graph G, we want to find a cut whose size is maximal among all possible cuts. A cut is a partition of the vertex set of G into two nonempty disjoint subsets. For an unweighted graph, the size of the cut is the number of edges that have one vertex on either side of the partition; we also consider a weighted version of the problem where each edge contributes a nonnegative weight to the cut.
We introduce the signless Ginzburg-Landau functional and prove that this functional $\Gamma$-converges to a Max-Cut objective functional. We approximately minimize this functional using a graph based signless MBO scheme, which uses a signless Laplacian. We show experimentally that on certain classes of graphs the resulting algorithm produces on average more accurate maximum cut approximations than the current state-of-the-art approximation algorithm. This work is done with collaboration with Yves van Gennip from The University of Nottingham.
Title: A combined graph clustering-Hough transform method for object segmentation and scale detection in images.
We consider the problem of image segmentation via the minimisation of a Ginzburg-Landau-type functional defined on graphs (the pixel images). Such approach has first been considered by A. Bertozzi and A. Flenner and provides a binary segmentation of the image through the extraction and the comparison of features (RGB, texture,...) in terms of an appropriate similarity measure and a training set. From a mathematical point of view, the desired segmentation is obtained by exploiting the spectral properties of the differential operators appearing when taking the $\ell^2$ discrete Allen-Cahn flow. In order to overcome the numerical difficulties due to the large amount of image data considered, Nyström matrix completion techniques and convex splitting methods are employed. We apply such method to the problem of scale detection in images where a fuzzy region of interest is present together with a measurement tool (e.g. a ruler). In particular, by means of a Hough transform based algorithm, the combined method is applied to several measurement tasks arising in real-world applications. We show the results obtained applying the combined model to an image classification problem in collaboration with the Zoology Department of the University of Cambridge, UK.
This is joint work with Yves van Gennip (University of Nottingham), Carola Schönlieb and Hannah Rowland (University of Cambridge), Arjuna Flenner (NAVAIR, USA).
Title: Heat kernels in graphs: A journey from random walks to geometry, and back
Heat kernels are one of the most fundamental concepts in physics and mathematics. In physics, the heat kernel is a fundamental solution of the heat equation and connects the Laplacian operator to the rate of heat dissipation. In spectral geometry, many fundamental techniques are based on heat kernels. In finite Markov chain theory, heat kernels correspond to continuous-time random walks and constitute one of the most powerful techniques in estimating the mixing time.
In this talk, we will briefly discuss this line of research and its relation to heat kernels in graphs. In particular, we will see how heat kernels are used to design the first nearly-linear time algorithm for finding clusters in real-world graphs. Some interesting open questions will be addressed in the end.
Title: Graph Methods for Manifold-Valued Data
There exist real applications in which measured data are not in a Euclidean vector space but rather are given on a Riemannian manifold. This is the case, e.g., when dealing with Interferometric Synthetic Aperture Radar (InSAR) data consisting of phase values or data obtained in Diffusion Tensor Magnetic Resonance Imaging (DT-MRI).
In this talk we present a framework for processing discrete manifold-valued data, for which the underlying (sampling) topology is modeled by a graph. We introduce the notion of a manifold-valued derivative on a graph and based on this deduce a family of manifold-valued graph operators. In particular, we introduce the graph p-Laplacian and graph infinity-Laplacian for manifold-valued data. We discuss a numerical scheme to compute a solution to the corresponding parabolic PDEs and apply this algorithm to different manifold-valued data, illustrating the diversity and flexibility of the proposed
framework in denoising and inpainting applications.
This is joint work with Ronny Bergmann (TU Kaiserslautern).
Title: Laplacian-based methods for ranking, constrained clustering and slow variable detection
We consider the classic problem of establishing a statistical ranking of a set of n items given a set of inconsistent and incomplete pairwise comparisons between such items. Instantiations of this problem occur in numerous applications in data analysis (e.g., ranking teams in sports data), computer vision, and machine learning. We formulate the above problem of ranking with incomplete noisy information as an instance of the group synchronization problem over the group SO(2) of planar rotations, whose usefulness has been demonstrated in numerous applications in recent years. Its least squares solution can be approximated by either a spectral or a semidefinite programming relaxation, followed by a rounding procedure. We perform extensive numerical simulations on both synthetic and real-world data sets, showing that our proposed method compares favorably to other algorithms from the recent literature.
We also present a simple spectral approach to the well-studied constrained clustering problem. It captures constrained clustering as a generalized eigenvalue problem with graph Laplacians. The algorithm works in nearly-linear time and provides concrete guarantees for the quality of the clusters, at least for the case of 2-way partitioning. In practice this translates to a very fast implementation that consistently outperforms existing spectral approaches both in speed and quality.
Finally, we introduce a method for detecting intrinsic slow variables in high-dimensional stochastic chemical reaction networks. It combines anisotropic diffusion maps (ADM) with approximations based on the chemical Langevin equation (CLE). The resulting approach, called ADM-CLE, has the potential of being more efficient than the ADM method for a large class of chemical reaction systems. The ADM-CLE approach can be used to estimate the stationary distribution of the detected slow variable, without any a-priori knowledge of it.
The problems we study share an important common feature: they can all be solved by exploiting the spectrum of their corresponding graph Laplacian.
Title: PDE-constrained optimisation for model-based learning
One of the most successful approaches to solve inverse problems in imaging is to cast the problem as a variational model. The key to the success of the variational approach is to define the variational energy such that its minimiser reflects the structural properties of the imaging problem in terms of regularisation and data consistency.
Variational models constitute mathematically rigorous inversion models with stability and approximation guarantees as well as a control on qualitative and physical properties of the solution. On the negative side, these methods are rigid in a sense that they can be adapted to data only to a certain extent.
Hence researchers started to apply machine learning techniques to “learn” more expressible variational models. The basic principle is to consider a bilevel optimization problem, where the variational model appears as the lower-level problem and the higher-level problem is the minimization over a loss function that measures the reconstruction error for the solution of the variational model. In this talk we discuss bilevel optimisation, its analysis and numerical treatment, and show applications to regularisation learning, learning of noise models and of sampling patterns in MRI.
This talk includes joint work with M. Benning, L. Calatroni, C. Chung, J. C. De Los Reyes, M. Ehrhardt, G. Maierhofer, F. Sherry, T. Valkonen, and V. Vladic.