Skip to main content Skip to navigation

Statistical Learning and Big Data


Commitment:

  • 30 hours of lectures + 10 hours tutorials via office hours

Content:

  • Statistical Learning
    • An introduction to statistical learning theory: from over-fitting to apparently complex methods which can work well. For example, VC dimension and shattering sets, PAC bounds.
    • Loss functions. Risk (in the learning theory sense); posterior expected risk. Generalisation error.
    • Supervised, unsupervised and semi-supervised learning.
    • The use of distinct training, test and validation sets particularly in the context of prediction problems.
    • The bootstrap revisited. Bags of little bootstraps. Bootstrap aggregation. Boosting.
    • ML method will be used to illustrate these ideas.
  • Big data and big model related issues and (partial) solutions
    • "Curse of dimensionality". Multiple testing: voodoo correlations; false-discovery rate and family-wise error rate; corrections: Bonferroni, Benjamini-Hochberg.
    • Sparsity and regularisation. Variable selection; regression. Spike and slab priors. Ridge regression. The Lasso. The Dantzig selector.
    • Concentration of measure, related inferential issues.
    • MCMC in high dimensions, preconditioned Crank Nicolson; MALA; HMC. Preconditioning. Rates of convergence.

Assessment:

  • 1 x 2-hour exam

Illustrative Bibliography:

  • Chris Bishop, Pattern recognition and machine learning, Springer, 2006.
  • Peter Buehlman, Statistics for high-dimensional data: methods, theory and applications, Springer, 2011.
  • Trevor Hastie, Robert Tibshirani and Jerome Friedman, The elements of statistical learning, Springer, 2009.
  • Trevor Hastie, Robert Tibshirani and Martin Wainwright, Statistical learning with sparsity, CRC Press, 2015.
  • Kevin Murphy, Probabilistic machine learning: a probabilistic perspective, MIT Press, 2012.

Examination Period: April