Taught By: Carl Edward Rasmussen and Zoubin Ghahramani
Code and Term: 4F13 Lent term
Year: 4th year (part IIB) Engineering, also open to MPhil and PhD students in any Department.
Structure & Assessment:14 lectures, 2 coursework revisions, 3 pieces of course work. The evaluation is by coursework only, all three pieces of course work carry an equal weight. There is no final exam.
Time: 9:00 - 10:00 Mondays and 9:00 - 10:00 Thursdays.
Location: Lecture Room 3 (LR3), Inglis Building, Trumpington Street, Cambridge
Prerequisites: A good background in statistics, calculus, linear algebra, and computer science. 3F3 Signal and Pattern Processing and 4F10 Statistical Pattern Processing would both be useful. You should thoroughly review the maths in the following cribsheet [pdf] [ps] before the start of the course. The following Matrix Cookbook is also a useful resource. If you want to do the optional coursework you need to know Matlab or Octave, or be willing to learn it on your own. Any student or researcher at Cambridge meeting these requirements is welcome to attend the lectures. Students wishing to take it for credit should consult with the course lecturers.
Textbook: There is no required textbook. However, the material covered is treated excellent recent text books:
Kevin P. Murphy Machine Learning: a Probabilistic Perspective, the MIT Press (2012).
David Barber Bayesian Reasoning and Machine Learning, Cambridge University Press (2012), avaiable freely on the web.
Christopher M. Bishop Pattern Recognition and Machine Learning. Springer (2006)
David J.C. MacKay Information Theory, Inference, and Learning Algorithms, Cambridge University Press (2003), available freely on the web.NOTE: If you want to see lecture slides from a similar but not identical course taught last year click on the Lent 2013 course website, but be warned that the slides will change slightly this year.
This year, the exposition of the material will be centered around three specific machine learning areas: 1) supervised non-paramtric probabilistic inference using Gaussian processes, 2) the TrueSkill ranking system and 3) the latent Dirichlet Allocation model for unsupervised learning in text.
| Jan 16, 20 |
Introduction to Machine Learning(2L): the concept of a model, linear in the parameters regression: makaing predictions, least squares fit, overfitting likelihood and the concept of noise: Gaussian iid noise, maximum likelihood fitting, equivalence to least squares, motivation for inference with multiple hypotheses probability basics: Medical diagnosis, joint, conditional and marginal probability, the two rules: sum and product, and Bayes rule Bayesian inference and prediction: likelihood and prior, posterior and predictive distribution Marginal Likelihood: Bayesian model selection, example: How Bayes avoid overfitting |
Lecture 1 and 2 slides |
| Jan 23, 27, 30 |
Gaussian Processes(3L): Distributions over parameters and over functions: Motivation: representaion of multiple hypothesis, concepts of prior over functions and over parameters, inference, priors over functions are priors over long vectors Gaussian process priors: from finite multi-variate Gaussians to Gaussia processes, GP definition, conditional generation and joint generation |
Lecture 3 and 4 slides Lecture 5 slides |
| Feb 3, 6, 10, 13 | Probabilistic Ranking | Lecture 6 and 7 slides Lecture 8 and 9 slides |
| Feb 17, 20, 24, 27, Mar 3, 6 | Text and Discrete Distributions: Motivation: unsupervised learning on text corpora, discrete distributions, Berboulli, Binomial, multinomial, categorical and Dirichlet distributions Mixture models for text and the Expectation Maximization (EM) algorithm. Latent Dirichlet Allocation (LDA) model |
Lecture 10 and 11 slides Lecture 12 slides Lecture 13 and 14 slides |
COURSE WORK
Course work is to be handed in to Michael Hawkes in Baker BNO-37 no later than 16:00 on the date due. Each of the three pieces of course work carry an equal weight in the evaluation.
Coursework #1