Probabilistic Machine Learning 4f13 Michaelmas 2017

Keywords: Machine learning, probabilistic modelling, graphical models, approximate inference, Bayesian statistics

Taught By: Professor Carl Edward Rasmussen

Code and Term: 4F13 Michaelmas term

Year: 4th year (part IIB) Engineering and MPhil in Machine Learning and Speech Technology; also open to MPhil and PhD students in any Department.

Structure & Assessment:14 lectures, 2 coursework revisions, 3 pieces of course work. The evaluation is by coursework only, all three pieces of course work carry an equal weight. There is no final exam.

Time: 16 lectures on Mondays at 14:00 - 15:00 and Thursdays 12:00 - 13:00, both in LT2. First lecture Thursday October 5th. There will also be an informal office hour on Fridays, 15:00 - 16:00, also in LT2. This is an opportunity to ask questions, discuss the material etc. There is of course no expectation or obligation that you attend. First office hour on Friday Oct 13th.

Location: Lecture Theatre 2 (LT2), Inglis Building, ground floor, Department of Engineering, Trumpington Street (map).

Prerequisites: A good background in statistics, calculus, linear algebra, and computer science. 3F3 Signal and Pattern Processing. You should thoroughly review the maths in the following cribsheet [pdf] [ps] before the start of the course. The following Matrix Cookbook is also a useful resource. If you want to do the optional coursework you need to know Matlab or Octave, or be willing to learn it on your own. Any student or researcher at Cambridge meeting these requirements is welcome to attend the lectures. Students wishing to take it for credit should consult with the course lecturers.

Textbook: There is no required textbook. However, the material covered is treated excellent recent text books:

Kevin P. Murphy Machine Learning: a Probabilistic Perspective, the MIT Press (2012).

David Barber Bayesian Reasoning and Machine Learning, Cambridge University Press (2012), avaiable freely on the web.

Christopher M. Bishop Pattern Recognition and Machine Learning. Springer (2006)

David J.C. MacKay Information Theory, Inference, and Learning Algorithms, Cambridge University Press (2003), available freely on the web.

Lecture Syllabus

This year, the exposition of the material will be centered around three specific machine learning areas: 1) supervised non-parametric probabilistic inference using Gaussian processes, 2) the TrueSkill ranking system and 3) the latent Dirichlet Allocation model for unsupervised learning in text.

The organisation of the handouts is changing. This year the material will be structured into small chunks, each containing a single core concept. Printed handouts won't be provided at the lectures, but will be available on this web site. I recommend that you don't bring printed slides to the lectures, but of course you can do so if you think it works better for you.

Note: the links in the table below aren't up to date. If you want to see lecture slides from a similar but not identical course taught last year go to Michaelmas 2016 course website, but be warned that the slides will change this year.

October 5th	Introduction to Probabilistic Machine Learning (2L): Modelling data Linear in the parameters regression Likelihood and the concept of noise
October 9th	Probability fundamentals Bayesian inference and prediction with finite regression models Marginal likelihood
October 12th	Gaussian Processes (3L): Parameters and functions Gaussian Process Posterior Gaussian Process
October 16th	GP marginal likelihood and hyperparameters Correspondence between linear models and GPs Should we use finite or infinite models?
October 19th	Covariance functions Quick introduction to the gpml toolbox
October 23rd	Probabilistic Ranking (3L): Introduction to ranking
Oct 26th, Oct 30th	Gibbs sampling Gibbs sampling demo Gibbs sampling in the TrueSkill model
Nov 2nd, Nov 6th	Factor graphs Message passing in TrueSkill Approximation by moment matching
Nov 9th, Nov 13th	Modelling Document Collections models of text discrete binary distributions categorical, multinomial, discrete distributions
Nov 16th, Nov 20th	Modelling Document Collections Simple categorical and mixture models Learning in models with latent variables: the EM algorithm
Nov 23rd, Nov 27th	Modelling Document Collections Gibbs sampling in mixture models, collapsed Gibbs Latent Dirichlet Allocation topic models

Coursework

Course work is to be submitted via moodle in electronic form no later than 17:00 on the date due. If you are not an egineering undergraduate, please make sure you are signed up for the module on moodle, check with Catherine Munn cm861@cam.ac.uk, in room BE4-45 if you are in doubt. Each of the three pieces of course work carry an equal weight in the evaluation. The course work will be similar, but not identical to last year's, and will be posted shortly on this web site. The due-dates this year are:

Coursework #1
Coursework 1 is about regression using Gaussian processes. You will need the following files cw1a.mat and cw1e.mat.
Due: Friday November 3rd, 2017 at 17:00 via moodle.

Coursework #2
Coursework 2 will be about Probabilistic Ranking. You will need the following files tennis_data.mat, cw2.m, gibbsrank.m and eprank.m.
Due: Friday November 17th, 2017 at 17:00 via moodle.

Coursework #3
Coursework 3 is about the Latent Dirichlet Allocation (LDA) model. You will need the following files kos_doc_data.mat, bmm.m, lda.m, sampDiscrete.m.
Due: Friday 1st December, 2017 at 17:00 via moodle.