Machine Learning 4f13 Lent 2012

Keywords: Machine learning, probabilistic modelling, graphical models, approximate inference, Bayesian statistics

Taught By: Carl Edward Rasmussen and Joaquin Quiñonero-Candela

Code and Term: 4F13 Lent term

Year: 4th year (part IIB) Engineering, also open to MPhil and PhD students in any Department.

Structure & Assessment:14 lectures, 2 coursework revisions, 3 pieces of course work. The evaluation is by coursework only, all three pieces of course work carry an equal weight. There is no final exam.

Time: 10:00 - 11:00 Mondays and 11:00 - 12:00 Thursdays.

Location: Lecture Room 5 (LR5), Baker Building, Trumpington Street, Cambridge

Prerequisites: A good background in statistics, calculus, linear algebra, and computer science. 3F3 Signal and Pattern Processing and 4F10 Statistical Pattern Processing would both be useful. You should thoroughly review the maths in the following cribsheet [pdf] [ps] before the start of the course. The following Matrix Cookbook is also a useful resource. If you want to do the optional coursework you need to know Matlab or Octave, or be willing to learn it on your own. Any student or researcher at Cambridge meeting these requirements is welcome to attend the lectures. Students wishing to take it for credit should consult with the course lecturers.

Textbook: There is no required textbook. However, the material covered is treated in:

Christopher M. Bishop (2006) Pattern Recognition and Machine Learning. Springer

and we will provide references to sections in this book. Another excellent textbook is:
David J.C. MacKay (2003) Information Theory, Inference, and Learning Algorithms, Cambridge University Press, available freely on the web.

NOTE: If you want to see lecture slides from a similar but not identical course taught last year click on the Lent 2011 course website, but be warned that the slides will change this year.


This year, the exposition of the material will be centered around three specific machine learning areas: 1) supervised non-paramtric probabilistic inference using Gaussian processes, 2) the latent Dirichlet Allocation model for unsupervised learning in text and 3) the TrueSkill ranking system.

Jan 19, 23 Introduction to Machine Learning(2L): probabilistic models, inference, Bayes rule Lecture 1 and 2 slides
Jan 26, 30 and Feb 2nd Gaussian processes (3L): Lecture 3 and 4 slides
Lecture 5 slides
Feb 6, 9, 13, 16 and 20 LDA (5L): Lecture 6: Discrete distributions
Lecture 7: Dirichelt distribtions and Text
Lecture 8: Graphical models for Text
Lecture 9: Latent Dirichlet Allocation for Topic Modelling
Lecture 10: Latent Dirichlet Allocation for Topic Modelling
Feb 23, 27, Mar 1, 5, 8, 12 TrueSkill (6L):

Lecture 11 and 12: Introduction to Ranking
Feedback on Assignment number 1
Lecture 13: Message passing on graphs


Course work is to be handed in to Rachel Fogg in Baker BNO-37 no later than 16:00 on the date due. Each of the three pieces of course work carry an equal weight in the evaluation.

Coursework #1
Coursework 1 is about regression using Gaussian processes. You will need the following files cw1d.mat, cw1e.mat and mauna.txt.
Due: 4pm Thursday Feb 16th, 2012 to Rachel Fogg, room BNO-37

Coursework #2
Coursework 2 is about the Latent Dirichlet Allocation (LDA) model. You will need the following files cw/kos_doc_data.mat, cw/bmm.m, cw/lda.m, cw/sampDiscrete.m.
Due: 4pm Thursday March 1st, 2012 to Rachel Fogg, room BNO-37

Coursework #3
Coursework 3 will be about Probabilistic Ranking. You will need the following files cw/tennis_data.mat, cw/cw3.m, cw/gibbsrank.mat and cw/eprank.m.
Due: 4pm Thursday March 15th, 2012 to Rachel Fogg, room BNO-37