Topics for Machine Learning Quals

• Foundations
  • Shannon's Source Coding Theorem
  • Bayes Rule
  • Dutch Books
  • Cox Axioms
  • Bayesian model comparison
• Models
  • Factor Analysis / PCA
  • Independent Components Analysis (ICA)
  • Mixture models / k-means
  • Hidden Markov models (HMMs)
  • State space models (SSMs)
  • Boltzmann machines
  • Graphical models: directed, undirected, factor graphs
• Algorithms
  • The EM Algorithm
  • Belief propagation
  • Forward-backward
  • Kalman filtering and extended Kalman filtering
  • Variational methods
  • Laplace approximation and BIC
  • Markov chain Monte Carlo (MCMC) methods
  • Particle filters
  • Expectation propagation
• Supervised Learning:
  • Linear regression
  • Logistic regression
  • Perceptrons
  • Neural networks (multi-layer perceptrons) and backpropagation
  • Gaussian processes
  • Support vector machines
• Reinforcement Learning
  • Value functions
  • Bellman's equation
  • Value iteration
  • Policy iteration
  • Q-Learning
  • actor-critic
  • TD(lambda)
• Basic Learning Theory
  • VC dimension
  • regularization

• Radford Neal's MCMC tech report
• Zoubin Ghahramani's Chapter on Unsupervised Learning
• Chris Bishop's textbook on Neural networks
• David MacKay's textbook on Information Theory, Inference and Learning Algorithms
• Other readings from my Unsupervised Learning course