For all but the simplest statistical models, exact learning and
inference are computationally intractable. Approximate inference
methods make it possible to learn realistic models from large data
sets. Generally, approximate inference methods trade off computation
time for accuracy. Some of the major classes of approximate inference
methods include Markov chain Monte Carlo methods, variational methods
and related algorithms such as Expectation Propagation.
See also Monte Carlo Methods.
Some relevant publications:
- Heller, K.A. and Ghahramani, Z. (2005)
Gatsby Unit Technical Report GCNU-TR 2005-002. [ps] [pdf]
version was published in the Twenty-second International Conference on Machine
Learning (ICML-2005). [pdf]
- Snelson, E., and Ghahramani, Z. (2005)
Compact approximations to
Bayesian predictive distributions.
In Twenty-second International
Conference on Machine Learning (ICML-2005).
- Qi, Y, Minka, T.P., Picard, R.W., and Ghahramani, Z. (2004)
Predictive Automatic Relevance Determination by Expectation
In Twenty-first International Conference on Machine
Learning (ICML-04). Banff, Alberta, Canada.
- Minka, T.P., and Ghahramani, Z. (2003)
Expectation Propagation for Infinite
Technical Report, presented at the NIPS 2003
Workshop on Nonparametric Bayesian Methods and Infinite
Talk and abstract at this
- Kim, H.-C. and Ghahramani, Z. (2003)
The EM-EP Algorithm for Gaussian Process
In the Proceedings of the Workshop on
Probabilistic Graphical Models for Classification (at
ECML). Dubrovnik, Croatia.
Beal, M. J. and Ghahramani, Z. (2002)
The Variational Bayesian EM Algorithm for Incomplete Data: with
Application to Scoring Graphical Model Structures [pdf]
In Bayesian Statistics 7
- Ueda, N. and Ghahramani, Z. (2002)
Bayesian model search for mixture models based on optimizing
Networks 15: 1223-1241.
- Ghahramani, Z. and
Beal, M.J. (2001)
Propagation algorithms for
variational Bayesian learning [pdf] [ps] [abstract]
In Leen, T.K., Dietterich, T.G., and Tresp, V. (eds) Neural
Information Processing Systems 13:507-513. MIT Press.
Ghahramani, Z. (2000)
Online Variational Bayesian Learning [pdf]
Slides from talk presented at NIPS 2000 workshop on Online
Joint work with H. Attias
- Ghahramani, Z. and
Beal, M.J. (2000)
Graphical models and variational methods
In Saad & Opper (eds)
Advanced Mean Field Method---Theory and Practice. MIT
Ghahramani, Z. and Beal, M.J. (1999)
Variational inference for Bayesian mixtures of factor
analysers [pdf] [ps]
In Neural Information Processing Systems 12
Jordan, M.I, Ghahramani, Z., Jaakkola, T.S., and Saul, L.K. (1999)
An introduction to variational methods for graphical models
Machine Learning 37:183-233.
Ghahramani, Z. and Hinton, G.E. (1998)
Variational learning for switching state-space models
Neural Computation, 12(4):963-996. [abstract]
- Ghahramani, Z. (1997, revised 2002)
On Structured Variational Approximations
University of Toronto Technical Report CRG-TR-97-1, 6 pages
- Ghahramani, Z. and Jordan, M.I. (1997)
Factorial Hidden Markov Models
Learning 29: 245-273. [abstract]
Software written in Matlab.
- Jordan, M.I, Ghahramani, Z. and Saul, L.K. (1997)
Hidden Markov Decision Trees
Advances in Neural Information Processing Systems 9,
- Ghahramani, Z. (1995)
Factorial Learning and the EM Algorithm
In G. Tesauro, D.S. Touretzky, and J. Alspector (eds.),
Advances in Neural Information Processing Systems 7, 8