| Approximate Inference
|
For all but the simplest statistical models, exact learning and
inference are computationally intractable. Approximate inference
methods make it possible to learn realistic models from large data
sets. Generally, approximate inference methods trade off computation
time for accuracy. Some of the major classes of approximate inference
methods include Markov chain Monte Carlo methods, variational methods
and related algorithms such as Expectation Propagation.
See also Monte Carlo Methods.
Some relevant publications:
- Heller, K.A. and Ghahramani, Z. (2005)
Bayesian Hierarchical
Clustering,
Gatsby Unit Technical Report GCNU-TR 2005-002. [ps] [pdf]
A shorter
version was published in the Twenty-second International Conference on Machine
Learning (ICML-2005). [pdf]
- Snelson, E., and Ghahramani, Z. (2005)
Compact approximations to
Bayesian predictive distributions.
In Twenty-second International
Conference on Machine Learning (ICML-2005).
- Qi, Y, Minka, T.P., Picard, R.W., and Ghahramani, Z. (2004)
Predictive Automatic Relevance Determination by Expectation
Propagation
In Twenty-first International Conference on Machine
Learning (ICML-04). Banff, Alberta, Canada.
- Minka, T.P., and Ghahramani, Z. (2003)
Expectation Propagation for Infinite
Mixtures.
Technical Report, presented at the NIPS 2003
Workshop on Nonparametric Bayesian Methods and Infinite
Models.
Talk and abstract at this
website.
- Kim, H.-C. and Ghahramani, Z. (2003)
The EM-EP Algorithm for Gaussian Process
Classification
In the Proceedings of the Workshop on
Probabilistic Graphical Models for Classification (at
ECML). Dubrovnik, Croatia.
-
Beal, M. J. and Ghahramani, Z. (2002)
The Variational Bayesian EM Algorithm for Incomplete Data: with
Application to Scoring Graphical Model Structures [pdf]
[abstract]
In Bayesian Statistics 7
- Ueda, N. and Ghahramani, Z. (2002)
Bayesian model search for mixture models based on optimizing
variational bounds
Neural
Networks 15: 1223-1241.
- Ghahramani, Z. and
Beal, M.J. (2001)
Propagation algorithms for
variational Bayesian learning [pdf] [ps] [abstract]
In Leen, T.K., Dietterich, T.G., and Tresp, V. (eds) Neural
Information Processing Systems 13:507-513. MIT Press.
-
Ghahramani, Z. (2000)
Online Variational Bayesian Learning [pdf]
Slides from talk presented at NIPS 2000 workshop on Online
Learning.
Joint work with H. Attias
- Ghahramani, Z. and
Beal, M.J. (2000)
Graphical models and variational methods
In Saad & Opper (eds)
Advanced Mean Field Method---Theory and Practice. MIT
Press
-
Ghahramani, Z. and Beal, M.J. (1999)
Variational inference for Bayesian mixtures of factor
analysers [pdf] [ps]
[abstract]
In Neural Information Processing Systems 12
-
Jordan, M.I, Ghahramani, Z., Jaakkola, T.S., and Saul, L.K. (1999)
An introduction to variational methods for graphical models
Machine Learning 37:183-233.
-
Ghahramani, Z. and Hinton, G.E. (1998)
Variational learning for switching state-space models
Neural Computation, 12(4):963-996. [abstract]
- Ghahramani, Z. (1997, revised 2002)
On Structured Variational Approximations
University of Toronto Technical Report CRG-TR-97-1, 6 pages
(short note)
- Ghahramani, Z. and Jordan, M.I. (1997)
Factorial Hidden Markov Models
Machine
Learning 29: 245-273. [abstract]
Software written in Matlab.
- Jordan, M.I, Ghahramani, Z. and Saul, L.K. (1997)
Hidden Markov Decision Trees
In
Advances in Neural Information Processing Systems 9,
7 pages.
- Ghahramani, Z. (1995)
Factorial Learning and the EM Algorithm
In G. Tesauro, D.S. Touretzky, and J. Alspector (eds.),
Advances in Neural Information Processing Systems 7, 8
pages. [abstract]