REPORT
moerland00:mix/IDIAP
Mixtures of latent variable models for density estimation and classification
Moerland, Perry
Bayesian latent variable model
evidence framework
Expectation-Maximization algorithm
factor analysis
latent variable
mixture models
normal distribution
principal component analysis
EXTERNAL
http://publications.idiap.ch/attachments/reports/2000/rr00-25.pdf
PUBLIC
Idiap-RR-25-2000
2000
IDIAP
Submitted for publication
This paper deals with the problem of probability density estimation with the goal of finding a good probabilistic representation of the data. One of the most popular density estimation methods is the Gaussian mixture model (GMM). A promising alternative to GMMs are the recently proposed mixtures of latent variable models. Examples of the latter are principal component analysis and factor analysis. The advantage of these models is that they are capable of representing the covariance structure with less parameters by choosing the dimension of a subspace in a suitable way. An empirical evaluation on a large number of data sets shows that mixtures of latent variable models almost always outperform various GMMs both in density estimation and Bayes classifiers. To avoid having to choose a value for the dimension of the latent subspace by a computationally expensive search technique such as cross-validation, a Bayesian treatment of mixtures of latent variable models is proposed. This framework makes it possible to determine the appropriate dimension during training and experiments illustrate its viability.