REPORT moerland00:mix/IDIAP Mixtures of latent variable models for density estimation and classification Moerland, Perry Bayesian latent variable model evidence framework Expectation-Maximization algorithm factor analysis latent variable mixture models normal distribution principal component analysis EXTERNAL http://publications.idiap.ch/attachments/reports/2000/rr00-25.pdf PUBLIC Idiap-RR-25-2000 2000 IDIAP Submitted for publication This paper deals with the problem of probability density estimation with the goal of finding a good probabilistic representation of the data. One of the most popular density estimation methods is the Gaussian mixture model (GMM). A promising alternative to GMMs are the recently proposed mixtures of latent variable models. Examples of the latter are principal component analysis and factor analysis. The advantage of these models is that they are capable of representing the covariance structure with less parameters by choosing the dimension of a subspace in a suitable way. An empirical evaluation on a large number of data sets shows that mixtures of latent variable models almost always outperform various GMMs both in density estimation and Bayes classifiers. To avoid having to choose a value for the dimension of the latent subspace by a computationally expensive search technique such as cross-validation, a Bayesian treatment of mixtures of latent variable models is proposed. This framework makes it possible to determine the appropriate dimension during training and experiments illustrate its viability.