CONF
misr03a/IDIAP
Confusion Matrix Based Entropy Correction in Multi-stream Combination
Misra, Hemant
Morris, Andrew
EXTERNAL
http://publications.idiap.ch/attachments/reports/2003/misra_2003_eurospeech.pdf
PUBLIC
http://publications.idiap.ch/index.php/publications/showcite/morris-rr-02-53
Related documents
Proceedings of ISCA European Conference on Speech Communication and Technology (Eurospeech)
2003
Geneve, Switzerland
September 2003
IDIAP-RR 2002 53
An MLP classifier outputs a posterior probability for each class. With noisy data, classification becomes less certain, and the entropy of the posteriors distribution tends to increase providing a measure of classification confidence. However, at high noise levels, entropy can give a misleading indication of classification certainty. Very noisy data vectors may be classified systematically into classes which happen to be most noise-like and the resulting confusion matrix shows a dense column for each noise-like class. In this article we show how this pattern of misclassification in the confusion matrix can be used to derive a linear correction to the MLP posteriors estimate. We test the ability of this correction to reduce the problem of misleading confidence estimates and to enhance the performance of entropy based full-combination multi-stream approach. Better word-error-rates are achieved for Numbers95 database at different levels of added noise. The correction performs significantly better at high SNRs.
REPORT
morris-RR-02-53/IDIAP
Confusion matrix based posterior probabilities correction
Morris, Andrew
Misra, Hemant
classifier correction
confidence measures
confusion matrix
MLP classifier
EXTERNAL
http://publications.idiap.ch/attachments/reports/2002/rr02-53.pdf
PUBLIC
Idiap-RR-53-2002
2002
IDIAP
An MLP classifier outputs a posterior probability for each class. With noisy data classification becomes less certain and the entropy of the posteriors distribution tends to increase, therefore providing a measure of classification confidence. However, at high noise levels entropy can give a misleading indication of classification certainty because very noisy data vectors may be classified systematically into whichever classes happen to be most noise-like. When this happens the resulting confusion matrix shows a dense column for each noise-like class. In this article we show how this pattern of misclassification in the confusion matrix can be used to derive a linear correction to the MLP posteriors estimate. We test the ability of this correction to reduce the problem of misleading confidence estimates and to increase the performance of individual MLP classifiers. Word and frame level classification results are compared with baseline results for the Numbers95 database of free format telephone numbers, in different levels of added noise.