CONF misr05a/IDIAP Spectral Entropy Feature in Full-Combination Multi-Stream for Robust ASR Misra, Hemant Bourlard, Hervé EXTERNAL http://publications.idiap.ch/attachments/reports/2005/rr05-10.pdf PUBLIC http://publications.idiap.ch/index.php/publications/showcite/misra-rr-05-10 Related documents Proceedings of ISCA European Conference on Speech Communication and Technology (Eurospeech) 2005 Lisbon, Portugal September 2005 IDIAP-RR 2005 10 In a recent paper, we reported promising automatic speech recognition results obtained by appending spectral entropy features to PLP features. In the present paper, spectral entropy features are used along with PLP features in the framework of multi-stream combination. In a full-combination multi-stream hidden Markov model/artificial neural network (HMM/ANN) hybrid system, we train a separate multi-layered perceptron (MLP) for PLP features, for spectral entropy features and for both combined by concatenation. The output posteriors from these three MLPs are combined with weights inversely proportional to the entropies of their respective posterior distributions. We show that on the Numbers95 database, this approach yields a significant improvement under both clean and noisy conditions as compared to simply appending the features. Further, in the framework of a Tandem HMM/ANN system, we apply the same inverse entropy weighting to combine the outputs of the MLPs before the softmax non-linearity. Feeding the combined and decorrelated MLP outputs to the HMM gives a 9.2\% relative error reduction as compared to the baseline. REPORT misra-rr-05-10/IDIAP Spectral Entropy Feature in Full-Combination Multi-stream for Robust ASR Misra, Hemant Bourlard, Hervé EXTERNAL http://publications.idiap.ch/attachments/reports/2005/rr05-10.pdf PUBLIC Idiap-RR-10-2005 2005 IDIAP Martigny, Switzerland In Proceedings of ISCA European Conference on Speech Communication and Technology {(Eurospeech)}, 2005 In a recent paper, we reported promising automatic speech recognition results obtained by appending spectral entropy features to PLP features. In the present paper, spectral entropy features are used along with PLP features in the framework of multi-stream combination. In a full-combination multi-stream hidden Markov model/artificial neural network (HMM/ANN) hybrid system, we train a separate multi-layered perceptron (MLP) for PLP features, for spectral entropy features and for both combined by concatenation. The output posteriors from these three MLPs are combined with weights inversely proportional to the entropies of their respective posterior distributions. We show that on the Numbers95 database, this approach yields a significant improvement under both clean and noisy conditions as compared to simply appending the features. Further, in the framework of a Tandem HMM/ANN system, we apply the same inverse entropy weighting to combine the outputs of the MLPs before the softmax non-linearity. Feeding the combined and decorrelated MLP outputs to the HMM gives a 9.2\% relative error reduction as compared to the baseline.