CONF hamed01/IDIAP Hierarchical Multi-Stream Posterior Based Speech Recognition System Ketabdar, Hamed Bourlard, Hervé Bengio, Samy EXTERNAL https://publications.idiap.ch/attachments/reports/2005/rr05-25.pdf PUBLIC https://publications.idiap.ch/index.php/publications/showcite/hamed-rr05-25 Related documents Proceedings MLMI workshop 2005 Edinburgh, U.K. IDIAP-RR 05-25 In this paper, we present initial results towards boosting posterior based speech recognition systems by estimating more informative posteriors using multiple streams of features and taking into account acoustic context (e.g., as available in the whole utterance,',','), as well as possible prior information (such as topological constraints). These posteriors are estimated based on ``state gamma posterior'' definition (typically used in standard HMMs training) extended to the case of multi-stream HMMs.%, resulting in new features. This approach provides a new, principled, theoretical framework for hierarchical estimation/use of posteriors, multi-stream feature combination, and integrating appropriate context and prior knowledge in posterior estimates. In the present work, we used the resulting gamma posteriors as features for a standard HMM/GMM layer. On the OGI Digits database and on a reduced vocabulary version (1000 words) of the DARPA Conversational Telephone Speech-to-text (CTS) task, this resulted in significant performance improvement, compared to the state-of-the-art Tandem systems. REPORT hamed-rr05-25/IDIAP Hierarchical Multi-Stream Posterior Based Speech Recognition System Ketabdar, Hamed Bourlard, Hervé Bengio, Samy EXTERNAL https://publications.idiap.ch/attachments/reports/2005/rr05-25.pdf PUBLIC Idiap-RR-25-2005 2005 IDIAP In this paper, we present initial results towards boosting posterior based speech recognition systems by estimating more informative posteriors using multiple streams of features and taking into account acoustic context (e.g., as available in the whole utterance,',','), as well as possible prior information (such as topological constraints). These posteriors are estimated based on ``state gamma posterior'' definition (typically used in standard HMMs training) extended to the case of multi-stream HMMs.%, resulting in new features. This approach provides a new, principled, theoretical framework for hierarchical estimation/use of posteriors, multi-stream feature combination, and integrating appropriate context and prior knowledge in posterior estimates. In the present work, we used the resulting gamma posteriors as features for a standard HMM/GMM layer. On the OGI Digits database and on a reduced vocabulary version (1000 words) of the DARPA Conversational Telephone Speech-to-text (CTS) task, this resulted in significant performance improvement, compared to the state-of-the-art Tandem systems.