CONF Ullmann_INTERSPEECH_2015/IDIAP Objective Intelligibility Assessment of Text-to-Speech Systems Through Utterance Verification Ullmann, Raphael Rasipuram, Ramya Magimai-Doss, Mathew Bourlard, Hervé KL-divergence KL-HMM objective measures Speech intelligibility text-to-speech synthesis utterance verification EXTERNAL http://publications.idiap.ch/attachments/papers/2015/Ullmann_INTERSPEECH_2015.pdf PUBLIC http://publications.idiap.ch/index.php/publications/showcite/Ullmann_Idiap-RR-06-2015 Related documents Proceedings of Interspeech Dresden, Germany 2015 3501-3505 http://isca-speech.org/archive/interspeech_2015/i15_3501.html URL Objective assessment of synthetic speech intelligibility can be a useful tool for the development of text-to-speech (TTS) systems, as it provides a reproducible and inexpensive alternative to subjective listening tests. In a recent work, it was shown that the intelligibility of synthetic speech could be assessed objectively by comparing two sequences of phoneme class conditional probabilities, corresponding to instances of synthetic and human reference speech, respectively. In this paper, we build on those findings to propose a novel approach that formulates objective intelligibility assessment as an utterance verification problem using hidden Markov models, thereby alleviating the need for human reference speech. Specifically, given each text input to the TTS system, the proposed approach automatically verifies the words in the output synthetic speech signal and estimates an intelligibility score based on word recall statistics. We evaluate the proposed approach on the 2011 Blizzard Challenge data, and show that the estimated scores and the subjective intelligibility scores are highly correlated (Pearson’s |R| = 0.94). REPORT Ullmann_Idiap-RR-06-2015/IDIAP Objective Intelligibility Assessment of Text-to-Speech Systems Through Utterance Verification Ullmann, Raphael Rasipuram, Ramya Magimai-Doss, Mathew Bourlard, Hervé KL-divergence KL-HMM objective measures Speech intelligibility text-to-speech synthesis utterance verification EXTERNAL http://publications.idiap.ch/attachments/reports/2015/Ullmann_Idiap-RR-06-2015.pdf PUBLIC Idiap-RR-06-2015 2015 Idiap April 2015 Objective assessment of synthetic speech intelligibility can be a useful tool for the development of text-to-speech (TTS) systems, as it provides a reproducible and inexpensive alternative to subjective listening tests. In a recent work, it was shown that the intelligibility of synthetic speech could be assessed objectively by comparing two sequences of phoneme class conditional probabilities, corresponding to instances of synthetic and human reference speech, respectively. In this paper, we build on those findings to propose a novel approach that formulates objective intelligibility assessment as an utterance verification problem using hidden Markov models, thereby alleviating the need for human reference speech. Specifically, given each text input to the TTS system, the proposed approach automatically verifies the words in the output synthetic speech signal and estimates an intelligibility score based on word recall statistics. We evaluate the proposed approach on the 2011 Blizzard Challenge data, and show that the estimated scores and the subjective intelligibility scores are highly correlated (Pearson’s |R| = 0.94).