REPORT Liang_Idiap-RR-16-2010/IDIAP An Analysis of Language Mismatch in HMM State Mapping-Based Cross-Lingual Speaker Adaptation Liang, Hui Dines, John EXTERNAL http://publications.idiap.ch/attachments/reports/2010/Liang_Idiap-RR-16-2010.pdf PUBLIC Idiap-RR-16-2010 2010 Idiap July 2010 This paper provides an in-depth analysis of the impacts of language mismatch on the performance of cross-lingual speaker adaptation. Our work confirms the influence of language mismatch between average voice distributions for synthesis and for transform estimation and the necessity of eliminating this mismatch in order to effectively utilize multiple transforms for cross-lingual speaker adaptation. Specifically, we show that language mismatch introduces unwanted language-specific information when estimating multiple transforms, thus making these transforms detrimental to adaptation performance. Our analysis demonstrates speaker characteristics should be separated from language characteristics in order to improve cross-lingual adaptation performance.