REPORT Szaszak_Idiap-RR-25-2013/IDIAP Automatic Speech Indexing System of Bilingual Video Parliament Interventions Szaszak, Gyorgy Cernak, Milos Garner, Philip N. Motlicek, Petr Nanchen, Alexandre Tarsetti, Flavio EXTERNAL http://publications.idiap.ch/attachments/reports/2013/Szaszak_Idiap-RR-25-2013.pdf PUBLIC Idiap-RR-25-2013 2013 Idiap July 2013 This paper presents the development and evaluation of an automatic audio indexing system designed for a special task: work in a bilingual environment in the Parliament of the Canton of Valais in Switzerland, with two official languages, German and French. As several speakers are bilingual, language changes may occur within speaker or even within utterance. Two audio indexing approaches are presented and compared: in the first, speech indexing is based on bilingual automatic speech recognition; in the second, language identification is used after speaker diarization in order to select the corresponding monolingual speech recognizer for decoding. The approaches are later combined. Speaker adaptive training is also addressed and evaluated. Accuracy of language identification and speech recognition for the monolingual and bilingual cases are presented and compared, in parallel with a brief description of the system and the user interface. Finally, the audio indexing system is also evaluated from an information retrieval point of view.