logo Idiap Research Institute        
 [BibTeX] [Marc21]
Using the Multi-Stream Approach for Continuous Audio-Visual Speech Recognition
Type of publication: Idiap-RR
Citation: dupont-RR-97-14
Number: Idiap-RR-14-1997
Year: 1997
Institution: IDIAP
Abstract: The Multi-Stream automatic speech recognition approach was investigated in this work as a framework for Audio-Visual data fusion and speech recognition. This method presents many potential advantages for such a task. It particularly allows for synchronous decoding of continuous speech while still allowing for some asynchrony of the visual and acoustic information streams. First, the Multi-Stream formalism is briefly recalled. Then, on top of the Multi-Stream motivations, experiments on the {\sc M2VTS} multimodal database are presented and discussed. To our knowledge, these are the first experiments about multi-speaker continuous Audio-Visual Speech Recognition (AVSR). It is shown that the Multi-Stream approach can yield improved Audio-Visual speech recognition performance when the acoustic signal is corrupted by noise as well as for clean speech.
Userfields: ipdmembership={vision},
Projects Idiap
Authors Dupont, St├ęphane
Luettin, Juergen
Added by: [UNK]
Total mark: 0
  • rr97-14.pdf
  • rr97-14.ps.gz