logo Idiap Research Institute        
 [BibTeX] [Marc21]
Crossmodal Matching of Speakers using Lip and Voice Features in Temporally Non-overlapping Audio and Video Streams
Type of publication: Idiap-RR
Citation: Roy_Idiap-RR-13-2010
Number: Idiap-RR-13-2010
Year: 2010
Month: 6
Institution: Idiap
Abstract: Person identification using audio (speech) and visual (facial appearance, static or dynamic) modalities, either independently or jointly, is a thoroughly investigated problem in pattern recognition. In this work, we explore a novel task : person identification in a cross-modal scenario, i.e., matching the speaker in an audio recording to the same speaker in a video recording, where the two recordings have been made during different sessions, using speaker specific information which is common to both the audio and video modalities. Several recent psychological studies have shown how humans can indeed perform this task with an accuracy significantly higher than chance. Here we propose two systems which can solve this task comparably well, using purely pattern recognition techniques. We hypothesize that such systems could be put to practical use in multimodal biometric and surveillance systems.
Keywords:
Projects Idiap
MOBIO
SNSF-MULTI
IM2
Authors Roy, Anindya
Marcel, Sébastien
Crossref by Roy_ICPR2010_2010
Added by: [ADM]
Total mark: 0
Attachments
  • Roy_Idiap-RR-13-2010.pdf (MD5: c3b8eb3643bc675d8312f5accafd4e02)
Notes