logo Idiap Research Institute        
 [BibTeX] [Marc21]
Object Category Detection using Audio-visual Cues
Type of publication: Conference paper
Citation: luo:icvs:2008
Booktitle: International Conference on Computer Vision Systems (ICVS08)
Year: 2008
Month: 5
Address: Santorini, Greece
Abstract: Categorization is one of the fundamental building blocks of cognitive systems. Object categorization has traditionally been addressed in the vision domain, even though cognitive agents are intrinsically multimodal. Indeed, biological systems combine several modalities in order to achieve robust categorization. In this paper we propose a multimodal approach to object category detection, using audio and visual information. The auditory channel is modeled on biologically motivated spectral features via a discriminative classifier. The visual channel is modeled by a state of the art part based model. Multimodality is achieved using two fusion schemes, one high level and the other low level. Experiments on six different object categories, under increasingly difficult conditions, show strengths and weaknesses of the two approaches, and clearly underline the open challenges for multimodal category detection.
Userfields: ipdmembership={Multimodal}, ipdxref={techreport:luo-rr-07-58.bib},
Projects Idiap
Authors Luo, Jie
Caputo, Barbara
Zweig, Alon
Back, Joerg-Henrik
Anemueller, Joern
Added by: [UNK]
Total mark: 0
  • luo_icvs08.pdf
  • luo_icvs08.ps.gz