REPORT Popescu-Belis_Idiap-RR-11-2009/IDIAP Comparing meeting browsers using a task-based evaluation method Popescu-Belis, Andrei EXTERNAL http://publications.idiap.ch/attachments/reports/2009/Popescu-Belis_Idiap-RR-11-2009.pdf PUBLIC Idiap-RR-11-2009 2009 Idiap June 2009 Information access within meeting recordings, potentially transcribed and augmented with other media, is facilitated by the use of meeting browsers. To evaluate their performance through a shared benchmark task, users are asked to discriminate between true and false parallel statements about facts in meetings, using different browsers. This paper offers a review of the results obtained so far with five types of meeting browsers, using similar sets of statements over the same meeting recordings. The results indicate that state-of-the-art speed for true/false question answering is 1.5-2 minutes per question, and precision is 70%-80% (vs. 50% random guess). The use of ASR compared to manual transcripts, or the use of audio signals only, lead to a perceptible though not dramatic decrease in performance scores.