Comparing meeting browsers using a task-based evaluation method
| Type of publication: | Idiap-RR |
| Citation: | Popescu-Belis_Idiap-RR-11-2009 |
| Number: | Idiap-RR-11-2009 |
| Year: | 2009 |
| Month: | 6 |
| Institution: | Idiap |
| Abstract: | Information access within meeting recordings, potentially transcribed and augmented with other media, is facilitated by the use of meeting browsers. To evaluate their performance through a shared benchmark task, users are asked to discriminate between true and false parallel statements about facts in meetings, using different browsers. This paper offers a review of the results obtained so far with five types of meeting browsers, using similar sets of statements over the same meeting recordings. The results indicate that state-of-the-art speed for true/false question answering is 1.5-2 minutes per question, and precision is 70%-80% (vs. 50% random guess). The use of ASR compared to manual transcripts, or the use of audio signals only, lead to a perceptible though not dramatic decrease in performance scores. |
| Keywords: | |
| Projects |
Idiap IM2 AMIDA |
| Authors | |
| Added by: | [ADM] |
| Total mark: | 0 |
|
Attachments
|
|
|
|
|
Notes
|
|
|
|
|
