Task-based evaluation of meeting browsers: from BET task elicitation to user behavior analysis
| Type of publication: | Conference paper |
| Citation: | Popescu-Belis_LREC-2_2008 |
| Booktitle: | 6th International Conference on Language Resources and Evaluation |
| Year: | 2008 |
| Location: | Marrakech, Morocco |
| Abstract: | This paper presents recent results of the application of the task-based Browser Evaluation Test (BET) to meeting browsers, that is, interfaces to multimodal databases of meeting recordings. The tasks were defined by browser-neutral BET observers. Two groups of human subjects used the Transcript-based Query and Browsing interface (TQB,',','), and attempted to solve as many BET tasks – pairs of true/false statements to disambiguate – as possible in a fixed amount of time. Their performance was measured in terms of precision and speed. Results indicate that the browser’s annotation-based search functionality is frequently used, in particular the keyword search. A more detailed analysis of each test question for each participant confirms that despite considerable variation across strategies, the use of queries is correlated to successful performance. |
| Keywords: | |
| Projects |
Idiap AMIDA IM2 |
| Authors | |
| Added by: | [UNK] |
| Total mark: | 0 |
|
Attachments
|
|
|
|
|
Notes
|
|
|
|
|
