CONF Popescu-Belis_LREC-2_2008/IDIAP Task-based evaluation of meeting browsers: from BET task elicitation to user behavior analysis Popescu-Belis, Andrei Flynn, Mike Wellner, Pierre Baudrion, Philippe EXTERNAL PUBLIC 6th International Conference on Language Resources and Evaluation Marrakech, Morocco 2008 This paper presents recent results of the application of the task-based Browser Evaluation Test (BET) to meeting browsers, that is, interfaces to multimodal databases of meeting recordings. The tasks were defined by browser-neutral BET observers. Two groups of human subjects used the Transcript-based Query and Browsing interface (TQB,',','), and attempted to solve as many BET tasks – pairs of true/false statements to disambiguate – as possible in a fixed amount of time. Their performance was measured in terms of precision and speed. Results indicate that the browser’s annotation-based search functionality is frequently used, in particular the keyword search. A more detailed analysis of each test question for each participant confirms that despite considerable variation across strategies, the use of queries is correlated to successful performance.