REPORT mccowan-rr-04-73/IDIAP On the Use of Information Retrieval Measures for Speech Recognition Evaluation McCowan, Iain A. Moore, Darren Dines, John Gatica-Perez, Daniel Flynn, Mike Wellner, Pierre Bourlard, Hervé EXTERNAL PUBLIC Idiap-RR-73-2004 2004 IDIAP Martigny, Switzerland This paper discusses the evaluation of automatic speech recognition (ASR) systems developed for practical applications, suggesting a set of criteria for application-oriented performance measures. The commonly used word error rate (WER,',','), which poses ASR evaluation as a string editing process, is shown to have a number of limitations with respect to these criteria, motivating alternative or additional measures. This paper suggests that posing speech recognition evaluation as an information retrieval problem, where each word is one unit of information, offers a flexible framework for application-oriented performance analysis based on the concepts of recall and precision.