REPORT barber:hypothesis:rr:04:57/IDIAP Are two Classifiers performing equally? A treatment using Bayesian Hypothesis Testing Barber, David EXTERNAL http://publications.idiap.ch/attachments/reports/2004/barber_dirichlet_test_idiap_rr.pdf PUBLIC Idiap-RR-57-2004 2004 IDIAP Rue de Simplon 4, Martigny, CH-1920, Switerland May 2004 IDIAP-RR 04-57 We consider here how to assess if two classifiers, based on a set of test error results, are performing equally well. This question is often considered in the realm of sampling theory, based on classical hypothesis testing. Here we present a simple Bayesian treatment that is quite general, and also is able to deal with the (practically common) case where the errors that two classifiers make are dependent.