CONF norman:MMUA:2006/IDIAP Revisiting Doddington's Zoo: A Systematic Method to Assess User-dependent Variabilities Poh, Norman Bengio, Samy Ross, Arun EXTERNAL https://publications.idiap.ch/attachments/papers/2006/norman-MMUA-2006.pdf PUBLIC https://publications.idiap.ch/index.php/publications/showcite/norman:rr06-04 Related documents Multimodal User Authentication (MMUA) 2006 IDIAP-RR 06-04 A systematic analysis of user-dependent performance variability in the context of automatic speaker verification was first studied by Doddington \etal (1998). Different categories of users were distinguished and were called by animal names such as sheep, goats, lambs and wolves. Although such distinctions are important, it does not directly discriminate ``well-behaved'' users from ``badly behaved'' users. In our context, the badly behaved users are those who will bring the performance down when added to the system. We then extend such a study to formulate a user-specific score normalization (called F-norm's variant) and show that the user-dependent variability can be reduced to obtain an enhanced performance. By introducing some constraints, the proposed framework can also provide a stable user-dependent performance in terms of DET despite the fact that few (genuine) samples are available. In the context of multimodal biometrics, we show that it is possible to decide whether or not fusing the output of several systems is better than selecting any one of them, on a per user basis. This strategy is called an ``OR-switcher''. Based on 15 multimodal fusion experiments, the performance of OR-switcher is \emph{significantly} better than the state-of-the-art score-level fusion algorithms. REPORT norman:rr06-04/IDIAP Revisiting Doddington's Zoo: A Systematic Method to Assess User-dependent Variabilities Poh, Norman Bengio, Samy Ross, Arun EXTERNAL https://publications.idiap.ch/attachments/reports/2006/norman-idiap-rr-06-04.pdf PUBLIC Idiap-RR-04-2006 2006 IDIAP Published in {\it Multimodal User Authentication 2006} (MMUA2006,',','), Toulouse. A systematic analysis of user-dependent performance variability in the context of automatic speaker verification was first studied by Doddington \etal (1998). Different categories of users were distinguished and were called by animal names such as sheep, goats, lambs and wolves. Although such distinctions are important, it does not directly discriminate ``well-behaved'' users from ``badly behaved'' users. In our context, the badly behaved users are those who will bring the performance down when added to the system. We then extend such a study to formulate a user-specific score normalization (called F-norm's variant) and show that the user-dependent variability can be reduced to obtain an enhanced performance. By introducing some constraints, the proposed framework can also provide a stable user-dependent performance in terms of DET despite the fact that few (genuine) samples are available. In the context of multimodal biometrics, we show that it is possible to decide whether or not fusing the output of several systems is better than selecting any one of them, on a per user basis. This strategy is called an ``OR-switcher''. Based on 15 multimodal fusion experiments, the performance of OR-switcher is \emph{significantly} better than the state-of-the-art score-level fusion algorithms.