logo Idiap Research Institute        
 [BibTeX] [Marc21]
Continuously Reproducing Toolchains in Pattern Recognition and Machine Learning Experiments
Type of publication: Conference paper
Citation: Anjos_ICML2017-2_2017
Publication status: Accepted
Booktitle: Thirty-fourth International Conference on Machine Learning
Year: 2017
Month: August
Location: Sidney, Australia
Note: https://openreview.net/group?id=ICML.cc/2017/RML
URL: https://www.idiap.ch/software/...
Abstract: Pattern recognition and machine learning research work often contains experimental results on real-world data, which corroborates hypotheses and provides a canvas for the development and comparison of new ideas. Results, in this context, are typically summarized as a set of tables and figures, allowing the comparison of various methods, highlighting the advantages of the proposed ideas. Unfortunately, result reproducibility is often an overlooked feature of original research publications, competitions, or benchmark evaluations. The main reason for such a gap is the complexity on the development of software associated with these reports. Software frameworks are difficult to install, maintain, and distribute, while scientific experiments often consist of many steps and parameters that are difficult to report. The increasingly rising complexity of research challenges make it even more difficult to reproduce experiments and results. In this paper, we emphasize that a reproducible research work should be repeatable, shareable, extensible, and stable, and discuss important lessons we learned in creating, distributing, and maintaining software and data for reproducible research in pattern recognition and machine learning. We focus on a specific use-case of face recognition and describe in details how we can make the recognition experiments reproducible in practice.
Keywords:
Projects BEAT
SWAN
Authors Anjos, André
Günther, Manuel
de Freitas Pereira, Tiago
Korshunov, Pavel
Mohammadi, Amir
Marcel, Sébastien
Added by: [UNK]
Total mark: 0
Attachments
  • Anjos_ICML2017-2_2017.pdf
Notes