CONF gatica04a-conf/IDIAP Assessing Scene Structuring in Consumer Videos Gatica-Perez, Daniel Triroj, Napat Odobez, Jean-Marc Loui, Alexander Sun, Ming-Ting EXTERNAL http://publications.idiap.ch/attachments/reports/2004/rr04-11.pdf PUBLIC http://publications.idiap.ch/index.php/publications/showcite/gatica04a Related documents Int. Conf. on Image and Video Retrieval (CIVR) 2004 Scene structuring is a video analysis task for which no common evaluation procedures have been fully adopted. In this paper, we present a methodology to evaluate such task in home videos, which takes into account human judgement, and includes a representative corpus, a set of objective performance measures, and an evaluation protocol. The components of our approach are detailed as follows. First, we describe the generation of a set of home video scene structures produced by multiple people. Second, we define similarity measures that model variations with respect to two factors: human perceptual organization and level of structure granularity. Third, we describe a protocol for evaluation of automatic algorithms based on their comparison to human performance. We illustrate our methodology by assessing the performance of two recently proposed methods: probabilistic hierarchical clustering and spectral clustering. REPORT gatica04a/IDIAP Assessing Scene Structuring in Consumer Videos Gatica-Perez, Daniel Triroj, Napat Odobez, Jean-Marc Loui, Alexander Sun, Ming-Ting EXTERNAL http://publications.idiap.ch/attachments/reports/2004/rr04-11.pdf PUBLIC Idiap-RR-11-2004 2004 IDIAP Scene structuring is a video analysis task for which no common evaluation procedures have been fully adopted. In this paper, we present a methodology to evaluate such task in home videos, which takes into account human judgement, and includes a representative corpus, a set of objective performance measures, and an evaluation protocol. The components of our approach are detailed as follows. First, we describe the generation of a set of home video scene structures produced by multiple people. Second, we define similarity measures that model variations with respect to two factors: human perceptual organization and level of structure granularity. Third, we describe a protocol for evaluation of automatic algorithms based on their comparison to human performance. We illustrate our methodology by assessing the performance of two recently proposed methods: probabilistic hierarchical clustering and spectral clustering.