REPORT smith:rr05-80/IDIAP Tracking the Multi Person Wandering Visual Focus of Attention Smith, Kevin C. Ba, Silèye O. Gatica-Perez, Daniel Odobez, Jean-Marc EXTERNAL http://publications.idiap.ch/attachments/reports/2005/smith-idiap-rr-05-80.pdf PUBLIC Idiap-RR-80-2005 2005 IDIAP To appear in International Conference on Multimodal Interfaces (ICMI'06) Estimating the {\em wandering visual focus of attention} (WVFOA) for multiple people is an important problem with many applications in human behavior understanding. One such application, addressed in this paper, monitors the attention of passers-by to outdoor advertisements. To solve the WVFOA problem, we propose a multi-person tracking approach based on a hybrid Dynamic Bayesian Network that simultaneously infers the number of people in the scene, their body and head locations, and their head pose, in a joint state-space formulation that is amenable for person interaction modeling. The model exploits both global measurements and individual observations for the VFOA. For inference in the resulting high-dimensional state-space, we propose a trans-dimensional Markov Chain Monte Carlo (MCMC) sampling scheme, which not only handles a varying number of people, but also efficiently searches the state-space by allowing person-part state updates. Our model was rigorously evaluated for tracking and its ability to recognize when people look at an outdoor advertisement using a realistic data set.