CONF
zhang-rr-04-24b/IDIAP
Multimodal Group Action Clustering in Meetings
Zhang, Dong
Gatica-Perez, Daniel
Bengio, Samy
McCowan, Iain A.
Lathoud, Guillaume
EXTERNAL
https://publications.idiap.ch/attachments/reports/2004/zhang-acm-04.pdf
PUBLIC
https://publications.idiap.ch/index.php/publications/showcite/zhang-rr-04-24
Related documents
ACM 2nd International Workshop on Video Surveillance & Sensor Networks in conjunction with 12th ACM International Conference on Multimedia
2004
IDIAP-RR 04-24
We address the problem of clustering multimodal group actions in meetings using a two-layer HMM framework. Meetings are structured as sequences of group actions. Our approach aims at creating one cluster for each group action, where the number of group actions and the action boundaries are unknown a priori. In our framework, the first layer models typical actions of individuals in meetings using supervised HMM learning and low-level audio-visual features. A number of options that explicitly model certain aspects of the data (e.g., asynchrony) were considered. The second layer models the group actions using unsupervised HMM learning. The two layers are linked by a set of probability-based features produced by the individual action layer as input to the group action layer. The methodology was assessed on a set of multimodal turn-taking group actions, using a public five-hour meeting corpus. The results show that the use of multiple modalities and the layered framework are advantageous, compared to various baseline methods.
REPORT
zhang-rr-04-24/IDIAP
Multimodal Group Action Clustering in Meetings
Zhang, Dong
Gatica-Perez, Daniel
Bengio, Samy
McCowan, Iain A.
Lathoud, Guillaume
EXTERNAL
https://publications.idiap.ch/attachments/reports/2004/rr-04-24.pdf
PUBLIC
Idiap-RR-24-2004
2004
IDIAP
Martigny, Switzerland
Published in ``ACM 2nd International Workshop on Video Surveillance & Sensor Networks in conjunction with 12th ACM International Conference on Multimedia'', October, 2004
We address the problem of clustering multimodal group actions in meetings using a two-layer HMM framework. Meetings are structured as sequences of group actions. Our approach aims at creating one cluster for each group action, where the number of group actions and the action boundaries are unknown a priori. In our framework, the first layer models typical actions of individuals in meetings using supervised HMM learning and low-level audio-visual features. A number of options that explicitly model certain aspects of the data (e.g., asynchrony) were considered. The second layer models the group actions using unsupervised HMM learning. The two layers are linked by a set of probability-based features produced by the individual action layer as input to the group action layer. The methodology was assessed on a set of multimodal turn-taking group actions, using a public five-hour meeting corpus. The results show that the use of multiple modalities and the layered framework are advantageous, compared to various baseline methods.