Looking at Hands: A new benchmark for Hand-Centered Gaze Understanding
| Type of publication: | Conference paper |
| Citation: | Haidar_ECCVW_2026 |
| Publication status: | Accepted |
| Booktitle: | Proceedings of the European Conference on Computer Vision (ECCV) Workshops (ABAW) |
| Year: | 2026 |
| Abstract: | Gaze following is typically evaluated through spatial target localization, but an accurate gaze point does not necessarily identify the hand involved in an interaction. We introduce Looking-at-Hands (LAHands), a multi-label task that identifies the specific hands associated with an observer's attention, including attention directed to actively manipulated objects. We provide LAHands annotations for GazeFollow, ChildPlay, and VideoAttentionTarget, together with calibrated pseudo-labels for large-scale GazeFollow training. We further propose MTGS-Hands, which augments multi-person gaze following with candidate-hand tokens encoding hand location, ownership, pose, and optional manipulation supervision. On GazeFollow, learned hand-aware association substantially outperforms the predicted-gaze geometric baseline while preserving competitive gaze localization. Experiments on ChildPlay further show that manipulation supervision can improve hand-attention prediction under domain adaptation. These results show that gaze-point proximity alone is insufficient for hand-directed attention prediction. The LAHands annotations will be available at https://www.idiap.ch/dataset/lahands. |
| Main Research Program: | Human-AI Teaming |
| Keywords: | Gaze Following, Hand-directed attention, Multi-person gaze and manipulation understanding |
| Projects: |
TUNASBE Idiap |
| Authors: | |
| Added by: | [UNK] |
| Total mark: | 0 |
|
Attachments
|
|
|
|
|
Notes
|
|
|
|
|
