Meaningful Pose-Based Sign Language Evaluation
| Type of publication: | Conference paper |
| Citation: | Jiang_WMT_2025 |
| Publication status: | Published |
| Booktitle: | Proceedings of the Tenth Conference on Machine Translation (WMT) |
| Year: | 2025 |
| URL: | https://aclanthology.org/2025.... |
| DOI: | 10.18653/v1/2025.wmt-1.4 |
| Abstract: | We present a comprehensive study on meaningfully evaluating sign language utterances in the form of human skeletal poses. The study covers keypoint distance-based, embedding-based, and back-translation-based metrics. We show tradeoffs between different metrics in different scenarios through (1) automatic meta-evaluation of sign-level retrieval, and (2) a human correlation study of text-to-pose translation across different sign languages. Our findings, along with the open-source pose-evaluation toolkit, provide a practical and reproducible approach for developing and evaluating sign language translation or generation systems. |
| Main Research Program: | Human-AI Teaming |
| Additional Research Programs: |
AI for Everyone |
| Keywords: | |
| Projects: |
Idiap SMILE-II |
| Authors: | |
| Added by: | [UNK] |
| Total mark: | 0 |
|
Attachments
|
|
|
Notes
|
|
|
|
|