CONF Jiang_WMT_2025/IDIAP Meaningful Pose-Based Sign Language Evaluation Jiang, Zifan Leong, Colin Moryossef, Amit Cory, Oliver Ivashechkin, Maksym Tarigopula, Neha Zhang, Biao Göhring, Anne Rios, Annette Sennrich, Rico Ebling, Sarah Proceedings of the Tenth Conference on Machine Translation (WMT) 2025 https://aclanthology.org/2025.wmt-1.4.pdf URL 10.18653/v1/2025.wmt-1.4 doi We present a comprehensive study on meaningfully evaluating sign language utterances in the form of human skeletal poses. The study covers keypoint distance-based, embedding-based, and back-translation-based metrics. We show tradeoffs between different metrics in different scenarios through (1) automatic meta-evaluation of sign-level retrieval, and (2) a human correlation study of text-to-pose translation across different sign languages. Our findings, along with the open-source pose-evaluation toolkit, provide a practical and reproducible approach for developing and evaluating sign language translation or generation systems.