logo Idiap Research Institute        
 [BibTeX] [Marc21]
Assessing the Accuracy of Discourse Connective Translations: Validation of an Automatic Metric
Type of publication: Conference paper
Citation: Hajlaoui_CICLING-2013_2013
Publication status: Published
Booktitle: 14th International Conference on Intelligent Text Processing and Computational Linguistics
Volume: 7817
Year: 2013
Month: March
Pages: 236-247
Publisher: Springer
Location: Samos, Greece
Organization: University of the Aegean
DOI: 10.1007/978-3-642-37256-8_20
Abstract: Automatic metrics for the evaluation of machine translation (MT) compute scores that characterize globally certain aspects of MT quality such as adequacy and fluency. This paper introduces a reference-based metric that is focused on a particular class of function words, namely discourse connectives, of particular importance for text structuring, and rather challenging for MT. To measure the accuracy of connective translation (ACT), the metric relies on automatic word-level alignment between a source sentence and respectively the reference and candidate translations, along with other heuristics for comparing translations of discourse connectives. Using a dictionary of equivalents, the translations are scored automatically, or, for better precision, semi-automatically. The precision of the ACT metric is assessed by human judges on sample data for English/French and English/Arabic translations: the ACT scores are on average within 2% of human scores. The ACT metric is then applied to several commercial and research MT systems, providing an assessment of their performance on discourse connectives.
Keywords: discourse connectives., Machine Translation, MT evaluation
Projects Idiap
Authors Hajlaoui, Najeh
Popescu-Belis, Andrei
Added by: [UNK]
Total mark: 0
  • Hajlaoui_CICLING-2013_2013.pdf