logo Idiap Research Institute        
 [BibTeX] [Marc21]
Aligning Multilingual Word Embeddings for Cross-Modal Retrieval Task
Type of publication: Conference paper
Citation: Mohammadshahi_EMNLP-FEVER_2019
Publication status: Published
Booktitle: Proceedings of the Second Workshop on Fact Extraction and VERification (FEVER)
Year: 2019
Month: November
Pages: 27-33
Publisher: Association for Computational Linguistics
Location: Hong Kong
Address: Hong Kong, China
URL: https://www.aclweb.org/antholo...
DOI: 10.18653/v1/D19-6605
Abstract: In this paper, we propose a new approach to learn multimodal multilingual embeddings for matching images and their relevant captions in two languages. We combine two existing objective functions to make images and captions close in a joint embedding space while adapting the alignment of word embeddings between existing languages in our model. We show that our approach enables better generalization, achieving state-of-the-art performance in text-to-image and image-to-text retrieval task, and caption-caption similarity task. Two multimodal multilingual datasets are used for evaluation: Multi30k with German and English captions and Microsoft-COCO with English and Japanese captions.
Projects Idiap
Authors Mohammadshahi, Alireza
Aberer, Karl
Lebret, RĂ©mi
Added by: [UNK]
Total mark: 0