Aligning Multilingual Word Embeddings for Cross-Modal Retrieval Task
| Type of publication: | Conference paper |
| Citation: | Mohammadshahi_EMNLP-FEVER_2019 |
| Publication status: | Published |
| Booktitle: | Proceedings of the Second Workshop on Fact Extraction and VERification (FEVER) |
| Year: | 2019 |
| Month: | November |
| Pages: | 27-33 |
| Publisher: | Association for Computational Linguistics |
| Location: | Hong Kong |
| Address: | Hong Kong, China |
| URL: | https://www.aclweb.org/antholo... |
| DOI: | 10.18653/v1/D19-6605 |
| Abstract: | In this paper, we propose a new approach to learn multimodal multilingual embeddings for matching images and their relevant captions in two languages. We combine two existing objective functions to make images and captions close in a joint embedding space while adapting the alignment of word embeddings between existing languages in our model. We show that our approach enables better generalization, achieving state-of-the-art performance in text-to-image and image-to-text retrieval task, and caption-caption similarity task. Two multimodal multilingual datasets are used for evaluation: Multi30k with German and English captions and Microsoft-COCO with English and Japanese captions. |
| Keywords: | |
| Projects |
Idiap |
| Authors | |
| Added by: | [UNK] |
| Total mark: | 0 |
|
Attachments
|
|
|
Notes
|
|
|
|
|