logo Idiap Research Institute        
 [BibTeX] [Marc21]
Learning Entailment-Based Sentence Embeddings from Natural Language Inference
Type of publication: Idiap-RR
Citation: rabeehkarimimahabadi_Idiap-RR-20-2019
Number: Idiap-RR-20-2019
Year: 2019
Month: 12
Institution: Idiap
Note: project SNSF-LAOS
Abstract: Large datasets on natural language inference are a potentially valuable resource for inducing semantic representations of natural language sentences. But in many such models the embeddings computed by the sentence encoder goes through an MLP-based interaction layer before predicting its label, and thus some of the information about textual entailment is encoded in the interpretation of sentence embeddings given by this parameterised MLP. In this work we propose a simple interaction layer based on predefined entailment and contradiction scores applied directly to the sentence embeddings. This parameter-free interaction model achieves results on natural language inference competitive with MLP-based models, demonstrating that the trained sentence embeddings directly represent the information needed for textual entailment. The inductive bias of this model leads to better generalisation to other natural language inference datasets, and the resulting sentence embeddings are better for other semantic tasks.
URL: https://openreview.net/forum?i...
Keywords:
Projects Idiap
Authors Karimi Mahabadi, Rabeeh
Mai, Florian
Henderson, James
Added by: [ADM]
Total mark: 0
Attachments
  • rabeehkarimimahabadi_Idiap-RR-20-2019.pdf (MD5: d97af78a1df5a578f92846dd0e6b481a)
Notes