logo Idiap Research Institute        
 [BibTeX] [Marc21]
Inducing Meaningful Units from Character Sequences with Dynamic Capacity Slot Attention
Type of publication: Journal paper
Citation: behjati2023inducing
Publication status: Published
Journal: Transactions on Machine Learning Research
Year: 2023
ISSN: 2835-8856
URL: https://openreview.net/forum?i...
Abstract: Characters do not convey meaning, but sequences of characters do. We propose an unsupervised distributional method to learn the abstract meaning-bearing units in a sequence of characters. Rather than segmenting the sequence, our Dynamic Capacity Slot Attention model discovers continuous representations of the objects in the sequence, extending an architecture for object discovery in images. We train our model on different languages and evaluate the quality of the obtained representations with forward and reverse probing classifiers. These experiments show that our model succeeds in discovering units which are similar to those proposed previously in form, content, and level of abstraction, and which show promise for capturing meaningful information at a higher level of abstraction.
Projects Idiap
Authors Behjati, Melika
Henderson, James
Added by: [UNK]
Total mark: 0
  • behjati2023inducing.pdf