CONF Henderson_ACL2020_2020/IDIAP The Unstoppable Rise of Computational Linguistics in Deep Learning Henderson, James Association for Computational Linguistics - Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics Online 2020 Association for Computational Linguistics 6294-6306 https://www.aclweb.org/anthology/2020.acl-main.561 URL 10.18653/v1/2020.acl-main.561 doi In this paper, we trace the history of neural networks applied to natural language understanding tasks, and identify key contributions which the nature of language has made to the development of neural network architectures. We focus on the importance of variable binding and its instantiation in attention-based models, and argue that Transformer is not a sequence model but an induced-structure model. This perspective leads to predictions of the challenges facing research in deep learning architectures for natural language understanding.