CONF Prasad_SID-2_2022/IDIAP Speech and Natural Language Processing Technologies for Pseudo-Pilot Simulator Prasad, Amrutha Zuluaga-Gomez, Juan Motlicek, Petr Sarfjoo, Seyyed Saeed Iuliia, Nigmatulina Vesely, Karel EXTERNAL http://publications.idiap.ch/attachments/papers/2022/Prasad_SID-2_2022.pdf PUBLIC 12th SESAR Innovation Days 2022 This paper describes a simple yet efficient repetition based modular system for speeding up air-traffic controllers (ATCos) training. E.g., a human pilot is still required in EUROCONTROL’s ESCAPE lite simulator https://www.eurocontrol.int/simulator/escape during ATCo training. However, this need can be substituted by an automatic system that could act as a pilot. In this paper, we aim to develop and integrate a pseudo-pilot agent into the ATCo training pipeline by merging diverse artificial intelligence (AI) powered modules. The system understands the voice communications issued by the ATCo, and, in turn, it generates a spoken prompt that follows the pilot’s phraseology to the initial communication. Our system mainly relies on open-source AI tools and air traffic control (ATC) databases, thus, proving its simplicity and ease of replicability. The overall pipeline is composed of the following: (1) a submodule that receives and pre-processes the input stream of raw audio, (2) an automatic speech recognition (ASR) system that transforms audio into a sequence of words; (3) a high-level ATCrelated entity parser, which extracts relevant information from the communication, i.e., callsigns and commands, and finally, (4) a speech synthesizer submodule that generates responses based on the high-level ATC entities previously extracted. Overall, we show that this system could pave the way toward developing a real proof-of-concept pseudo-pilot system. Hence, speeding up the training of ATCos while drastically reducing its overall cost.