ARTICLE Ohneiser_AEROSPACE_2023/IDIAP Assistant Based Speech Recognition Support for Air Traffic Controllers in a Multiple Remote Tower Environment Ohneiser, Oliver Helmke, Hartmut Shetty, Shruthi Kleinert, Matthias Ehr, heiko Schier-Morgenthal, Sebastian Sarfjoo, Seyyed Saeed Motlicek, Petr Šarūnas Murauskas, Pagirys, Tomas Usanovic, Haris Meštrović, Mirta Černá, Aneta air traffic controller Assistant Based Speech Recognition automatic speech recognition and understanding electronic flight strips multiple remote tower EXTERNAL http://publications.idiap.ch/attachments/papers/2023/Ohneiser_AEROSPACE_2023.pdf PUBLIC Aerospace 10 6 2023 https://www.mdpi.com/2226-4310/10/6/560 URL https://doi.org/10.3390/aerospace10060560 doi Assistant Based Speech Recognition (ABSR) systems for air traffic control radiotelephony communication have shown their potential to reduce air traffic controllers’ (ATCos) workload. Related research activities mainly focused on utterances for approach and en-route traffic. This is one of the first investigations of how ABSR could support ATCos in a tower environment. Ten ATCos from Lithuania and Austria participated in a human-in-the-loop simulation to validate ABSR support within a prototypic multiple remote tower controller working position. The ABSR supports ATCos by (1) highlighting recognized callsigns, (2) inputting recognized commands from ATCo utterances in electronic flight strips, (3) offering correction of ABSR output, (4) automatically accepting ABSR output, and (5) feeding the digital air traffic control system. This paper assesses human factors such as workload, situation awareness, and usability when ATCos are supported by ABSR. Those assessments result from a system with a relevant command recognition rate of 82.9% and a callsign recognition rate of 94.2%. Workload reductions and usability improvement with p-values below 0.25 are obtained for the case when the ABSR system is compared to the baseline situation without ABSR support. This motivates the technology to be brought to a higher technology readiness level, which is also confirmed by subjective feedback from questionnaires and objective measurement of workload reduction based on a performed secondary task.