CONF Iuliia_INTERSPEECH_2023/IDIAP Implementing contextual biasing in GPU decoder for online ASR Iuliia, Nigmatulina Madikeri, Srikanth VILLATORO-TELLO, Esaú Motlicek, Petr Juan, Zuluaga-Gomez. S, Karthik Pandia D Ganapathiraju, Aravind Contextual Adaptation finite-state transducers GPU decoding real-time speech recognition EXTERNAL https://publications.idiap.ch/attachments/papers/2023/Iuliia_INTERSPEECH_2023.pdf PUBLIC https://publications.idiap.ch/index.php/publications/showcite/Iuliia_Idiap-RR-02-2023 Related documents Proc. Interspeech 2023 2023 GPU decoding significantly accelerates the output of ASR predictions. While GPUs are already being used for online ASR decoding, post-processing and rescoring on GPUs have not been properly investigated yet. Rescoring with available contextual information can considerably improve ASR predictions. Previous studies have proven the viability of lattice rescoring in decoding and biasing language model (LM) weights in offline and online CPU scenarios. In real-time GPU decoding, partial recognition hypotheses are produced without lattice generation, which makes the implementation of biasing more complex. The paper proposes and describes an approach to integrate contextual biasing in real-time GPU decoding while exploiting the standard Kaldi GPU decoder. Besides the biasing of partial ASR predictions, our approach also permits dynamic context switching allowing a flexible rescoring per each speech segment directly on GPU. The code is publicly released1 and tested with open-sourced test sets. REPORT Iuliia_Idiap-RR-02-2023/IDIAP Implementing contextual biasing in GPU decoder for online ASR Iuliia, Nigmatulina Madikeri, Srikanth VILLATORO-TELLO, Esaú Motlicek, Petr Juan, Zuluaga-Gomez. S, Karthik Pandia D Ganapathiraju, Aravind EXTERNAL https://publications.idiap.ch/attachments/reports/2023/Iuliia_Idiap-RR-02-2023.pdf PUBLIC Idiap-RR-02-2023 2023 Idiap May 2023 GPU decoding significantly accelerates the output of ASR predictions. While GPUs are already being used for online ASR decoding, post-processing and rescoring on GPUs have not been properly investigated yet. Rescoring with available contextual information can considerably improve ASR predictions. Previous studies have proven the viability of lattice rescoring in decoding and biasing language model (LM) weights in offline and online CPU scenarios. In real-time GPU decoding, partial recognition hypotheses are produced without lattice generation, which makes the implementation of biasing more complex. The paper proposes and describes an approach to integrate contextual biasing in real-time GPU decoding while exploiting the standard Kaldi GPU decoder. Besides the biasing of partial ASR predictions, our approach also permits dynamic context switching allowing a flexible rescoring per each speech segment directly on GPU. The code is publicly released1 and tested with open-sourced test sets.