CONF Kumar_SLT2026-2_2026/IDIAP Abstract Compression of Conversational Audio Context for LLM-based ASR Kumar, Shashi Villatoro-Tello, Esaú Burdisso, Sergio Hacioğlu, Kadri Bañeras-Roux, Thibault Watawana, Hasindri Sankalpana Sanchez-Cortes, Dairazalia Madikeri, Srikanth Motlicek, Petr Stolcke, Andreas EXTERNAL https://publications.idiap.ch/attachments/papers/2026/Kumar_SLT2026-2_2026.pdf PUBLIC IEEE Spoken Language Technology 2026 2026 https://arxiv.org/abs/2603.26246 URL Standard LLM-based speech recognition systems process utterances in isolation, limiting their ability to leverage conversational context. Here we study whether multimodal context from prior turns improves LLM-based ASR and how to represent that context efficiently. We find that, after supervised multi-turn training, conversational context mainly helps with the recognition of contextual entities. However, conditioning on raw context is expensive because the prior-turn audio embedding sequence grows rapidly with conversation length. To address this, we propose Abstract Compression, a learned latent abstraction of prior-turn audio that replaces each prior-turn audio sequence with a fixed number of latent tokens while retaining corresponding transcripts explicitly. On both in-domain and out-of-domain test sets, the compression-enabled model recovers part of the gains of raw-context conditioning with a smaller prior-turn audio footprint. We also provide targeted analyses of the compression setup and its tradeoffs.