How to Leverage Synthetic Speech for LLM-Based ASR Systems?
| Type of publication: | Conference paper |
| Citation: | Labrak_SLT2026_2026 |
| Publication status: | Accepted |
| Booktitle: | IEEE Spoken Language Technology 2026 |
| Year: | 2026 |
| URL: | https://arxiv.org/abs/2606.290... |
| Abstract: | In regulated domains such as banking and healthcare, where privacy makes real speech costly to collect, synthetic speech from text-to-speech is an appealing alternative for training automatic speech recognition (ASR). Yet a distributional gap limits how far synthetic data can replace real recordings. While prior work treats this gap as a black box to engineer around, we probe its origin in a SLAM-ASR architecture. We localise the signal separating real from synthetic speech to early-to-middle layers of the LLM backbone, where temporal and prosodic perturbations disrupt it most. Representation-level separability helps but does not directly predict downstream ASR gains. Convolving synthetic audio with room impulse responses (RIR) narrows the gap not by making speech more natural, but by reproducing the acoustic irregularities of real recordings. Combining a layer-selection module with RIR augmentation matches an all-real baseline using only 25% of real speech (13.6h) and surpasses it at all higher proportions. |
| Main Research Program: | AI for Everyone |
| Additional Research Programs: |
AI for Life |
| Keywords: | |
| Projects: |
UNIPHORE ELOQUENCE |
| Authors: | |
| Added by: | [UNK] |
| Total mark: | 0 |
|
Attachments
|
|
|
|
|
Notes
|
|
|
|
|
