logo Idiap Research Institute        
 [BibTeX] [Marc21]
Gradient-based spectral visualization of CNNs using raw waveforms
Type of publication: Idiap-RR
Citation: Muckenhirn_Idiap-RR-11-2018
Number: Idiap-RR-11-2018
Year: 2018
Month: 7
Institution: Idiap
Abstract: Modeling directly raw waveform through neural networks for speech processing is gaining more and more attention. Despite its varied success, a question that remains is: what kind of information are such neural networks capturing or learning for different tasks from the speech signal? Such an insight is not only interesting for advancing those techniques but also for understanding better speech signal characteristics. This paper takes a step in that direction, where we develop a gradient based approach to estimate the relevance of each speech sample input on the output score. We show that analysis of the resulting ``relevance signal" through conventional speech signal processing techniques can reveal the information modeled by the whole network. We demonstrate the potential of the proposed approach by analyzing raw waveform CNN-based phone recognition and speaker identification systems.
Projects Idiap
Authors Muckenhirn, Hannah
Abrol, Vinayak
Magimai.-Doss, Mathew
Marcel, S├ębastien
Crossref by Muckenhirn_INTERSPEECH_2019
Added by: [ADM]
Total mark: 0
  • Muckenhirn_Idiap-RR-11-2018.pdf (MD5: 8e0e8ad9e7657970297e951a1be8ceb0)