logo Idiap Research Institute        
 [BibTeX] [Marc21]
Understanding and Visualizing Raw Waveform-based CNNs
Type of publication: Conference paper
Citation: Muckenhirn_INTERSPEECH_2019
Booktitle: Proceedings of Interspeech
Year: 2019
Crossref: Muckenhirn_Idiap-RR-11-2018:
Abstract: Modeling directly raw waveforms through neural networks for speech processing is gaining more and more attention. Despite its varied success, a question that remains is: what kind of information are such neural networks capturing or learning for different tasks from the speech signal? Such an insight is not only interesting for advancing those techniques but also for understanding better speech signal characteristics. This paper takes a step in that direction, where we develop a gradient based approach to estimate the relevance of each speech sample input on the output score. We show that analysis of the resulting ``relevance signal" through conventional speech signal processing techniques can reveal the information modeled by the whole network. We demonstrate the potential of the proposed approach by analyzing raw waveform CNN-based phone recognition and speaker identification systems.
Keywords: CNN visualization, deep learning, raw waveforms
Projects Idiap
Authors Muckenhirn, Hannah
Abrol, Vinayak
Magimai.-Doss, Mathew
Marcel, S├ębastien
Added by: [UNK]
Total mark: 0
  • Muckenhirn_INTERSPEECH_2019.pdf