logo Idiap Research Institute        
 [BibTeX] [Marc21]
Can Self-Supervised Neural Representations Pre-Trained on Human Speech distinguish Animal Callers?
Type of publication: Conference paper
Citation: Sarkar_INTERSPEECH_2023
Publication status: Accepted
Booktitle: Proceedings of Interspeech
Year: 2023
Abstract: Self-supervised learning (SSL) models use only the intrinsic structure of a given signal, independent of its acoustic domain, to extract essential information from the input to an embedding space. This implies that the utility of such representations is not limited to modeling human speech alone. Building on this understanding, this paper explores the cross-transferability of SSL neural representations learned from human speech to analyze bio-acoustic signals. We conduct a caller discrimination analysis and a caller detection study on Marmoset vocalizations using eleven SSL models pre-trained with various pretext tasks. The results show that the embedding spaces carry meaningful caller information and can successfully distinguish the individual identities of Marmoset callers without fine-tuning. This demonstrates that representations pre-trained on human speech can be effectively applied to the bio-acoustics domain, providing valuable insights for future investigations in this field.
Projects Idiap
Authors Sarkar, Eklavya
Magimai.-Doss, Mathew
Added by: [UNK]
Total mark: 0
  • Sarkar_INTERSPEECH_2023.pdf