CONF Parra-Gallego_INTERSPEECH_2024/IDIAP Cross-transfer Knowledge between Speech and Text Encoders to Evaluate Customer Satisfaction Parra-Gallego, Luis Felipe Purohit, Tilak Vlasenko, Bogdan Orozco-Arroyave, Juan Rafael Magimai.-Doss, Mathew cross-transfer knowledge Customer satisfaction Spoken Language Understanding EXTERNAL https://publications.idiap.ch/attachments/papers/2024/Parra-Gallego_INTERSPEECH_2024.pdf PUBLIC Interspeech Kos Island, Greece 2024 ISCA Customer Satisfaction (CS) in call centers influences customer loyalty and the company's reputation. Traditionally, CS evaluations were conducted manually or with classical machine learning algorithms; however, advancements in deep learning have led to automated systems that evaluate CS using speech and text analyses. Previous studies have shown the text approach to be more accurate but relies on an external ASR for transcription. This study introduces a cross-transfer knowledge technique, distilling knowledge from the BERT model into speech encoders like Wav2Vec2, WavLM, and Whisper. By enriching these encoders with BERT’s linguistic information, we improve speech analysis performance and eliminate the need for an ASR. In evaluations on a dataset of customer opinions, our methods achieve over 92% accuracy in identifying CS categories, providing a faster and cost-effective solution compared to traditional text approaches.