Skip to main navigation Skip to search Skip to main content

Ensembling End-to-End Deep Models for Computational Paralinguistics Tasks: ComParE 2020 Mask and Breathing Sub-Challenges

  • Maxim Markitantov
  • , Denis Dresvyanskiy
  • , Danila Mamontov
  • , H. Kaya
  • , Wolfgang Minker
  • , Alexey Karpov
  • Ulm University
  • St. Petersburg National Research University of Information Technologies, Mechanics and Optics (ITMO)
  • Institute for Informatics and Automation
  • Russian Academy of Sciences

Research output: Contribution to conference › Paper › Academic

Abstract

This paper describes deep learning approaches for the Mask and Breathing Sub-Challenges (SCs), which are addressed by the INTERSPEECH 2020 Computational Paralinguistics Challenge. Motivated by outstanding performance of state-of-the-art end-to-end (E2E) approaches, we explore and compare effectiveness of different deep Convolutional Neural Network (CNN) architectures on raw data, log Mel-spectrograms, and Mel-Frequency Cepstral Coefficients. We apply a transfer learning approach to improve model’s efficiency and convergence speed. In the Mask SC, we conduct experiments with several pretrained CNN architectures on log-Mel spectrograms, as well as Support Vector Machines on baseline features. For the Breathing SC, we propose an ensemble deep learning system that exploits E2E learning and sequence prediction. The E2E model is based on 1D CNN operating on raw speech signals and is coupled with Long Short-Term Memory layers for sequence modeling. The second model works with log-Mel features and is based on a pretrained 2D CNN model stacked to Gated Recurrent Unit layers. To increase performance of our models in both SCs, we use ensembles of the best deep neural models obtained from N-fold cross-validation on combined challenge training and development datasets. Our results markedly outperform the challenge test set baselines in both SCs.
Original languageEnglish
Pages2072-2076
Number of pages5
DOIs
Publication statusPublished - 25 Oct 2020
EventINTERSPEECH 2020 - Virtual Event, Shanghai, China
Duration: 25 Oct 2020 → 29 Oct 2020
http://interspeech2020.org/

Conference

ConferenceINTERSPEECH 2020
Abbreviated titleINTERSPEECH
Country/TerritoryChina
CityShanghai
Period25/10/20 → 29/10/20
Internet address

Keywords

  • machine learning
  • computational paralinguistics
  • speech processing
  • breathing prediction
  • information fusion
  • neural networks
  • transfer learning
  • end-to-end models

Fingerprint

Dive into the research topics of 'Ensembling End-to-End Deep Models for Computational Paralinguistics Tasks: ComParE 2020 Mask and Breathing Sub-Challenges'. Together they form a unique fingerprint.

Cite this