Freshly published: Automatic Analysis of the Emotional Content of Speech in DaylongChild-Centered Recordings from a Neonatal Intensive Care Unit

Image by Gordon Johnson from Pixabay

Researchers have recently started to study how the emotional speech heard by young infants can affect their developmental outcomes. As a part of this research, hundreds of hours of daylong recordings from preterm infants’ audio environments were collected from two hospitals in Finland and Estonia in the context of so-called APPLE study.

In order to analyze the emotional content of speech in such a massive dataset, an automatic Speech Emotion Recognition (SER) system is required. However, there are no emotion labels or existing SER systems in the domain to be used for this purpose. In this paper, the authors (including our consortium team member K. Drosos from Tampere University (TAU)) introduce this initially unannotated large-scale real-world audio dataset. They describe the development of a functional SER system for the Finnish subset of the data. They also explore the effectiveness of alternative state-of-the-art techniques to deploy a SER system to a new domain, comparing cross-corpus generalization, WGAN-based domain adaptation, and active learning in the task. As a result, they show that the best-performing models are able to achieve a classification performance of 73.4 for valence and arousal, respectively. The results also show that active learning achieves the most consistent performance compared to the two alternatives.

The results of the paper are highly relevant to the research being performed in MARVEL for extreme scale audio visual analytics. Such techniques can be leveraged in the context of smart cities like for example in cases of crowd observation and crowd monitoring. The paper has been accepted and will be presented at the Intespeeech conference which is taking place on the 30th of August until the 3rd of September in Brno, Czech Republic. You can read it here.

For more direct updates regarding our project as well as the topics we are studying and more, follow us on LinkedIn and on Twitter. For regular updates including a collection of news and relevant information sign up on our Newsletter below.



[email-subscribers-form id="1"]

Key Facts

  • Project Coordinator: Dr. Sotiris Ioannidis
  • Institution: Foundation for Research and Technology Hellas (FORTH)
  • E-mail: 
  • Start: 01.01.2021
  • Duration: 36 months
  • Participating Organisations: 17
  • Number of countries: 12

Get Connected



This project has received funding from the European Union’s Horizon 2020 Research and Innovation program under grant agreement No 957337. The website reflects only the view of the author(s) and the Commission is not responsible for any use that may be made of the information it contains.