64-859 P Entfällt_Masterprojekt Deep Learning for Emotion Recognition and Synthesis

Lehrende: Navin Laxminarayanan Raj Prabhu

Veranstaltungsart: Projekt

Anzeige im Stundenplan: MProj - SP

Semesterwochenstunden: 6

Credits: 6,0

Unterrichtssprache: Englisch

Min. | Max. Teilnehmerzahl: - | 1

Kommentare/ Inhalte:
Conversational chatbots, personal assistants, and social robots, e.g., Siri, Alexa, ChatGPT, and Furhat robot, have become increasingly prominent in the consumer market over the past few years, but do any of them truly perceive our emotional expressions and respond to them like a human conversational partner would? The tasks of emotion recognition (perception) and synthesis (emotionally expressive response) precisely intend to solve this shortcoming of the latest conversational AI systems.

Emotions are expressed in a multimodal manner via social signals such as speech (audio), gestures (video), facial expressions (video), and lexical content (text). In this project, students will have an opportunity to design and develop multimodal deep learning algorithms for the recognition and synthesis of human emotional expressions.

Firstly, an introduction into the topic of emotion modelling and the respective deep learning techniques used will be given and relevant scientific literature will be discussed. Then, the concrete scope and the goals of the project on specific project ideas will be defined, and the students will draft and present a work plan with milestones. Examples of potential project ideas and topics are as follows:

1. Deep Learning for Emotion Recognition (either audio-based or audio-visual)
   
2. Emotion Speech Synthesis: Text-to-Speech with Expressive Emotions
   
3. Recognition and Synthesis of nonverbal behaviour for Social Robots: Through this topic students also have an opportunity to work on an actual social robot (The Furhat Robot)
   

Lernziel:
- expanding knowledge in the area of deep learning and multimodal signal processing
- independently comprehending a scientific topic
- translating scientific knowledge into practice within a project
- improving scientific writing and presentation skills
- improving teamwork

Vorgehen:
First, an introduction into the topic of social signal processing and affective computing will be given and relevant scientific literature will be discussed. Then, the concrete scope and the goals of the project will be defined and the students will draft and present a work plan with milestones. Periodically, while working on the project, status updates will be presented and the work plan will be adopted accordingly when necessary.

Literatur:
Literature recommendations will be provided at the start of the project.