Research project Prosody in conversation
The purpose of this project was to deepen our knowledge about prosodic characteristics that are specific for conversation with the larger goal of creating an artificial interlocutor.
Today's speech technology is deficient when it comes to interpreting human conversational behavior. The purpose of this project, therefore, was to deepen our knowledge about prosodic characteristics that are specific for conversation, and that have an interactional function, with the larger goal of creating an artificial conversational partner.
Project description
The main track of the project has been to investigate and model rhythmic patterns and intonation patterns in connection to turntaking and verbal feedback in different Swedish and English spoken language databases. The
project has also made efforts to verify effects of observed prosodic characteristics through perception and production tests, as well as through the generation of interactional behaviours in human-robot interaction.
Three important results
The three most important results of the project are summarized in these bullets:
- Quantitative descriptions of the prosody of conversation, e.g. timing of feedback vocalizations/backchannels in relation to the speech of the interlocutor, prosodic patterns inviting feedback, how frequent feedback may occur, prosodic realisation of feedback and how these vocalizations are adapted to the prosody of the interlocutor.
- Stochastic models of turntaking in interaction given prosodic characteristics, e.g. speech, silence, overlap, speaking rate change, intonation patterns, intensity patterns) providing a framework for an artificial speaker to understand and produce a more humanlike conversational behaviour.
- Implementation of prosodic behaviour in a physical realisation of an embodied conversational agent (ECA) for verification of the effects of conversation specific prosodic characteristics.
Taken together, these three items represent a large step forward on our long-term goal to create an artificial conversational partner that actually sounds and behaves as if it is participating in a conversation.
Project members
Project managers
Mattias Heldner
Professor, Head of Department
Members
Kornel Laskowski
Lecturer
Jens Edlund
Associate professor
Anna Hjalmarsson
Researcher
Publications
Publication list