Research project Let's talk about non-verbal communication
This project investigates interpersonal psychotherapeutic interactions and their effect on treatment outcomes using AI and time series analysis.
According to a WHO report published in 2011, mental ill-health is expected to be the main cause of illness and death in the world by 2030. The need for effective psychotherapies can therefore be expected to become ever more urgent over the next few decades.
There has been very little research into the factors that make a therapist competent and psychotherapy effective. But there is a growing consensus in psychotherapy research that psychotherapy can be studied as dyadic dynamic systems. From this viewpoint, it is important to explore the verbal as well as the non-verbal communication patterns occurring over time between client and therapist. This interpersonal communication is highly complex, since both parties are simultaneously sender and recipient of non-verbal and verbal messages. Verbal communication is readily accessible for analysis, but to understand interpersonal communication in its entirety, non-verbal communication must also be analyzed systematically, and be linked to the verbal content.
Non-verbal communication is multimodal and includes body and head position, eye movements, emotional prosody and physiological variables such as heart activity. These non-verbal signals also form part of the complex interactional synchronization patterns that include mirroring of emotional expressions. Following this complex communication dynamic is seldom possible, even for trained observers.
The researchers in the project intend to use new digital technology to measure the above non-verbal variables in an objective manner. Data will be analyzed using methods such as machine learning and time series analyses to identify the non-verbal behavioral patterns capable of predicting therapy outcomes. This will pave the way for completely new methods of analyzing the interpersonal processes that are key to the outcome of psychotherapy, and a better understanding of them.
One overall aim is to fill the gaps in our knowledge about non-verbal communication and its relationship to verbal communication components and to treatment outcomes. This new knowledge is expected to improve training in psychotherapy, which will in turn produce better trained psychotherapists and more effective therapies, as well as lower levels of mental ill-health and reduced costs for society as a whole.
Full name of the project: Let's talk about non-verbal communication: Investigation of interpersonal psychotherapeutic interactions and their effect on treatment outcomes using AI and time series analysis.
Project description
In this groundbreaking interdisciplinary project the fields of psychology and data science are brought together. The combination of research and evaluation methods from the field of data science with the research methods of psychotherapy research, as well as the combination of verbal and non-verbal data in the evaluation strategy, creates an innovative progression in this interdisciplinary field.
Psychotherapy is conceptualized as a hypercomplex interactional process, involving different communication modalities and affective behavior which develops over time. From a data science viewpoint, it is always a challenge to build predictive models from data sources of heterogeneous nature, multimodality, and high complexity. At the same time, the field of psychology calls for machine learning methods that can assist in discovering new knowledge and defining new practices when it comes to improving the psychotherapeutic approaches.
From the psychotherapy research viewpoint, new developments in data science, especially with respect to applying AI-based evaluation procedures of complex data over time makes it possible for the first time to investigate data extracted from the hyper-complex human interaction processes in psychotherapy in an adequate way. In addition, this research project breaks new ground with respect to the combination of analysis of verbal and nonverbal data. The objective measures of non-verbal interaction variables, that enables psychotherapists to reveal interactive patterns, may not only have an impact on outcome of importance for psychotherapy process researchers but also for any kind of researchers in the field of humanities investigating human interaction processes.
Prevention and early intervention efforts are necessary to lessen the disease burden and the economic costs associated with mental disorders (Beames 2021). Current treatments are estimated only to reduce about one-third of the disease burden (van Zoonen 2014). One possible reason is that there is no sufficient knowledge of the interactions and interactive communication features involved in therapist and patient interpersonal communication. For example, the therapist may well be aware of many of the patient’s non-verbal signals, but it may not be possible even for a highly trained and experienced person to pay attention to all of the subtle signals, including micro-affects and own non-verbal signals. We want to investigate how increased knowledge about non-verbal communication and interactive patterns in psychotherapies can lead to improved psychotherapy education and better trained psychotherapists that can perform better working treatments. This, in turn, would lead to less psychological ill-health in the population, i.e., to better life quality for a large group of individuals and to reduced costs for society as a whole.
Machine learning techniques have been recently employed in psychology research for identifying generalizable predictors for diagnosis, treatment, and decision support (for a scoping review, see Shatte 2019). Moreover, evidence shows that the effectiveness of existing prevention and early intervention approaches is greater when delivered early. Hence, there is an evident need for effective and interpretable machine learning models for the early prediction and prevention of mental disorders. The aims of this project are to use explainable artificial intelligence (XAI) and time-series technology to:
- Systematically document relevant dimensions involved in psychotherapeutic communication, e.g., affective expressions as well as nonverbal communication.
- Investigate interactive patterns and reciprocal effects of non-verbal behavior in dyadic interactions.
- Investigate the effects of nonverbal communication on therapy outcome.
- Investigate the development and processing of the so-called “working alliance” based on non-verbal dyadic communication features.
- Investigate the dynamic features of the “working alliance” as a continuously changing relational tool relevant for treatment outcome.
- Investigate early process indicators within the patient-therapist dyad with predictive potential for the outcome of psychotherapy.
- Develop a training model including relevant parameters for improving psychotherapeutic capacities to improve the working alliance.
- Improve psychotherapy education by suggesting new educational techniques in the different psychotherapy education programs in the Nordic countries.
Project members
Project managers
Stephan Hau
Professor
Members
Therese Anderbro
Assistant Professor
Nora Choque Olsson
Associate Professor, Lic. Psychologist, Lic. Psychotherapist, Specialist in Clinical Psychology
Lillian Döllinger
Lecturer
Håkan Fischer
Professor in Human biological psychology
Lennart Högman
Assistant Professor
Petri Laukka
Professor
Ioanna Miliou
Senior lecturer
Panagiotis Papapetrou
Professor, deputy head of department
Luis Eduardo Velez Quintero
Utbildningsassistent
Thomas Jack Samuels
Gäst
