Therapy Meets Metrics: How KPIs Make Success Visible in Blended Psychotherapy

In an interdisciplinary collaboration between the start-up YLAH and the Departments of Engineering and Computer Science as well as Health Science at the Bern University of Applied Sciences, KPI-based metrics in blended therapy were investigated.

Context

Blended Psychotherapy, the combination of digital and analog therapy elements, is gaining increasing importance. But how can the success of such treatments be meaningfully measured? The answer lies in well-designed metrics – so-called Key Performance Indicators (KPIs).

A strategic thematic field of the Bern University of Applied Sciences [1] is humane digital transformation. People should be placed at the center of all efforts surrounding the development of new technologies, leading to responsible implementation and greater acceptance. As part of this thematic field, innovation vouchers are regularly awarded: Companies can benefit from BFH’s expertise in this subject area and apply for such a voucher. The start-up YLAH successfully did this, which enabled the interdisciplinary project described in this article to be carried out.

An Assessment with Quadruple Aims

The goal of KPIs in the context of Blended Psychotherapy is to recognize patterns through systematic feedback, make progress visible, and continuously improve therapy. Different perspectives play a role: How does the patient perceive herself? How does the therapist assess the relationship? What experiences were made with the digital platform? From these puzzle pieces emerges a complex picture that is far more than the sum of its parts.

As part of the project, researchers examined which instruments are suitable for measuring KPIs. The focus was on the Quadruple Aims [2] , covering the following four areas: therapy success from the patients’ perspective (PROMs), their experiences with the therapy (PREMs), economic aspects (costs), and feedback from the professional staff (Staff Reported Outcomes).

Among the most important tools are the Patient Health Questionnaire (PHQ-9) for measuring depressive symptoms and the GAD-7 for anxiety symptoms. Both are compact, validated, and already integrated into the product. For the subjective experience of therapy, the WAI-SR-P and WAI-SR-T scales are suitable, which capture the therapeutic alliance from the perspectives of patients and therapists. Particularly interesting: Comparing both perspectives can provide valuable insights into the quality of the relationship, which is a central factor for therapeutic success. Economic indicators also play a role, for example through the calculation of so-called QALYs (“Quality Adjusted Life Years”), which represent the healthy years of life gained through therapy. Additionally, the TIC-P instrument can help capture illness-related costs. It’s important to note: Not all data can be collected continuously. Therefore, a meaningful KPI selection requires a balance between effort, informative value, and practical relevance. These should ideally be visualized in clear dashboards where different types of data are coordinated with each other.

Metrics Without User Interventions

In addition, technical metrics can also be collected indirectly, meaning without user interventions. These are based on “proxies” that provide insights into acceptance, perceived benefit, and user engagement through user behavior in the application.

In the literature, there are numerous metrics and measurement methods to quantitatively assess user engagement on web platforms [3] . However, the specific application and interpretation can vary greatly depending on the type of website and task. Consequently, there is no “one-fits-all metric” for engagement analyses [4] .

As part of this project, it was therefore specifically investigated for ylah®’s use case how such measurements can be conducted and evaluated using the company’s own data and locally in a protected environment.

References

1 BFH Strategic thematic field https://www.bfh.ch/de/strategische-themenfelder/digitalisierung/

2 Sikka R, Morath JM, Leape L The Quadruple Aim: care, health, cost and meaning in work

BMJ Quality & Safety 2015;24:608-610. https://qualitysafety.bmj.com/content/24/10/608

3 Lalmas, Mounia & O’Brien, Heather & Yom-Tov, Elad. (2014). Measuring User Engagement. 10.2200/S00605ED1V01Y201410ICR038.

4 Lehmann, Janette & Lalmas, Mounia & Yom-Tov, Elad & Dupret, Georges. (2012). Model of User Engagement. 7379. 10.1007/978-3-642-31454-4_14.

ylah® is an approved medical device and the first Swiss platform for Blended Psychotherapy. Our goal is to make psychotherapeutic treatments more efficient and accessible. ylah® combines personal therapy with digital tools and enables seamless integration of therapy into everyday life – innovative, flexible, and evidence-based. https://ylah.ch

Healthcare and Personnel Development innovation field https://www.bfh.ch/de/forschung/forschungsbereiche/innovationsfeld-gesundheitsversorgung-personalentwicklung/

BFH Applied Machine Intelligence https://bfh.ch/ami

BFH Generative AI Lab https://www.bfh.ch/ti/de/aktuell/generative-ai-lab/

Christoph Golz is Head of Healthcare Innovation Field - Human Resources Development at the Bern University of Applied Sciences Health. His research focuses on the future-oriented and demand-driven development of health care.

Yannis Schmutz is a research associate at the Generative AI Lab at Bern University of Applied Sciences. His research focuses on audio and image generation as well as deep learning-based weather reconstruction.

Dr Mascha Kurpicz-Briki is Professor of Data Engineering at the Institute for Data Applications and Security IDAS at Bern University of Applied Sciences, and Deputy Head of the Applied Machine Intelligence research group. Her research focuses, among other things, on the topic of fairness and the digitalisation of social and community challenges.