The IMMERSE blog

Please follow us at LinkedIn or BlueSky to stay tuned

A closer look into the black box
— process evaluation

“What causes specific symptoms?” “How can we target and improve them?” As a PhD student and psychotherapist in training, I am continuously confronted with questions like these. This curiosity is what motivated me to study psychology in the first place, a science committed to understanding, explaining, and predicting human behaviour and experience. From the early days of behaviourism to modern psychotherapeutic models, the discipline has continually evolved in its pursuit to make the invisible visible.

B.F. Skinner, as one of behaviorism’s most influential thinkers, stated that “we undertake to predict and control the behavior of the individual organism” (B.F. Skinner, 1953). Behaviorists focused on observable behavior - stimulus in, response out. The inner workings of the mind were considered a “black box”, inaccessible, invisible, and therefore unmeasurable. Classical experiments, such as Pavlov’s dogs salivating at the sound of a bell, underscored this stimulus-response model and laid the foundation for the first wave of cognitive behavioral therapy (CBT), which emphasized conditioning and behavioral exposure techniques.

Later, a second wave of CBT marked a shift, an attempt to open that black box. Cognitive processes became the focus of interest. Aaron Beck, who is known as the father of cognitive therapy, stated that “if beliefs do not change, there is no improvement. If beliefs change, symptoms change. Beliefs function as little operational units” (Aaron Beck, 2000). This highlights the role of cognitive processes that are still crucial in today’s understanding of CBT. The focus expanded to thoughts, beliefs, and interpretations that drive emotional responses and behaviors. Or, as Albert Ellis, a key figure in rational emotive behavior therapy, put it: “You mainly feel the way you think.” (Albert Ellis, 2001).

The third wave of CBT brought emotions into the foreground, integrating concepts such as mindfulness, acceptance, and value-based approaches (Gilbert, 2009, 2013; Hayes et al., 2004). As research continues, psychotherapeutic models and interventions evolve as well, including new CBT approaches such as process-based therapy (Hofmann & Hayes, 2019). We are still striving to improve our understanding of the black box – to capture the inner workings of the mind clearer. To develop effective mental health interventions, we must understand psychological processes to identify key mechanisms of change as well as uncover what works in therapy and why - what a complex task!

In recent years, with increasing digital methods, digital interventions gain interest, especially given the gap in demand and supply of psychotherapy, but also given its possibility to offer interventions outside of sessions (Reininghaus & Myin-Germeys, 2023). Digital interventions represent a promising frontier for psychotherapy and research, offering new ways to examine and deliver care. In IMMERSE, a digital mobile mental health (DMMH) intervention was developed with the goal to investigate, in detail, reach, effectiveness, adoption, implementation, and maintenance as well as contextual factors, processes, and costs of implementing ESM-based monitoring, reporting, and feedback into routine mental health care in four European countries (i.e., Belgium, Germany, Scotland, Slovakia). IMMERSE comprises a parallel-group, assessor-blind, multi-centre cluster randomized controlled trial as well as a process and economic evaluation (Reininghaus et al., 2024).

As part of the process evaluation, qualitative, semi-structured interviews are held with service users, clinicians, and managers/system administrators. By capturing voices from these three important stakeholders, we gain a clearer picture of how digital interventions are used, experienced, and integrated into people’s lives. Combining a quantitative and qualitative lens helps us refine our approach, ensure relevance, and improve effectiveness. It offers insight into how people relate to digital tools, what makes them engage, or disengage, and how therapeutic mechanisms unfold in real life – helping us explore what works, for whom, in what respects, to what extent, in what contexts, and how (Westhorp et al., 2011).

This all comes together to form the focus of my PhD project – analysing mechanisms and processes of digital, ESM-based mental health interventions with the aim to detect relevant mechanisms of change and ways to target them, which is not so far from my initiating questions. IMMERSE is such an interesting study to me since it provides valuable and diverse insights into a digital mental health intervention and its implementation into routine mental health care, all while also including the voices of the participants sharing their individual experiences. Understanding psychotherapy means understanding its process - not just its effects. We aim to improve our understanding of the intervention by these in-depth insights into lived experience – to take a deeper look into the black box.

References

Gilbert, P. (2009). Introducing compassion-focused therapy. Adv Psychiatr Treat, 15, 199-208. https://doi.org/https://doi.org/10.1192/apt.bp.107.005264

Gilbert, P. (2013). Compassion Focused Therapy (Vol. 3). Junfermann.

Hayes, S. C., Strosahl, K. D., & Strosahl, K. (2004). A practical guide to acceptance and commitment therapy. Springer Science & Business Media.

Hofmann, S. G., & Hayes, S. C. (2019). The future of intervention science: Process-based therapy. Clinical Psychological Science, 7(1), 37-50.

Reininghaus, U., & Myin-Germeys, I. (2023). Mental Health Reform, Ecological Translation and the Future of Public Mental Healthcare. In M. Wensing & C. Ullrich (Eds.), Foundations of Health Services Research: Principles, Methods, and Topics (pp. 223-233). Springer International Publishing. https://doi.org/10.1007/978-3-031-29998-8_18

Reininghaus, U., Schwannauer, M., Barne, I., Beames, J. R., Bonnier, R. A., Brenner, M., Breznoščáková, D., Dančík, D., De Allegri, M., Di Folco, S., Durstewitz, D., Gugel, J., Hajdúk, M., Heretik, A., Izáková, Ľ., Katreniakova, Z., Kiekens, G., Koppe, G., Kurilla, A.,…Schick, A. (2024). Strategies, processes, outcomes, and costs of implementing experience sampling-based monitoring in routine mental health care in four European countries: study protocol for the IMMERSE effectiveness-implementation study. BMC Psychiatry, 24(1), 465. https://doi.org/10.1186/s12888-024-05839-4

Westhorp, G., Prins, E., Kusters, C., Hultink, M., Guijt, I., & Brouwers, J. (2011). Realist evaluation: an overview.