Synopsis

Background

Some text in this section is reproduced from Tallon et al.1 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) licence, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The text below includes minor additions and formatting changes to the original text. The paragraph starting ‘Innovative use of technological developments’ was originally written by INTERACT coauthors as part of our INTEgrated theRApist and online CbT for depression (INTERACT) NHS REC application form for the therapist survey and Delphi study; this wording was subsequently published on the Health Research Authority (HRA) website.2

Depression is one of the leading causes of disability worldwide.3 Ways to increase the availability of effective treatments are needed to reduce the unsustainability of current spending on health care.

National Institute for Health and Care Excellence (NICE) depression guidelines advocate psychological therapies for those with severe or chronic depression or who have not responded to initial treatments.4 The importance of patient preference is highlighted, with a meta-analysis demonstrating a threefold preference for psychological over pharmacological interventions.5 Many patients are willing to take antidepressants, but medication is not always effective; only half respond after the prescribed course.6

Cognitive–behavioural therapy (CBT) is an effective treatment for depression. Substantial investment in the NHS Talking Therapies (for anxiety and depression) programme (www.england.nhs.uk/mental-health/adults/nhs-talking-therapies/) (formerly, the Improving Access to Psychological Therapies services)7 has increased provision of brief, low-intensity interventions in England. Such low-intensity interventions are a key part of the stepped care model of treatment4 and are particularly appropriate for patients with less severe/complex depression. Although NHS Talking Therapies has increased the provision of high-intensity CBT (16–20 sessions), demand greatly exceeds supply. The lack of availability of CBT has implications for patient choice and clinical outcomes.

Therefore, a key question for commissioners and healthcare providers is how to increase access to high-intensity CBT without delay and without greatly increasing costs. The Chief Medical Officer’s annual report on mental health published in 2013 argued there was a need for pragmatic clinical trials to evaluate new technologies, including online interventions.8

The growth in CBT-based computerised interventions (CCBT) (e.g. MoodGym, Beating the Blues, Silvercloud) is an example of attempts to increase the efficiency of delivery in health care through the use of technology. Some CCBT packages have been endorsed by NICE.9 However, current UK evidence does not support the idea that CCBT materials alone can be an alternative to high-intensity CBT.10 Adherence to CCBT without support is poor and, in the absence of therapist support, effects are modest and short-term.11 As Helgadóttir (p. 246)12 argues, the lack of flexibility in these programmes is ‘theoretically problematic, and inconsistent with the emphasis on individual formulations that is the crux of traditional CBT’.

Nonetheless, CBT is particularly suited to the integration of computer and mobile technology with a therapist led approach. Exercises that take place outside the psychotherapeutic session are an important part of CBT and adherence to these can increase effectiveness.13 Working online with specially adapted interactive materials can be supported by timely reminders using mobile devices and wider use of digital media such as ‘apps’ that can be used to target emotions, thoughts and behaviours, and may improve adherence and engagement.14 Such Behavioural Intervention Technologies15 are low cost and can be integrated into daily life in a responsive and personalised way.

We had previously shown that delivering CBT online using instant messaging (IM) is clinically16 and cost-effective.17 However, therapy via IM alone is limited as it does not support the practice of CBT (e.g. thought records and other homework). Technological advancements make it possible to reduce therapist time and incorporate online materials as an integral part of therapy. One ‘proof of concept’ study (n = 15) suggests that an integrated or ‘blended’ approach is acceptable.18 Developing materials that are integrated with modern technologies, yet permit the therapist to tailor treatment to the individual, is critical and aligns with the CBT competencies framework19 and the idea of ‘meta-competences’ whereby the therapist adapts CBT to each patient’s needs.

Access to the internet and use of mobile devices (smart phones/tablets) has greatly increased over the last 15 years. At the outset of this programme of work in 2016, 75% of UK households were online, 92% of UK adults owned a mobile phone, 51% a smart phone and 24% a tablet.20 Since then, smartphone use has increased to more than 80% of the UK population21 and these developments are mirrored in other countries, including low- and middle-income countries. Interventions that incorporate online resources accessed via mobile devices may reach more effectively into people’s lives, allowing more immediate recording of thoughts and experiences. Mobile devices may also enable discreet completion of worksheets (such as thought diaries) thereby addressing concerns about the risk of ‘getting caught’ completing homework tasks in public.22 Use of such devices may therefore increase engagement with CBT ‘homework’ which is an important mediator of outcome.13,23 Engendering a sense of control and ownership are necessary to maximise engagement, and require collaboration with experts in human–computer interaction24 to develop platforms and materials that meet modern users’ expectations.

Experience-based design methods emphasise the importance of engaging users in the design and development of a new intervention.25 Previous reviews have focused on the uptake and acceptability of low-intensity CCBT.26,27 A recent meta-synthesis found that ‘absence of the self’ was prominent in reports of user experience of CCBT.28 An integrated approach where the therapist adapts the CBT to the individual’s needs will address such concerns.

Innovative use of technological developments and integration of online materials into therapy could offer increased engagement, reduce costs and widen access to populations that are difficult to reach (e.g. those who are disabled or have difficulty attending appointments for other reasons), while maintaining the effectiveness of CBT delivered by high-intensity therapists.2 Moreover, such platforms could be extended, as recommended by others,29 through the development of additional online materials as ‘bolt-ons’ to enhance utility within NHS services by addressing the needs of service users with a range of mental health problems or long-term conditions, further increasing utility and efficiency in terms of service provision.

At the start of our programme in 2016, there were no registered trials (http://apps.who.int/trialsearch/) integrating online materials with therapist led CBT for depression. An EU project (E-COMPARED: 2013–8: www.e-compared.eu/) of ‘blended care’ [face-to-face (F2F) CBT and self-help] for depression had some similarities, but in our model therapist, contact took place in real-time via IM rather than in person, and online materials were an integral part of therapy. Since 2016, no other trials have examined an integrated approach to delivering CBT.

Our intervention integrating therapist and online materials is a novel method of delivering CBT. It blends high-intensity therapy with innovative use of technology and aims to maintain the effectiveness of in-person CBT. The intervention was developed in collaboration with NHS service providers, ensuring it is fit for clinical purpose, and, if clinically and cost-effective, this model of integrated CBT could easily be introduced into existing psychological services, increasing access to, and equity of, CBT provision.

The INTEgrated theRApist and online CbT for depression programme

Aims and objectives

The aim of this National Institute for Health and Care Research (NIHR) programme was to develop and evaluate a novel approach to delivering CBT for adults with depression in primary care that integrates online CBT materials with input from an accredited CBT therapist (or ‘integrated CBT’). The specific objectives were:

Overview of programme

Figure 1 provides an overview of the INTERACT programme and the four work packages (WPs), and Table 1 summarises the programme’s aims, workstreams and outputs to date. Table 2 summarises changes made to the programme since the original grant application.

FIGURE 1 The INTERACT research pathway diagram. (Figure not reproduced here; see the original article.)

Our intervention development phase (WP1) was conducted between April 2016 and December 2019. The systematic review and network meta-analysis (NMA), Delphi study of CBT experts and therapist survey aimed to evaluate the most clinically and cost-effective components of CBT for depression, and informed early designs of the integrated CBT intervention and platform. We then used iterative research methods with stakeholders to design, pilot and refine our novel intervention. Security testing of the final platform took place in September 2020.

Once the intervention had been finalised, we commenced the evaluation of its clinical and cost-effectiveness, and acceptability [WPs2–4: randomised controlled trial (RCT); parallel economic evaluation and nested qualitative study]. Ethical approvals were in place by January 2020. However, the start of trial recruitment was delayed by 6 months (from June 2020 to December 2020) by the COVID-19 pandemic. The last data collection from patient general practice records took place in June 2024. As each WP used different methodology, we have reported the methods and results of each WP separately.

Work package 1: developing the intervention

The aim of this WP was to identify optimal combinations of CBT components and develop an innovative approach to delivering CBT. The focus was to provide a solution to demand on mental health services, by reducing therapist time, increasing availability and introducing online materials as an integral part of therapy, without compromising the efficacy of treatment.

The main objectives of this WP were to:

1. Identify the clinical (WP1.1) and cost-effective (WP1.2) components of CBT to inform the development of the intervention.
2. Develop an online platform to support the delivery of Integrated CBT (WP1.3); the online materials and a training package for therapists (WP1.4).

Work package 1.1: identifying the clinically effective components of cognitive–behavioural therapy

Aim

To identify clinically effective components of CBT to inform the development of the integrated CBT intervention.

This WP comprised two pieces of work: (1) building a data set for a NMA to identify effective and acceptable components of CBT; and (2) a modified Delphi study to identify the most effective components of CBT through a consensus approach.

Part 1: systematic review and network meta-analysis

Aim

We aimed to compare the effectiveness of different types of therapy, different components and combinations of components and aspects of delivery used in CBT interventions.

Methods

We undertook a systematic review of RCTs in adults with a primary diagnosis of depression where the effectiveness of a CBT intervention was compared with treatment as usual (TAU), no treatment, a wait list, psychological/attention placebo and/or another CBT intervention. We included RCTs comparing F2F CBT with computerised CBT, online CBT (with or without an interactive element), and other partial/full remote access provision (such as telephone support and text reminders).

Outcomes were pooled using standard (intervention-level) and component-level NMA. NMA is a form of meta-regression that enables the simultaneous comparison of multiple interventions in a single analysis. It allows all pairwise relative effects to be estimated from a connected network, even when no direct evidence is available, while respecting the randomised structure of the evidence. Interventions were classified based on the type of therapy and mode of delivery for our primary analysis. In addition, more advanced component-level models were fitted to examine the effectiveness of each content component or combination of components.

Key findings

In total, 91 studies were included in our review. There was strong evidence that CBT interventions resulted in a larger short-term decrease in depressive symptoms compared with TAU. The standardised difference in mean change for F2F CBT compared with TAU was −1.11 (95% credible interval −1.62 to −0.60); for hybrid CBT was −1.06 (−2.05 to −0.08); and for multimedia CBT was −0.59 (−1.20 to 0.02). In contrast, there was evidence that wait list increased depressive symptoms compared with TAU [0.72 (0.09 to 1.35)]. While multimedia and hybrid CBT may be as effective as F2F CBT, there was substantial uncertainty in the estimates of treatment effectiveness. We found no evidence of specific effects of any content components or combinations of components.

Limitations

Our efforts were hampered by inconsistent reporting of interventions. Some studies provided sufficient detail about the CBT intervention(s) evaluated to enable coding of components, while many others referred to a therapy treatment manual and only included a brief description. It is possible that some interventions included additional components that were not reported and that some of the reported components may not have been received by all study participants.

Related publications

The systematic review and NMA was reported in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses-NMA guidelines.49

López-López JA, Davies SR, Caldwell DM, Churchill R, Peters TJ, Tallon D, et al. The process and delivery of CBT for depression in adults: a systematic review and network meta-analysis. Psychol Med 2019;49:1937–47. https://doi.org/10.1017/S003329171900120X

The Cochrane protocol for the systematic review and network meta-analysis on the process and delivery of CBT is accessible at the following URL: www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD013140/epdf/full

Davies SR, Caldwell DM, López‐López JA, Dawson S, Wiles N, Kessler D, et al. The process and delivery of cognitive behavioural therapy (CBT) for depression in adults: a network meta‐analysis. Cochrane Database Sys Rev 2018;10:CD013140. https://doi.org/10.1002/14651858.CD013140

The Cochrane protocol for the systematic review on multimedia‐delivered CBT versus face‐to‐face CBT is accessible at the following URL: www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD013184/full

Davies SR, Caldwell DM, Dawson S, Sampson SJ, Welton NJ, Wiles N, et al. Multimedia‐delivered cognitive behavioural therapy versus face‐to‐face cognitive behavioural therapy for depression in adults. Cochrane Database Sys Rev 2018;11:CD013184. https://doi.org/10.1002/14651858.CD013184

Part 2: modified Delphi survey of cognitive–behavioural therapy experts

Aim

We aimed to establish an expert consensus on the effective components of CBT for adults with depression.

Methods

An international panel of CBT experts (n = 120) was invited to take part via an online survey. In round 1, experts were asked to rate the components of CBT that were most effective in bringing about clinically helpful change. Experts were provided with a list of 35 items covering both content (that facilitate behaviour change) and process components (procedures for the delivery of therapy) of CBT on a 9-point Likert scale from 1 (not at all effective) to 9 (very effective). Suggestions for additional components could be given. We followed the quality criteria proposed by Diamond et al.50 in conducting the study. This included specifying a priori our definition of consensus and the choice to stop data collection after two rounds. A priori rules were used to identify components carried forward to a second round, in which experts were asked to rerate components in light of feedback on the (first round) scores of other experts, with the final consensus items identified from the second round.

Key findings

Of the 120 CBT experts invited to participate, 32 participated in round 1 and 21 also provided data in round 2. Consensus was achieved in relation to nine content components (that facilitate behaviour change, e.g. self-monitoring and thought restructuring) and three process components (procedures for delivery of therapy, e.g. mode of delivery, number of sessions, support between sessions). The content components were: ensuring understanding; developing and maintaining a good therapeutic alliance; explaining the rationale for CBT; eliciting feedback; identifying and challenging avoidant behaviour; activity monitoring; undertaking an initial assessment; methods to prevent relapse; and homework assignments. Process components were: ensuring therapist competence; scheduling CBT sessions flexibly; and scheduling sessions for 45–60 minutes’ duration. Generic therapeutic competences (rather than items specific to CBT) comprised five of the nine content components. There was less agreement about the effectiveness of the cognitive components of CBT. The latter may reflect a greater emphasis on the behavioural aspects of CBT, aligning with the low-intensity treatments offered by NHS talking therapy services, but this warrants further research.

Limitations

There is no standard definition of ‘consensus’ and hence our definition was arbitrary, but a lower threshold may not have reflected a consensus. The results may have been different had a different group of experts taken part. However, the Delphi technique aims to recruit a small subset of those who can provide in-depth expertise rather than a representative sample.

Related publications

The accepted author manuscript for the modified Delphi survey is available at the following URL: https://research-information.bris.ac.uk/ws/portalfiles/portal/203250524/2019_07_04_Delphi_paper_revised2_clean_with_figure1.pdf

Taylor A, Tallon D, Kessler D, Peters TJ, Shafran R, Williams C, Wiles N. An expert consensus on the most effective components of cognitive behavioural therapy for adults with depression: a modified Delphi study. Cogn Behav Ther 2019;49:242–55. https://doi.org/10.1080/16506073.2019.1641146

Work package 1.2: identifying the cost-effective components of cognitive–behavioural therapy

Aim

To evaluate the cost-effectiveness of various components and combinations of CBT in depression in order to inform the development of the intervention.

Methods

This evaluation utilised the systematic review and NMA data set compiled in WP1.1. We developed a decision model to evaluate the cost-effectiveness of different CBT delivery formats that varied in terms of modes/intensities F2F CBT, multimedia CBT and hybrid CBT given in addition to TAU, in comparison with TAU alone for treating depression.

Key findings

All CBT delivery modes given in addition to TAU were found to be more cost-effective than TAU alone. Probabilistic sensitivity analyses found that F2F CBT with intensities of six 30-minute sessions and sixteen 60-minute sessions had the highest probability of being cost-effective. However, neither option reached a 50% probability of being most cost-effective (32.5% and 31.1%, respectively).

Limitations

There was substantial uncertainty around the estimates derived.

Related publications

The paper reporting the decision-analytic model evaluating the cost-effectiveness of different CBT delivery modes and variations in intensity is available at: https://doi.org/10.1016/j.jval.2020.07.008

Wu Q, Li J, Parrott S, López-López JA, Davies SR, Caldwell DM, et al. The cost effectiveness of different formats for delivery of cognitive behavioural therapy for depression: a systematic review based economic model. Value Health 2020;23:1662–70. https://doi.org/10.1016/j.jval.2020.07.008

Links between work packages 1.1 and 1.2 and other work packages

To optimise the design of new approaches to delivering CBT and improve the efficiency of treatment, it is important to understand the ‘active ingredients’. The NMA and Delphi survey were complementary pieces of work that highlighted the challenges in understanding such factors.

The findings of the NMA were encouraging as they suggested that hybrid interventions that combine online treatment and therapist contact, such as INTERACT, might be as effective as F2F (in-person) CBT, although these results should be interpreted with caution given the sparse data. However, there was little robust evidence as to the relative effectiveness of specific components or combinations of components. The NMA also found that wait list had a detrimental effect on outcome, and this informed our design of the multicentre RCT in WP2. The consensus on the importance of the provision of resources found in the Delphi study, especially those that supported the generic therapeutic competences and behavioural modification, informed the design of the platform and materials to be included.

Without robust new evidence regarding potential cost-effectiveness, we followed our original plans to ensure that the platform encouraged and supported between session work, and reduced the total number of therapy sessions (compared with F2F CBT) with the aim of increasing the intervention’s cost-effectiveness (WP3).

Work package 1.3: development of the online therapy platform

Aim

To develop an IT platform to support the use of online materials for integrated CBT that would be evaluated in the RCT (WP2).

This work consisted of three main stages: the development of the platform (phase 1); the evaluation with primary care patients with depression (phase 2); and further refinements (phase 3).

Phase 1: iterative design

Aim

1. To explore patients’ and therapists’ views and expectations on using an integrated approach to delivering CBT for depression and to establish design requirements.
2. To evaluate the usability and acceptability of prototypes and further inform the platform design.

Methods

Four user-centred design studies were conducted that aimed at understanding the requirements for the technical platform. Data on general requirements for an online integrated therapy platform were gathered from design workshops with people who had completed CBT in the past (n = 12) and individual interviews with CBT therapists (n = 12). More detailed design requirements were explored through prototype testing with people who had received CBT in the past (n = 7) and role-play sessions with CBT therapists (n = 5). All research activities were recorded with participant consent and transcribed. Affinity mapping was used to identify key themes from workshops, prototype testing sessions and role plays. A thematic approach was used to analyse transcripts of individual interviews. In addition, as part of usability testing and role-play sessions, we collected participants’ feedback on their experience using the standardised System Usability Scale51,52 which asks about subjective experience of using the system, including its perceived effectiveness and efficiency, and users’ satisfaction. Additional work was also conducted to identify factors that support user engagement, including a review of mental well-being apps35 and an online survey of app users to evaluate user feedback on mental health apps.36

Key findings from phase 1

Three key requirements were identified for integrated CBT therapy platforms: (1) features to overcome depression-related barriers; (2) features that support engagement; and (3) features that reinforce learning and support the acquisition and learning of new skills. Therapists highlighted the importance of collaborative working, the impact of technology on therapists’ workflow and workload, and potential of technology in supporting clients’ engagement with therapy. A list of 12 design considerations for the development of an integrated therapy platform was outlined.

These considerations focused on: strengthening the therapeutic relationship; personalising treatment; supporting learning; enhancing engagement and accountability; and adapting to the workload, schedules and expectations of therapists. F2F sessions help build rapport and trust, while online sessions are well-suited for supporting skills development. Therapists and patients should collaborate on exercises, such as worksheets, with therapy records and resources available for ongoing use. Flexibility and personalisation are important, allowing therapists to tailor exercises and worksheets to individual patient needs, and offering the option to modify materials as necessary. A balance between therapist-led and patient-led approaches is crucial, with clear expectations for patients alongside opportunities for independent exploration of relevant resources. Engagement is fostered by making patient commitments explicit, enabling progress tracking, and providing personalised feedback from therapists, supported by automated feedback where appropriate. Finally, therapists’ workloads and boundaries must be considered, with guidance on between-session contact and a clear approach to risk management that avoids overburdening therapists outside of sessions.

Limitations

A relatively small number of participants were involved in the study. Others may hold different views. However, the number of participants was comparable with other user-centred design studies and this methodology has been shown to provide design guidelines that are generalisable.

Related publications

The paper reporting the user-centred design studies is available at: https://mental.jmir.org/2020/9/e15972

Stawarz K, Preist C, Tallon D, Wiles N, Kessler D, Turner K, et al. Design considerations for the integrated delivery of cognitive behavioral therapy for depression: user-centered design study. JMIR Mental Health 2020;7. https://doi.org/10.2196/15972

Additional work was conducted to identify factors that support user engagement, including a review of mental well-being apps and an online survey of app users to evaluate user feedback on mental health apps.

Stawarz K, Preist C, Tallon D, Wiles N, Coyle D. User experience of CBT apps for depression: an analysis of app functionality and user reviews. J Med Internet Res 2018;20:e10120. https://doi.org/10.2196/10120

Stawarz K, Preist C, Coyle D. Use of smartphone apps, social media and online resources to support mental health and wellbeing: an online survey. JMIR Ment Health 2019;6:e12546. https://doi.org/10.2196/12546

Phase 2: usability and user experience evaluation

Aim

The aim of this evaluation was to explore primary care patients’ and CBT therapists’ views and experiences of the platform developed during WP1.3 phase 1 (iterative design) to ensure the online platform and CBT materials are acceptable to patients and therapists; and to identify areas for improvement prior to the main trial.

Methods

This evaluation comprised three strands of work that ran in parallel:

1. a longitudinal evaluation study with patients with depression recruited from primary care;
2. participatory design work with therapists providing CBT and their clinical supervisor;
3. usability testing sessions with people who had received CBT in the past.

We conducted an 11-month pilot study, in which 18 patients with depression were recruited from primary care and offered a course of treatment (up to nine 1-hour sessions of CBT) delivered by therapists trained in using the newly developed therapy platform. Therapy was delivered by three CBT therapists who also had a co-design role, providing substantial feedback on design and usability issues, and the therapist training package.

The first session of therapy was delivered in person at the patient’s general practitioner (GP) surgery, with subsequent sessions held online using IM in the therapy platform. Up to three shorter ‘check-in’ sessions to facilitate engagement and/or to review between session tasks (maximum 20 minutes) could be scheduled between regular therapy sessions as part of the 9 hours’ contact time.

Patients were interviewed twice: first over the telephone after three to six sessions of therapy to gather their initial views on the platform and online materials, and again (in-person) after completing therapy (or withdrawal or discharge) to explore their views and experiences of receiving integrated CBT. With participant consent, we also collected usage statistics for their engagement with the platform, including the number of times they logged in, worksheets completed and shared with the therapist, homework tasks set and completed, and their access to session notes and transcripts.

Initial interviews with therapists took place shortly after starting to use the platform, and further interviews were conducted after the end of therapy, along with a focus group for therapists and their supervisor to inform final revisions to the platform prior to the main trial.

Data collection and analysis (of individual patient interviews) proceeded in parallel to enable analytical insights from earlier interviews to shape later data collection. Data were analysed thematically. Affinity mapping was used to analyse data from therapist debriefing sessions, design sessions and the final focus group. A task-oriented analysis of usability testing sessions was conducted to identify the most common and serious issues that need to be addressed. Usage statistics were aggregated to identify the most commonly used (and ‘underused’) features and materials. These data, together with feedback provided by users (patients and therapists) via the platform feedback mechanism, allowed identification of areas that required further improvements and/or clarifications and addressed on a case-by-case basis following review and discussion within the research team.

In addition, we recruited eight patients who had had CBT in the past from local psychological services. These patients had a single usability testing session (approximately 1 hour) with a researcher during which time they were asked to use platform’s features (e.g. share a worksheet, look at session preparation questions) and to talk aloud while completing tasks.

This phase of development also included completion of a thorough information security risk (privacy impact) assessment, which was approved by the University of Bristol Information Security Manager, and the preparation of detailed documents to evidence the system readiness to ‘go live’. The platform also underwent extensive, independent security testing. The privacy impact assessment was reviewed prior to the main RCT and updated in September 2021.

Key findings from phase 2

A total of 18 patients with depression were recruited from primary care. Of these, 10 completed therapy; 1 withdrew before the start of therapy; 4 withdrew during therapy and 3 were discharged for non-attendance. Initial interviews were conducted with 13 patients (after 3–6 therapy sessions), and 9 ‘end of therapy’ interviews were completed.

Interview findings suggested that patients appreciated the first session being F2F so they could meet their therapist and build rapport, making subsequent online contact by IM easier. Typing limited the amount discussed during online sessions, but some patients noted it aided focus and promoted reflection. Some patients said that the integrated approach made therapy more accessible, but not all participants engaged with the between-session tasks (often due to time constraints). Some found the worksheets too complex. Usage data showed that patients reviewed transcripts, and patients commented that these were a useful aid to learning. Some technical issues were experienced using the platform, and sometimes these led to lower engagement.

Therapists echoed patients’ comments regarding the importance of an initial F2F meeting in establishing the therapeutic relationship and the lack of nonverbal cues when communicating online. Therapists commented on the challenges of being able to cover less material when typing compared with meeting clients in person. They raised concerns related to managing risk that may require flexibility in terms of mode of contact for vulnerable patients and flexibility in terms of the number of sessions offered.

Therapists were unsure as to how best to use the short ‘check-in’ sessions and one did not use them at all. However, therapists commented on the flexibility and accessibility of an integrated approach, and valued access to the library of resources that could be shared with clients to support therapy.

Usability testing sessions provided additional feedback that informed platform development and contributed to improving usability of the platform.

Limitations

Most of the patients interviewed had completed therapy, and therefore, there is a potential bias in terms of the views represented. However, patients who completed therapy expressed a range of views regarding the acceptability of the integrated approach. Most participants were female, and while there was some diversity in terms of socioeconomic status, all participants were White. A more diverse group of participants should be included in future research.

Related publications

The paper reporting the findings from the work piloting the delivery of integrated CBT using the INTERACT therapy platform in 17 patients with depression from primary care is available at: https://dl.acm.org/doi/pdf/10.1145/3313831.3376510

Stawarz K, Preist C, Tallon D, Thomas L, Turner K, Wiles N, et al. Integrating the Digital and the Traditional to Deliver Therapy for Depression: Lessons from a Pragmatic Study. CHI Conference paper; 2020. https://dl.acm.org/doi/pdf/10.1145/3313831.3376510

Phase 3: refinement and development of the final platform

Aim

To refine and develop the final online therapy platform for use in the main RCT (WP2).

Methods

The process of refining the platform began in tandem with phase 2. This was an iterative process led by the researcher with expertise in human-computer interaction. It involved the incorporation of feedback from patients, therapists and wider stakeholders, input from our patient and public involvement (PPI) representative at monthly team meetings, as well as regular meetings between the programme leads, human–computer interaction researchers and software developers. Feedback was assimilated into a list of ‘tickets’ that each outlined a proposed modification or new functionality to the online therapy platform. These ‘tickets’ were prioritised through discussion between the programme leads, human–computer interaction researchers and software developers based on the importance and/or frequency with which an issue had been raised by patients and/or therapists, and the time and resources available within the project. An external assessment of the security of the platform was completed prior to the start of recruitment to the main trial to ensure system readiness.

Key findings

Data from the phase 2 patient and therapist interviews informed the final changes and adjustments to the platform and enabled the development of a system that was user-friendly. Key changes included: (1) adding a typing indicator to the IM feature of our platform in order to improve the communication process during online therapy sessions; (2) redesigning the structure of the library of resources to make it easier for therapists to find and share resources with patients; (3) simplifying the way resources were displayed to patients; (4) removing functionality for brief ‘check-in’ sessions; and (5) integrating a telephony system within the therapy platform. In addition, in response to therapist feedback, the therapy protocol was revised in terms of the number of sessions offered (to permit (up to) an additional three sessions if clinically indicated) to provide flexibility for managing risk and/or patient complexity.

Limitations

Budgetary constraints limited the incorporation of all suggestions from stakeholders with regards to platform design and functionality. However, feedback was prioritised to ensure that critical changes were implemented. There were several features that we would have included if additional time and resources had been available. These included functionality for uploading images, having a whiteboard to enable drawing during therapy session, and the ability for the therapist and patient to be able to edit worksheets simultaneously. These would have made the platform and therapy more interactive.

Links to other work packages

The online therapy platform that was developed in this WP1.3 was then evaluated in the RCT (WP2), parallel economic evaluation (WP3), and nested qualitative study (WP4). The original plan was for the first therapy session to take place in-person F2F. However, due to the COVID-19 pandemic, this was switched to a videocall.

Work package 1.4: intervention content development

Aim

This WP aimed to: (1) identify the resources used by CBT therapists to support Beckian CBT for depression and (2) to develop and pilot an online training package for CBT therapists that covers use of the online materials and technical aspects of the INTERACT therapy platform.

Part 1: selection of materials for inclusion in the platform

Therapist survey

Aim

To identify materials commonly used by therapists delivering individual CBT with patients who have depression, and to identify those materials perceived as most effective in bringing about clinically helpful change.

Methods

Questionnaires were sent to 3665 British Association for Behavioural and Cognitive Psychotherapies (BABCP) accredited UK CBT therapists asking about their use of resources described in CBT manuals, and their views on the effectiveness of such resources.

Key findings

Of 3665 approached by post/e-mail, 994 individuals (27%) responded, with a further 33 completing the questionnaire via the study website. Most respondents (80%; n = 818/1027) were BABCP accredited CBT therapists who deliver one-to-one therapy. Over 85% of respondents used symptom measures, lists of problems/goals, activity schedules, behavioural activation diaries/plans, and case formulation worksheets ‘frequently’ or ‘very frequently’. Resources that were regarded as the most effective in bringing about change were: activity schedules, behavioural activation diaries/plans, case formulation worksheets, thought records and resources to support identification of conditional beliefs.

Limitations

The low response rate may have introduced bias. However, there was no evidence of differences in resource use, or ratings of effectiveness, between those who responded to the initial invitation or subsequent reminders.

Final selection of cognitive–behavioural therapy materials

The selection of materials to be incorporated into the platform was based on consideration of four sources of information:

1. the competences framework for the delivery of CBT for depression which describes the activities that need to be brought together in order to carry out CBT effectively;19
2. the findings from the therapist survey (outlined above);
3. the consensus regarding the effective components of CBT from the Delphi study (WP1.1);
4. the findings from the systematic review/NMA (WP1.1), which identified that social skills training was a component of treatment for depression in previous research and hence we included a specific worksheet on this in the platform.

Materials were divided into three categories: psychoeducational materials, worksheets for completion by patients, and short animations. Worksheets were adapted for the platform to be interactive and usable online. Psychoeducational materials and worksheets came mainly from four sources used by CBT therapists: (1) Psychology Tools (www.psychologytools.com/); (2) GetSelfHelp (www.getselfhelp.co.uk/); (3) Centre for Clinical Interventions (www.cci.health.wa.gov.au/); and (4) the Five Areas^® Approach/Living Life to the Full Approach (www.llttf.com/). Worksheets were identified for each of the core competences for CBT for depression. A wide variety of materials was incorporated to allow therapists more choice and flexibility to tailor treatment for the individual. A series of animations were created by the research team in collaboration with a graphic designer. These were designed as brief (< 2 minutes) introductions to key topics in CBT.

Links to other work packages

Through the therapist survey, we learnt that therapists use a wide range of CBT materials in their practice. This was reflected in the materials that we added to the platform. We also learnt about the sources from which they draw their materials and gained appropriate permissions to use such materials, so that therapists would be working with materials with which they were familiar. We discovered that therapists used these materials both within the session with the patient and recommended them for use outside the session, and this was reflected in the platform design. In addition, we developed 16 brief animations designed to introduce patients to the platform, provide psychoeducation, and support patients in their work outside therapy sessions. The animations were voiced and subtitled, and their content drew on available worksheets and other materials, and on the research outlined above.

Related publications

The accepted author manuscript for the paper reporting the findings of the therapist survey is available at: https://research-information.bris.ac.uk/ws/portalfiles/portal/173757515/Submitted_manuscript_proof_23_10_2018.pdf

Tallon D, McClay CA, Kessler D, Lewis G, Peters TJ, Shafran R, et al. Materials used to support cognitive behavioural therapy for depression: a survey of therapists’ clinical practice and views. Cogn Behav Ther 2019;48:463–81. https://doi.org/10.1080/16506073.2018.1541927

Part 2: development and implementation of a training package for therapists

Aim

To develop an online training package for therapists.

Methods

Training for the therapists employed for WP1.3 – phase 2 was developed and delivered with CBT experts in the research team and supported by the therapist’s supervisor. The emphasis of the training was on the use of the platform and delivery of Beckian CBT in the online context. Therapists were provided with a therapist manual and platform user guide, and undertook self-directed learning and familiarisation with the platform and embedded materials (worksheets, psychoeducational sheets and videos). Therapists attended 1.5 days of formal in-person training and undertook a series of individually tailored training sessions, including role play, with members of the research team. The training was streamlined and delivered with a therapist who joined the team part-way through the phase 2 work. This informed the development of an online training package for the main trial.

Key findings

Final revisions to the training manual were made for the RCT with emphasis on: (1) making clear the differences between delivering CBT F2F and delivering therapy using the INTERACT therapy platform; (2) when/how to incorporate the telephone into online sessions where IM is the main mode of communication); and (3) time management. Refinements were made to the platform user guides (for patients and therapists) and relevant standard operating procedures were prepared for the RCT.

Links to other work packages

In the main trial, therapists were employed 1 month prior to the start of providing therapy in order to familiarise themselves with the trial and therapy platform. For the first two therapists, training was delivered online in real-time (‘live’) via videoconferencing. These sessions formed the basis for the development of an online training package (accessed via a password-protected page on the INTERACT website) for all other therapists. Online training comprised a mixture of asynchronous materials (e.g. recorded lectures covering background to the trial, therapy protocol, using the INTERACT therapy platform, managing risk and research procedures). There was the opportunity to ask questions about each week’s material during a live online question and answer session with the INTERACT study leads and other members of the research team. Therapists also gained hands-on practice in using the platform in role play sessions (with research team members playing the role of the ‘client’).

Work package 2: randomised controlled trial

Full details of the trial are provided in Appendix 1. The full protocol1 and WP results41 have been published.

Aim

The aim of this WP was to evaluate the clinical effectiveness of an integrated approach to delivering CBT in reducing depressive symptoms and improving quality of life (QoL) over 12 months [compared with usual care (UC)] for primary care patients with depression, in an individually randomised pragmatic trial.

Methods

We used broad and pragmatic inclusion criteria to recruit individuals with depression from primary care. Eligible patients were those who were aged 18 or over, scored ≥ 14 on the Beck Depression Inventory, version 2 (BDI-II)53 and met International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) criteria for a depressive episode.54,55 We recruited participants from 67 general practices in areas surrounding the 3 trial sites: Bristol, London and York. Participants were identified either via a search of practice electronic records or by direct referral from their GP. Those who agreed to contact via either recruitment method were telephoned by a researcher and asked a few screening questions. Those who met the screening criteria were invited to an appointment with a researcher (baseline assessment) to explain the trial, undertake the baseline assessment, establish eligibility and obtain written informed consent. Most participants were e-mailed a link to an online consent form, and completed this using their own smartphone, tablet or computer during their baseline appointment. If they were unable to complete this electronically (e.g. due to technical issues), they were posted a consent form to complete on paper and post back to the researcher. Participants were sent a copy of their completed consent form. If eligible, participants were randomised to one of two groups: usual GP care or integrated CBT (in addition to usual GP care). Randomisation was by means of a computer-generated code from an automated, remote randomisation service, stratified by study centre and minimised on gender, current antidepressant use and depression (BDI-II) severity.

Key features of the Integrated CBT intervention are illustrated in Figure 2. Integrated CBT comprised nine therapist-led sessions, with (up to) a further three sessions where clinically appropriate. The first session took place by videocall (90 minutes) with subsequent 50-minute therapy sessions taking place online via IM (typing) within the INTERACT therapy platform. Therapists and patients could use a collaborative workspace to view, edit and discuss CBT resources (worksheets, information sheets, brief videos) within the platform. Therapists could agree tasks (e.g. worksheets) with the patient for them to complete between therapy sessions. Patients could access transcripts of their online IM therapy sessions.

FIGURE 2 Key features of integrated CBT. (Figure not reproduced here; see the original article.)

The library of resources within the platform covered: avoidance; behavioural activation; assumptions, rules and beliefs; formulation; general psychoeducation; goal setting; problem solving; relapse prevention; risk; sleep; working with thoughts; and a small number of resources related to anxiety. Additional information on the origin of the resources is given earlier in Final selection of cognitive–behavioural therapy materials.

The intervention was delivered by nine part-time CBT therapists, accredited as CBT practitioners by the BABCP. Therapists received training in use of the platform and provided standard Beckian CBT.56

All participants were followed up at 3, 6, 9 and 12 months post randomisation.

Primary comparative analyses between the two randomised groups were conducted according to the groups ‘as randomised’, without imputing missing values. Linear regression was used to compare the primary (continuous) outcome (BDI-II score) at 6 months between randomised groups, adjusting for the study centre, baseline BDI-II score and the other two minimisation variables. Similar regression models were used for secondary outcomes. The effect of the intervention on outcomes over 12 months was summarised using repeated-measures models, with an interaction between treatment and time to formally test whether effects varied with time.

Additional sensitivity analyses were used to examine the effect of: (1) missing data; (2) any baseline imbalances in prognostic factors; (3) number of treatment sessions attended; (4) potential clustering by therapist57 and (5) timing of questionnaire completion. Complier-average-causal effect (CACE) analyses were conducted to estimate the treatment effect among compliers. A priori specified subgroup analyses explored the potential for differential treatment effects based on chronicity and severity of depression, and personality difficulties.58

Key findings

From 12,182 letters of invitation and 381 direct referrals, 1455 patients agreed to being contacted by the research team (see Appendix 1, Figure 4). In total, 761 individuals were identified as potentially eligible at telephone screen and agreed to attend a baseline assessment. Of these, 451 patients were eligible and randomised to receive either integrated CBT (n = 225) or to continue with usual GP care (n = 226). Three hundred and forty-six (77%) were followed up at 3 months (Figure 3), 334 (74%) at 6 months, 328 (73%) at 9 months and 327 (73%) at 12 months.

FIGURE 3 INTEgrated theRApist and online CbT for depression Consolidated Standards of Reporting Trials flow diagram. (Figure not reproduced here; see the original article.)

Participants were predominantly female (n = 313, 69%) and, on average, aged 39 years (see Appendix 1, Table 3). The mean BDI-II score at baseline was 32.8, indicative of severe depression. Most had a history of depression (with nearly half having had 5 or more prior episodes of depression), were taking antidepressants (70%) and the duration of the current episode of depression was ≥ 2 years for 51% of participants (see Appendix 1, Table 3).

Fourteen individuals (6.2%) had no therapy sessions. A total of 137 participants (60.9%) completed therapy (defined as receiving at least 9 sessions or reaching an agreed end with their therapist in fewer than 9 sessions) and 88 (39.1%) either withdrew from therapy or were discharged for repeated non-attendance. The mean duration of the intervention from randomisation was 3.2 months [standard deviation (SD) 1.6] (for n = 211 who had ≥ 1 session). Participants received a median of 8 therapy sessions [interquartile range (IQR): 3–10], with a slightly higher figure for those who completed therapy [median: 9 (IQR: 9–11)]. A total of 152 participants (67.6%) received at least 6 sessions of integrated CBT or reached an agreed end in < 6 sessions (CACE definition).

Those assigned to the intervention logged into the INTERACT therapy platform a median of 28 times (IQR: 12–42) with a median of 10 resources (worksheets/information sheets) shared by the therapist to their client, and 5 practice at home tasks (homework) agreed. Use of the platform was higher among the CACE sample [median (IQR) number of logins: 37 (27–49.5); number of resources shared: 13 (9.5–17); and number of homework tasks set: 9 (4–13)].

Those in the intervention group had a BDI-II score that was, on average, 4.4 points lower (less depressed) than those in the UC group at 6 months {difference in means: −4.4 [95% confidence interval (CI) −7.0 to −1.9] p = 0.001}. This equated to an effect size of 0.43 (using baseline pooled SD). Adjustment for baseline imbalances slightly increased the between-group difference, but adjustment for actual time to follow-up had no effect (see Appendix 1, Table 4).

Individuals in the intervention group had nearly a threefold increased odds of meeting criteria for response [odds ratio (OR): 2.78 (95% CI 1.63 to 4.74) p < 0.001] and remission (BDI-II score < 10) [OR: 2.74 (1.46, 5.15) p = 0.001] at 6 months, equating to a number needed to treat (NNT) of 6 (95% CI 4 to 11) and 8 (95% CI 5 to 19). The difference in mean proportional change in depressive symptoms on the BDI-II was –14.6% (see Appendix 1, Table 7). In other words, individuals in the intervention group had experienced (on average) a 15% greater reduction in depressive symptoms (relative to baseline) compared with those in the UC group. There were beneficial effects of the intervention in terms of other secondary outcomes. The intervention group had fewer depressive symptoms on the Patient Health Questionnaire-9 (PHQ-9), fewer symptoms of anxiety [Generalised Anxiety Disorder-7 (GAD-7)] and less impairment in terms of functioning at 6 months (see Appendix 1, Tables 8–10).

There was little evidence of clustering of outcomes by therapist [intraclass correlation coefficient (ICC): < 0.0001] and, commensurate with this, the results from a fully heteroscedastic model that accounted for clustering by therapist [difference in means: −4.4 (95% CI −6.9 to −1.9)] were identical to the results of the primary analysis.

For the a priori subgroup analyses, there was no evidence that personality difficulties (p-value for interaction: 0.47), depression severity (BDI-II score: p = 0.22) or chronicity of symptoms (p = 0.77) had any effect on the difference between the intervention and UC group at 6 months.

In repeated-measures analyses using data from 6 and 12 months, individuals in the intervention group had a BDI-II score that was, on average, 3.8 points lower than those in the UC group [see Appendix 1, Table 11: difference in means: −3.8 (−6.1, −1.5) p = 0.001]. In repeated-measures analyses, the intervention group also had a twofold increased odds of response and remission, and benefits in terms of reductions in PHQ-9 and GAD-7 scores and function [Work and Social Adjustment Scale (WSAS)] (see Appendix 1, Tables 5–10). There was little evidence that the effects varied over time (p-values for interactions ranged from 0.12 to 0.96).

The results of CACE analyses [difference in means: −5.8 (95% CI −9.1 to −2.6) p < 0.001] showed that the effect of the intervention was larger among who completed treatment as intended (see Appendix 1, Table 12).

The results following the imputation of missing data were consistent with those of the primary analysis (see Appendix 1, Table 13). Other scenarios for missing data were also considered (see Appendix 1, Table 13) that represent extremes in the pattern of missingness.

In terms of receipt of other treatments (outside of the trial), by 6 months, 13/324 (4.0%) of participants had received computerised CBT [intervention group: n = 2/166 (1.2%); UC group: n = 11/158 (7.0%); median number of sessions was 7 (6, 8) and 4 (3, 9), respectively]. By 6 months, 74 individuals [intervention: n = 24/168 (14.3%); UC: n = 50/161 (31.1%)] had received another talking therapy outside the trial. Of these, 15 individuals had received high-intensity individual CBT [intervention group: n = 5/168 (3.0%), median number of sessions: 10 (IQR: 8–12); UC group: n = 10/160 (6.3%), median number of sessions: 10 (7, 11)]. At 6 months, 64.9% (n = 216) of participants [107 of 170 (62.9%) intervention group; 109 of 163 (66.9%) UC group] were taking antidepressant medication [difference: −3.9% (95% CI −14.2% to 6.3%)].

There were 54 adverse events (AEs) and 14 serious adverse events (SAEs) during the study (see Appendix 1, Tables 14–17). There were a higher number of AEs in the intervention group, possibly due to more frequent contact. No SAEs were definitely/probably related to the intervention.

Limitations

The follow-up rate at 6 months was slightly below the original retention target (74% vs. target 80%). Some imbalances in baseline characteristics between randomised groups were identified but there was no evidence that adjustment for these impacted on the conclusions drawn. Results imputing missing data were consistent with the findings of the primary analysis.

Links to other work packages

The INTERACT trial evaluated the clinical effectiveness of the integrated approach to delivering CBT that was developed in the first part of the programme (WP1). The cost-effectiveness and acceptability of the intervention were evaluated in WP3 and WP4, respectively.

Related publications

Work package 2 is reported in full in Tallon et al.41

Work package 3: cost-effectiveness evaluation

Full details of the cost-effectiveness evaluation are given in Appendix 2.

Aim

The aim of this WP was to establish the cost-effectiveness of integrated CBT in reducing depressive symptoms and improving QoL over and above UC over 12 months for primary care patients with depression.

Methods

Costs of integrated CBT included costs of training, post-training supervision, intervention sessions, and operating and maintaining the INTERACT platform. The costs of staff were estimated by multiplying the time they spent by their respective hourly costs (salary + oncosts + overheads + capital). The costs of operating and maintaining the INTERACT platform was estimated based on information provided by the University of Bristol Research information technology team where it was hosted during the study. These were extracted from study records.

Usual care included psychological services, including packages of CCBT, counselling or talking therapy provided by NHS, and prescriptions of antidepressant medication. Prescription of antidepressant medication and GP consultation were extracted from GP records for 12 months post randomisation. Use of UC and other healthcare services for mental health, and costs from wider perspectives were collected at 6 and 12 months post randomisation, as part of the self-reported questionnaire. All monetary outcomes were presented in Great British pounds 2022–3.

The effectiveness measure was quality-adjusted life-years (QALYs), derived from EuroQol-5 Dimensions, five-level version (EQ-5D-5L).59 It was administered at baseline, 6 and 12 months. The response to EQ-5D-5L was converted to utilities using a mapping approach, following the recommendation by NICE.60,61 The utilities were then used to derive QALYs following area under the curve approach.62

The primary analysis was an incremental cost–utility analysis (CUA) from an NHS and Personal Social Services (PSS) perspective.60 The costs comprised costs of integrated CBT and UC, primary, community, secondary health care, and social services for mental health.

Incremental costs and incremental QALYs were estimated using generalised linear regression, costs with the gamma family and log link while QALYs with the default family and link. The incremental cost-effectiveness ratio (ICER) was calculated by dividing the incremental costs by incremental QALYs, when both were positive. Uncertainty surrounding the point estimate was assessed using non-parametric bootstrap resampling technique.63 A cost-effectiveness plane (CEP) and cost-effectiveness acceptability curves (CEACs)64 were constructed based on the bootstrap iterations to illustrate the uncertainty of conclusion of cost-effectiveness of integrated CBT over and above UC. The conclusion was drawn against the maximum acceptable ICER thresholds at £20,000–30,000 per QALY gain recommended by NICE.60

Secondary analyses followed the methods used in the primary analysis but were undertaken from the participants’ and informal carers’ perspective and societal perspective, respectively. The costs from the participants’ and informal carers’ perspective comprised participants’ payments in relation to treatments and remedies for mental health, and lost income of participants and/or their informal carers due to participants’ mental health conditions. The costs from the societal perspective comprised both the costs from a NHS and PSS perspective and from the patient and informal carer perspective; and costs of treatments or therapies provided by entities other than NHS, free of charge or at a nominal price to participants, informal carer’s allowance, paid leave and extra costs to arrange for cover for participants. However, as there were no recommended maximum ICER thresholds for these two perspectives, the results are presented without reference to thresholds.

Primary and secondary analyses were undertaken using a multiple imputed data set, following Rubin’s rule.65 Multiple imputation with chained equation was used to handle missing data.

Sensitivity analyses were conducted using complete-case analyses (CCAs) from three perspectives, considering those who had complete data on all variables. To examine the missing at random (MAR) assumption, a set of sensitivity analyses using pattern mixture modelling60 was also undertaken. The missing not at random (MNAR) assumptions were those who had missing values at follow-ups and were either in need of more healthcare services or experienced worse health, or both conditions at the same time.

Key findings

The mean costs of integrated CBT were estimated at £987 [standard error (SE) £14] per participant (see Appendix 2, Table 18). The mean costs of UC were estimated at £382 (SE £57) in the UC group and £193 (SE £42) in the integrated CBT group (see Appendix 2, Table 19).

The costs from NHS/PSS perspective were £753 (SE £77) per participant in the UC group and £1754 (SE £127) in the integrated CBT group, with adjusted incremental costs at £1009 (95% CI £737 to £1286) (see Appendix 2, Table 19). The mean QALYs were 0.554 (SE 0.017) in the UC group and 0.597 (SE 0.017) in the integrated CBT group, with adjusted incremental QALYs at 0.033 (95% CI −0.002 to 0.059). The ICER was £30,576 per QALY gain (probability of cost-effectiveness from £20,000 to £30,000 : 13–39%) (see Appendix 2, Table 19 and Figure 5b).

Complete-case analysis from NHS/PSS perspective included 109 participants in the UC group and 103 participants in the integrated CBT group. It showed a slightly more favourable picture with an ICER at £22,421 (probability of cost-effectiveness from £20,000 to £30,000 : 37–66%) (see Appendix 2, Table 19 and Figure 5d).

The mean out-of-pocket payments (OOPs) and lost income were £808 (SE £190) in the UC group and £1127 (SE £318) in the integrated CBT group (see Appendix 2, Table 20). The mean societal costs were £2734 (SE £439) in the UC group and £3890 (SE £475) in the integrated CBT group. The incremental costs from the perspective of participants and informal carers and from a societal perspective were £179 (95% CI −£249 to £680) and £1259 (95% CI £164 to £2099), respectively (see Appendix 2, Table 20). With incremental QALYs the same, the ICER from the participants’ and informal carers’ perspective was £5424 per QALY gain (probability of cost-effectiveness reaching 50% at approximately £7000 per QALY gain) and from a societal perspective £38,152 per QALY gain (probability of cost-effectiveness reaching 50% at approximately £38,400 per QALY gain) (see Appendix 2, Table 20 and Figure 6a and 6c).

The MNAR sensitivity analyses found no significant impact on the conclusions. Excluding seven participants identified post hoc as outliers in costs, OOPs, lost income or non-NHS benefits (see Appendix 2, Table 21), while reduced ICERs from the three perspectives, did not impact on the conclusions already drawn.

Limitations

Missing data (in terms of resource use) was an issue. While CCAs showed lower ICERs than those estimated from imputed data, it should be noted that fewer than half of participants were included in the CCAs.

Links to other work packages

The evaluation of cost-effectiveness was conducted alongside the trial that evaluated the clinical effectiveness of the integrated approach to delivering CBT (WP2) and the nested qualitative study that evaluated the acceptability of the intervention (WP4).

Work package 4: qualitative evaluation

Aim

The aims of the nested qualitative study were to:

1. explore patients’, therapists’ and supervisors’ views and experiences of using an integrated approach to delivering CBT;
2. understand patients’ reasons for completing or not completing treatment;
3. assess patients’, therapists’ and supervisors’ views on how this novel approach affected the therapist-patient relationship.

Methods

Semistructured interviews were held with 30 trial participants (n = 20 intervention; n = 10 UC) after they had completed their primary trial outcome measures at 6 months post randomisation. These patients were purposively sampled across the trial sites to achieve maximum variation in relation to age, gender, depression severity, treatment response, adherence to treatment and therapeutic alliance. Twelve of those interviewed in the intervention arm had completed therapy (defined as having at least nine sessions of therapy or reaching an agreed end) and eight had not.

Interviews were conducted with all trial therapists (n = 9) and their clinical supervisors (n = 3) towards the end of their involvement in the trial.

Interviews were held remotely by telephone or videocall. Topic guides were developed for each group (patients, therapists and supervisors) but included similar questions to facilitate triangulation of accounts. All interviews were audio-recorded, transcribed, anonymised and analysed using an inductive thematic approach.67 A coding frame was then developed and applied to all transcripts. NVivo software (QSR International, Warrington, UK) was used to facilitate electronic coding and data retrieval. Data from the three groups were analysed separately before findings were triangulated to identify areas of commonality and differences across them. Data collection and analysis preceded in parallel. The final number of interviews conducted was guided by information power.68

Key findings

Integrated CBT – that combined therapist-led sessions with CBT resources within an online platform – was acceptable for primary care patients with depression. Patients who completed therapy valued being able to talk to a therapist and access resources within a single platform and had learnt skills to better manage their depression. The ability to review transcripts of IM therapy sessions assisted with completion of homework tasks and allowed patients to track their progress. Typing allowed reflection but less could be covered in one session, compared with an in-person therapy session. This meant therapists set expectations regarding the session agenda and used more focused questions to ensure benefit. Therapists believed that delivering therapy this way was a useful initial first step to enable people to learn CBT skills. Patients who did not complete therapy found the CBT approach too structured or, between session tasks, too onerous. Many also found therapy by IM challenging – they found it difficult to express themselves through typing, were confused by overlapping messages and frustrated by the time taken waiting for a reply. Individuals in the UC group were disappointed not to get the intervention but described a range of ways that they managed their depression, such as medication and lifestyle changes. Many were glad to have participated in the trial, to support research about managing depression.

Limitations

The views of those interviewed (patients, therapists and supervisors) may not be representative of those outside the trial context. Because the intervention was being assessed within a trial context, therapists had less flexibility in how it could be used and were limited in the CBT resources they could share.

Related publications

The paper reporting the results of the nested qualitative study is available at: https://doi.org/10.1111/hex.70002

Fox F, Wiles N, Kessler D, Tallon D, Thomas L, Williams C, et al. Patients’ and therapists’ views of integrated online CBT for depression. Health Expect 2004;27:e70002.

Links to other work packages

The nested qualitative study was part of the evaluation of the integrated approach to delivering CBT. It was conducted alongside the randomised trial and economic evaluation that evaluated the clinical and cost-effectiveness of the intervention (WP2 and WP3, respectively). The findings from all three WPs have been brought together in the conclusions from the programme.

Involvement of patients and/or the public

Patients, service users and members of the public were involved in the design and management of this research and advised on dissemination activities.

Intervention development

Prior to undertaking the RCT, we conducted substantial pilot work to gather stakeholder feedback on the trial intervention. For example, the first phase of intervention development focused on user-centred design and included design workshops and prototype testing sessions with 18 people who had received CBT for depression in the past, and qualitative interviews and CBT role-play sessions with 12 CBT therapists experienced in the treatment of depression.

Next, the phase 2 study gathered detailed feedback from 18 primary care patients who received the intervention as part of the pilot study, and from the therapists and supervisor who delivered this treatment. This work was pivotal in informing the final design of the intervention evaluated in the RCT.

Patient and public involvement advice on remote research protocols

Due to the pandemic, we needed to redesign the trial protocol so that we could deliver the trial remotely. It was important for us to understand whether remote research procedures, such as e-consent, video-call assessments, online surveys, and holding the first therapy ‘F2F’ via videocall (instead of in-person) would be acceptable to potential participants. In August/September 2020, we conducted seven in-depth PPI interviews with people who have taken part in previous trials, to ask for their input on these changes. These interviews were very helpful in reassuring us that these methods would be acceptable (and perhaps even beneficial) to potential participants. Furthermore, there were many helpful suggestions on how we could offer more flexibility to participants and improve their experience of the remote assessments. For example, we were advised to: consider offering further support for people who might struggle to use online tools; minimise participants’ anxiety by being clear about what taking part in the trial would involve; offer breaks during the remote appointment; and provide the flexibility for participants to do some tasks independently if this makes them feel more comfortable. We incorporated these suggestions into our protocols.

Patient and public involvement advisors

The INTERACT Programme Management Group included a PPI member (Paul Lanham), who was also a service user representative co-applicant, and member of the management group responsible for developing the intervention and the trial. The Independent Steering Committee included two PPI members. More recently, we have established a PPI group comprising people who have taken part in previous depression trials.

Patient and public involvement advisors provided ongoing input into design and management of the programme. They also provided valuable feedback on the patient information sheet, invitation letter, consent form and newsletters for trial participants.

Dissemination

We have published a summary of the results of the therapist survey in the BABCP members’ magazine, ‘CBT Today’ in order to reach professional stakeholder groups, and the therapist survey participants (most of whom are BABCP members).

We also circulated the results of the Delphi study to the survey participants. We have circulated the results of the pilot study to participants who indicated they wished to receive this. Similarly, we plan to send a summary of results to the RCT participants and to the GP practices involved in recruitment activities. We also aim to disseminate findings through relevant community organisations. We will seek advice from our service user co-applicant and PPI group with regards to our dissemination strategy. A PPI advisor (Paul Lanham) has coauthored several INTERACT manuscripts.

Future plans

We will ask our PPI group to advise us on the planned results newsletters for INTERACT trial participants and to provide input into the design of future depression studies.

Climate, health and sustainability

Who is the INTEgrated theRApist and online CbT for depression platform for?

In the past, psychotherapies such as CBT have been delivered in person. In 2003, the first ‘computer therapy programme for anxiety and depression’, Beating the Blues, was tested in the UK.69 Since then, there has been a rapid growth in online therapies, and it is likely that this will continue. However, lack of flexibility and resultant poor levels of adherence to computerised psychotherapy has been a barrier to clinical effectiveness. INTERACT offers the personalisation and responsiveness of in-person CBT and delivers comparable health benefits. The platform is clinically effective and acceptable for patients with depression. There are benefits in terms of accessibility for patients and convenience for both therapist and patient, which may improve the reach of therapies for those who find travel difficult or cannot manage office hours. There is also evidence that even without specific materials for the management of anxiety disorder, patients who undertook the therapy were less anxious at the primary outcome time point. The platform could therefore be expanded to support treatment of other disorders, such as anxiety.

Climate and sustainability and relevance to the health service

Both therapists and patients can work from home and the reduction in travel to sessions could result in lower carbon emissions, as could less printing of materials provided by therapists to patients. The environmental impact of delivering the service itself, from an energy and resource perspective based on scoping attributional footprint data from the University of Bristol Computer Science department, is likely to be significantly less than these gains. The platform was designed for use in psychological services, such as NHS talking therapies. It can be adapted to local needs and practices.

Plans for implementation

We have held discussions with the Research Manager at the Bristol, North Somerset and South Gloucestershire Integrated Care Board (BNSSG ICB) who hosted the programme. We agreed the next steps would be to discuss licensing issues for the software with our Sponsor, the University of Bristol, if the platform is to be rolled out to existing providers. We will also approach Health Innovation Networks (previously Academic Health Science Networks) who can link us with decision-makers in different parts of the country.

Equality, diversity and inclusion

Language and terminology

The development of the platform was an iterative process, including a longitudinal evaluation study involving patients with depression recruited from primary care, participatory design work with therapists providing CBT and their clinical supervisor and usability testing sessions with people who had received CBT in the past. Patient and GP-facing materials in the trial were reviewed by our PPI colleagues. The platform was designed from the first to be accessible and understandable.

Participant representation, recruitment and retention

Our inclusion criteria for participants from primary care were as broad as possible. All participants were aged 18 years or over and had depression. We excluded those who were under psychiatric care, had a current history of alcohol or substance abuse, had dementia, bipolar disorder or psychotic illness. We did not exclude participants with psychiatric comorbidity. In some of our previous primary care mental health trials (of both psychotherapy and medication), we have not been successful in recruiting from an ethnically diverse population; our average proportion of non-White participants was < 5% in three recent trials. We improved on this for the INTERACT trial where the proportion of non-White participants was 15%, in part by recruiting from a centre with a diverse population. We also had more ethnic diversity among the therapists providing the intervention, which may have had an impact on retention for ethnic minority participants. There were two important limitations to our capacity to be inclusive in this study. The first is that we were not able to include those unable to read or write in English. We did not have capacity within the time frame of the study to provide both translations for the online materials and employ therapists (and researchers) who could work in other languages. However, one of the strengths of an online CBT platform such as INTERACT is that it has the flexibility to provide access to non-English-speaking therapists for non-English speakers if the intervention is adapted for wider use. In addition, there is the flexibility to incorporate additional content and/or materials to meet different cultural needs. The second issue is more complex. It has been argued that there is a ‘digital divide’ between those who have reliable internet access and the necessary hardware, and those who do not. This divide worsens with increasing levels of deprivation and age. It is, therefore, likely that there is a selection bias in studies of online treatments, and this is borne out by data from our previous work. The participants in INTERACT have relatively high levels of educational attainment and employment, similar to an earlier study of online CBT (IPCRESS)16 and higher compared with a study of in-person CBT for treatment-resistant depression (CoBalT).70 It may be that increasing access to and familiarity with online interventions in the population will gradually lessen this divide over time.

Reflections on what was and what was not successful in the programme

Our efforts to identify the clinically effective components of CBT through our systematic review and NMA to inform the development of the intervention were hampered by the limited reporting of the components of interventions. Similarly, our ability to identify the cost-effectiveness of different modes of delivering CBT and variations in intensity was limited by a high level of uncertainty around estimates. Nonetheless, we used data derived from other aspects of the programme – the Delphi study on effective components of CBT, the therapist survey which identified commonly used resources to support CBT alongside the competency framework for CBT for depression – to inform the selection of materials for inclusion in the platform. Extensive iterative design work with stakeholders (patients and therapists) enabled us to develop an online therapy platform that enabled the delivery of integrated CBT and that we were able to evaluate in a large RCT set in UK primary care. Importantly, we found that this integrated approach to delivering CBT was acceptable to both patients and therapists.

Due to the COVID-19 pandemic, we had to adapt our trial processes and procedures to enable the INTERACT RCT to proceed. This was a substantial amount of work. All research appointments with participants were subsequently conducted remotely (either by videocall or telephone). The first therapy appointment changed from being held in person to taking place by videocall. Implementing these changes enabled us to successfully deliver the RCT. We recruited 451 patients and followed up 73% of participants for 12 months. The latter follow-up rate was lower than our original target (80%); however, results of analyses imputing missing data were consistent with the findings of the primary analyses. Encouragingly, engagement with the integrated CBT approach was similar to engagement with in-person therapy, and much higher than low-intensity CBT interventions.

The eligibility criteria for the INTERACT RCT were deliberately broad and included those experiencing a new episode of depression, as well as those with more chronic depression or who had not responded to initial treatments. These are all groups for whom NICE advocate psychological treatment. It is noteworthy that INTERACT participants were not dissimilar in terms of their disease severity and chronicity to participants in a primary care trial of treatment-resistant depression (CoBalT).70 In INTERACT, on average participants had moderate to severe depression. While just 19% of INTERACT participants had experienced their current depressive episode for < 6 months, 51% had a current episode lasting more than 2 years. Many (27%) had previously been referred to a psychiatrist (CoBalT: 40%), and 70% were taking antidepressants. This contextual information is key in terms of thinking about the population to whom the results of this trial can be generalised.

It was surprising that, while the intervention was clinically effective, the intervention was not cost-effective given prior evidence of cost-effectiveness of other CBT interventions for depression in similar populations.17,71 The incremental QALY gain for INTERACT was 0.033, similar to that seen in an earlier trial of IM therapy (0.027),17 but smaller than the incremental QALY gain (0.053) in a trial of in-person CBT for patients with treatment-resistant depression.71 Costs have risen over the 10–15 years since these earlier studies were conducted [mean cost of CBT per participant: IPCRESS, 2009 (mean 8 sessions): £493;17 CoBalT, 2013 (median 12 sessions): £91071 compared with INTERACT (median 8 sessions): £987], but the threshold used by NICE for assessing cost-effectiveness has remained the same.

The intervention costs for INTERACT included training costs. However, it is likely that therapists had more time to familiarise themselves with the platform than would be the case if the intervention were to be rolled out within NHS talking therapy services. In addition, in a service, these fixed costs would also be spread across a larger number of patients as the number of patients treated by an individual therapist would not be constrained by the trial recruitment target. Together, these factors would likely lead to a reduction in the per-patient costs of training.

Conclusions from the whole programme

We successfully developed an online therapy platform that integrates therapist-led sessions with online CBT materials and offers a novel approach to delivering therapy.

Integrated CBT was effective in reducing depressive symptoms in primary care patients with depression. Benefits were also seen for the more stringent outcomes of response and remission, and in terms of symptoms of anxiety and improved function. Improvements were maintained over 12 months. Benefits were likely to be clinically important.

Engagement with the integrated CBT approach was similar to engagement with in-person therapy and substantially higher than low-intensity CBT interventions. In qualitative interviews, we found that many of the reasons that patients did not complete therapy were similar to those for in-person CBT. However, aspects of the typed message exchange (e.g. overlapping messages or delays in replies) were confusing or frustrating for some patients. Although less could be covered in a typed session (compared with in-person therapy), the ability to reflect, review transcripts of sessions and have access to resources that enabled patients to learn skills to manage their depression were important. Crucially, the model of integrated CBT enables the therapist to tailor treatment to the individual in line with the idea of meta-competences set out in the CBT competences framework.

Based on the current threshold used by NICE for decision-making, the integrated CBT approach would not be considered cost-effective from the perspective of the NHS/PSS, but the estimate was only just over the upper threshold. Training costs are one component of the intervention costs. These costs would be unlikely to be maintained at the same level if the intervention was rolled out within the NHS. Moreover, once trained, therapists would be likely to treat more patients than was the case within the trial context, which would reduce the per-patient training cost and may make the cost–benefit more favourable in the longer term.

There are no official acceptable thresholds for ICERs from participants’ and their carers’ perspective and from a societal perspective. Such thresholds would need to be determined by the relevant stakeholders independently in order to draw conclusions regarding the cost-effectiveness of the intervention from a wider perspective. Stakeholders may have disparate objectives in terms of outcomes, and hence QALYs might not be the outcome of interest. The analysis of interventions which impact upon wider stakeholders pose a challenge in health economic evaluation. Future evaluations may strive to impose weights upon outcomes to different interests and raises the discussion about the value of output created in different sectors and to different socioeconomic groups.

Recommendations for future research

Given the often chronic nature of depression, future research needs to examine the longer-term clinical and cost-effectiveness from the perspective of the healthcare provider and wider society in general. CBT is an educational approach; hence, those in the intervention group may be better equipped to manage their depression in the longer term. Studies of in-person CBT have found that CBT is a clinically and cost-effective intervention over 4 years.72 Evidence is thus needed to quantify and substantiate the potential for long-term gain using this integrated approach, including the potential for economies of scale such as spreading training costs over time more efficiently. If clinically and cost-effective over the long term, this would have important implications for the management of depression.

In addition, future research needs to examine which aspects of the platform led to improvements in depression. This may enable refinement of the intervention to increase benefits.

Implications for practice (and any lessons learnt)

More and more people have access to the internet and their own personal devices that would enable them to engage in therapy online. The COVID-19 pandemic and social distancing requirements resulted in many services moving to remote delivery of therapy, either by telephone or videocall. This switch to remote therapy has persisted post COVID. There has also been a proliferation of mental health apps (including those based on CBT). However, there is little empirical evidence to support the use of many of these. The integrated CBT approach that we have developed was acceptable to patients and clinically effective. While there was some uncertainty around its cost-effectiveness, this novel mode of delivery has the potential to increase the availability of CBT and increase access for those who find it difficult to attend appointments in person.