Some parts of these sections have been reproduced with permission from Wells et al.19 This is an Open Access article, distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The text below includes minor additions and formatting changes to the original text.
Work stream 2 overview
A full-scale, two-arm, single-blind RCT with a nested qualitative study was conducted to assess the clinical effectiveness and cost-effectiveness associated with the Group-MCT intervention plus usual CR (MCT + CR). CR services from five NHS trusts across north-west England (University Hospital of South Manchester, Central Manchester University Hospitals NHS Foundation Trust, East Cheshire NHS Trust, Stockport NHS Foundation Trust and Pennine Acute Hospitals NHS Trust) recruited 332 patients attending CR with symptoms of anxiety and/or depression. Participants were randomly assigned to receive MCT + CR or usual CR alone. Patients assigned to receive MCT + CR attended six additional weekly sessions of Group-MCT led by two CR staff members, with each session lasting 60–90 minutes. The primary outcome was level of anxiety/depression as measured by total HADS score at 4-month follow-up. Secondary outcomes included HADS score at 12 months plus scores on the Impact of Events Scale Revised (IES-R), Metacognitive Beliefs Questionnaire-30 (MCQ-30), EuroQol-5 Dimensions, five-level version (EQ-5D-5L) and Cognitive Attentional Syndrome Scale-1 Revised (CAS-1R) at 4- and 12-month follow-up. At 4 months, patients in the MCT + CR arm had significantly reduced total HADS scores compared with those in the CR-alone arm. At 12 months, the group difference was reduced but still statistically significant (p < 0.01). The results for most secondary outcomes also favoured the MCT + CR arm at both 4 and 12 months. The protocol of this trial and the principal results have been published.17,19
A qualitative interview study found Group-MCT to be effective, positive and beneficial. Patients identified advantages of the group format linked to non-specific supportive factors, and they valued the techniques used in treatment and supported delivery of the intervention by non-mental health specialists. The full qualitative results have been published.22
Work stream 2 aims
- Evaluate the effectiveness of MCT + CR compared with usual CR alone in alleviating depression and/or anxiety in patients attending CR.
- Evaluate the impact of Group-MCT on secondary outcomes including post-traumatic stress, metacognitive beliefs, health status, adherence to CR, health and social care utilisation and work resumption.
- Assess the durability of treatment outcomes at 4- and 12-month follow-up.
- Obtain patient qualitative data to help interpret evidence of effectiveness, including processes that might underpin or compromise effectiveness or explain heterogeneity in effectiveness.
- Obtain practitioner qualitative data to evaluate practitioner experience of Group-MCT delivery and understanding of patients’ emotional needs and identify potential enablers of/barriers to the recruitment and retention of patients.
- Obtain data from a stated preferences survey about participants’ relative preferences, utility and willingness to pay (WTP) for components of Group-MCT to inform future policy and commissioning decisions.
- Establish the cost-effectiveness of Group-MCT.
Methods
We conducted a multicentre, two-arm, single-blind RCT with 4- and 12-month follow-up comparing Group-MCT plus usual CR (MCT + CR) with usual CR alone.
Participants were recruited from CR centres across five NHS trusts in north-west England (University Hospital of South Manchester, Central Manchester University Hospitals NHS Foundation Trust, East Cheshire NHS Trust, Stockport NHS Foundation Trust and Pennine Acute Hospitals NHS Trust). No changes were made to the study eligibility criteria following WS1; see WS1 for a full description. For details of patient recruitment and study eligibility, see the published trial protocol.17 Patients were randomly allocated to the trial arms by Manchester Academic Health Science Centre Clinical Trials Co-ordination Unit using a computer in a 1 : 1 ratio using a minimisation algorithm to balance the trial arms with respect to hospital site, sex and HADS scores. Patients were informed of their trial arm allocation by a member of the research team. The trial chief investigator, trial statistician and research assistants collecting assessment data were masked to treatment allocation.
Group-MCT intervention (MCT + CR)
No changes were made to the Group-MCT intervention following WS1 (pilot feasibility study); see WS1 for a description of the intervention.
Usual CR
Usual CR was delivered as described above.
Data collection and outcomes
Data collection and outcomes mirrored the pilot study; see the trial protocol for details.17
Analysis
Analysis was conducted in accordance with a prespecified analysis plan specifying the analytical models, primary and secondary outcomes, choice of covariates, sensitivity analyses and other key aspects of the analysis. Prior to data analysis or unmasking, the analysis plan was finalised and approved by the TSC.
The primary outcome was:
- HADS total score at 4-month follow-up (after treatment).
The secondary outcomes included:
- HADS total score at 12-month follow-up
- Post-traumatic stress symptoms measured on the IES-R at 4- and 12-month follow-up
- Metacognitive beliefs measured on the MCQ-30 total and uncontrollability and danger subscale at 4- and 12-month follow-up
- Health status measured on the EQ-5D-5L at 4- and 12-month follow-up
- Repetitive negative thinking and coping mechanisms measured on the CAS-1R at 4- and 12-month follow-up
- Adverse events related or unrelated to the study
The primary analyses used intention-to-treat principles. A linear mixed-effects regression model was applied for continuous outcomes, incorporating all three time points (baseline, 4 months and 12 months). The prespecified covariates used were randomisation factors (hospital site, sex, baseline total HADS score), age and medication for depression or anxiety (never taken/currently taking/taken in the past). All other potential covariates were below predefined imbalance criteria for sensitivity testing [standardised mean difference (SMD) > 0.25 or category difference of > 10% between arms]. We applied hierarchical regression models with random effects at the levels of the patient and the CR (or MCT + CR) course attended. The covariance matrix for the model was chosen as either unstructured or first-order autoregressive depending on whichever gave the lower Bayesian information criteria score.
The effects associated with the intervention at 4- and 12-month follow-up were examined using the treatment-group-by-time-point interaction terms from the mixed-effects model analysis, where time point was a categorical variable to provide independent tests of effect at 4 and 12 months. No adjustments for multiple testing were applied, and an alpha value of 5% was used throughout. Sensitivity analysis was conducted using multiple imputation (MI) to assess the robustness of the results against missing values. There were very few missing values at baseline (one missing outcome value and a maximum of three missing values on any covariate); therefore, these were imputed by simple regression imputation using all available variables at baseline but excluding trial arm. MI was then used to impute missing outcome values at 4 and 12 months using the full set of variables and including the interaction term between trial arm and time point (for consistency with the analysis model). The chained-equations MI procedure was used and 20 MI data sets.
A mixed-effects logistic regression was conducted to assess the differences between the arms in engagement in economic activity (as a binary outcome) at 4- and 12-month follow-up. Covariates in the model were the same as for the continuous outcome measures.
All outcome measures demonstrated skewness and kurtosis below the threshold of 1.0 specified in the analysis plan and so sensitivity against non-normality was not assessed. The trial eligibility criteria allowed the inclusion of participants without clinically relevant anxiety provided they had at least mild depression, and vice versa; 23% and 40% of participants respectively fell into these categories at baseline, closely balanced between the arms. To determine how this might have impacted on analysis results for HADS anxiety and depression as separate outcomes we conducted sensitivity analyses excluding these individuals. Analyses were conducted using Stata version 14 (StataCorp LP, College Station, TX, USA).
Results
Participants: overview
Between July 2015 and January 2018, 3808 patients were referred to CR across all five sites. A total of 992 patients had a score of ≥ 8 on the HADS subscales; of these, 332 were consented to the trial following eligibility screening and initial contact. One hundred and sixty-three patients were randomly allocated to MCT + CR and 169 patients were randomly allocated to usual CR alone (see Figure 3). For further details, see Wells et al.19
FIGURE 3 Work stream 2 trial profile. Figure reproduced with permission from Wells et al.19 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original figure. (Figure not reproduced here; see the original article.)
Effects associated with Group-MCT
Mean HADS total scores and 95% confidence intervals (CIs) for each trial arm at pre- and post-treatment assessment points are presented in Figure 4. The mean HADS total score under CR alone declined gradually over time in an almost linear fashion, compared with a large reduction over the first 4 months under MCT + CR followed by a plateauing of scores. On the primary outcome – HADS total at 4 months – the results significantly favoured MCT + CR [adjusted mean difference (AMD) −3.24, 95% CI −4.67 to −1.81, p < 0.001; SMD 0.52]. The between-group difference in HADS remained significant at 12 months, albeit at a lower level (AMD −2.19, 95% CI −3.72 to −0.66, p = 0.005; SMD 0.33). Mean HADS anxiety was significantly lower in patients in the MCT + CR arm at 4 months (AMD −1.67, 95% CI −2.54 to −0.81, p < 0.001; SMD 0.44) and 12 months (AMD −1.35, 95% CI −2.22 to −0.48, p = 0.002; SMD 0.34). MCT + CR patients achieved a lower HADS depression mean score at 4 months (AMD −1.58, 95% CI −2.37 to −0.79, p < 0.001; SMD 0.47) but not at 12 months (AMD −0.85, 95% CI −1.75 to 0.05, p = 0.065; SMD 0.23).
FIGURE 4 Unadjusted mean total HADS scores at baseline and at 4- and 12-month follow-up. Note: bars are 95% CIs. Figure reproduced with permission from Wells et al.19 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original figure. (Figure not reproduced here; see the original article.)
Most other outcomes also favoured the MCT intervention: adjusted mean IES-R scores were lower with MCT + CR at 4 months (AMD −4.92, 95% CI −9.04 to −0.81; p = 0.019) but not at 12 months (AMD −3.28, 95% CI −7.92 to 1.36; p = 0.166); MCQ-30 total scores were lower at both 4 and 12 months (AMD −8.57, 95% CI −11.95 to −5.18, p < 0.001; AMD −7.37, 95% CI −11.24 to −3.50, p < 0.001, respectively); MCQ-30 negative beliefs subscale scores were lower at both time points (AMD −3.15, 95% CI −4.16 to −2.14, p < 0.001; AMD −2.35, 95% CI −3.43 to −1.26, p < 0.001) and the CAS-1R was also lower with MCT + CR at both 4 and 12 months (AMD −126.25, 95% CI −165.83 to −86.67, p < 0.001; AMD −116.29, 95% CI −159.13 to −73.45, p < 0.001). EQ-5D-5L utility scores showed no statistically significant group difference at 4 months (AMD 0.03, 95% CI −0.02 to 0.09; p = 0.200) or 12 months (AMD 0.03, 95% CI −0.02 to 0.10; p = 0.201); the difference on the EQ-5D-VAS was not significant at either time point (4 months: AMD 4.62, 95% CI −0.10 to 9.34, p = 0.055; 12 months: AMD 0.66, 95% CI −4.12 to 5.45, p = 0.786). Sensitivity analysis using MI changed the statistical significance of one secondary outcome, the IES-R at 4 months, which ceased to be statistically significant (p > 0.05). There was no significant difference between the arms in engagement in economic activity at 4-month (odds ratio 0.94, 95% CI 0.26 to 3.47; p = 0.93) or 12-month follow-up (odds ratio 1.10, 95% CI 0.24 to 4.99; p = 0.90).
To aid in the interpretation of the clinical impact of findings, the Reliable Change Index (RCI)44 was computed for the primary outcome. The RCI represents the difference between two measurements made in a single individual that would be statistically significant at a p-value of < 0.05. It was computed for the HADS total score at the primary 4-month follow-up. Using the control sample, we calculated a Cronbach’s alpha of 0.91 at 4 months for the HADS to estimate reliability for the usual CR population. Based on this, a reduction of 6 points in an individual’s score was defined as statistically reliable improvement, while an increase in 6 points was defined as a reliable worsening of symptoms. Calculations of the HADS total score at 4-month follow-up showed that 21% of patients in the CR-alone arm reliably improved compared with 33% of patients in the MCT + CR arm. The proportion of patients exhibiting psychological deterioration was 15% in the CR-alone arm compared with 4% in the MCT + CR arm.
Qualitative evaluation: overview
From the intervention arm of the trial, 32 patients took part in qualitative interviews prior to starting Group-MCT but during CR [time point 1 (T1)]. Patients who attended four or more Group-MCT sessions were defined a priori as having completed a minimal dose of the intervention likely to produce benefit. Among intervention patients who consented to take part in qualitative interviews, 22 completed the intervention, with 20 completing time point 2 (T2) interviews. Ten did not complete the intervention but five completed T2 interviews; four of these patients attended two or more Group-MCT sessions and their interviews were included in the analysis, and one patient interviewed did not attend any Group-MCT sessions due to work commitments and therefore was excluded from the analysis.
Interviews were conducted at T1 and T2 and were conversational in nature. Topic guides with a mixture of open and closed questions with open-ended prompts were used to encourage patients to share experiences and probe specific points. T1 interviews are discussed in WS1. Data gathered in T1 interviews were used to inform T2 interviews, which explored patients’ emotional experiences since T1, their views and experiences of Group-MCT, and their engagement with techniques from the intervention. Interviews were tailored to the individual, drawing on specifics from T1 interviews. Interview guides were modified iteratively as the interviews and analysis proceeded so that developing ideas could be tested. The interviews lasted an average of 52 minutes (range 14–88 minutes). The interviews were audio-recorded, transcribed verbatim and pseudonymised. Full detailed results have been published.22
Ten CR staff delivering MCT were interviewed about their experiences of training in and delivery of Group-MCT. Group-MCT was initially delivered by seven CR practitioners across three CR services participating in PATHWAY. Six of these practitioners were interviewed before training, with one practitioner declining to be interviewed at this point. However, all seven were interviewed during training and after they had delivered Group-MCT, as the practitioner who had originally declined later contacted the research team to take part. Two more CR services later joined the study. Two CR practitioners from one of these services and two clinical research nurses (CRNs) were trained in Group-MCT. The two CR practitioners provided written informed consent and were interviewed during training and after they had delivered Group-MCT. One CRN was interviewed during training only, as she left the study shortly afterwards, and the other CRN declined to take part in the study.
Qualitative studies received ethics approval from NRES Committee Northwest REC (reference 15/NW/0163). All participants provided written informed consent prior to being interviewed.
Qualitative data analysis
Inductive thematic analysis was used. Data in each transcript were coded to explore patients’ experiences and understanding of Group-MCT. Generated codes were discussed within the research team and discrepancies were resolved during discussion. Coded data were reviewed and collated into candidate themes. Candidate themes were discussed by the research team and on agreement semantic themes were identified.
Patients experience of Group-MCT
Two main themes were identified in patient experience of Group-MCT: general therapy factors and MCT-specific factors. The first theme concerned general therapy factors central to positive experiences of treatment, with subthemes of interaction with other CR patients and CR staff’s delivery of the intervention. Interaction with other CR patients was an important factor to most patients, providing reassurance, normalisation of feelings in a positive environment and facilitation of the intervention. Patients who were in small groups found that this negatively impacted their experience of Group-MCT, as small groups affected the delivery of the therapy. This highlighted the importance of having a minimum of three or four patients in a group to optimise patient experience. The delivery of the therapy by CR staff was generally received positively by patients as it enabled a positive and relaxing environment. For some patients, CR staff’s specific knowledge and experience of cardiology was important to their delivery of the therapy. However, a minority of patients criticised the staff’s delivery by because of a perceived lack of knowledge and the style of delivery. Patients’ perceptions of CR staff’s delivery were overall positive and demonstrated that CR staff, who were not mental health specialists, established and maintained a therapeutic alliance valued by patients.
The second theme related to MCT-specific factors, with subthemes of patients’ perceptions and understanding of the aims, experiences of individual techniques and perceptions of effectiveness of Group-MCT. Accounts of the aims of Group-MCT varied from those consistent with the model to those that were ‘off-model’. Patients who did not complete Group-MCT had negative perceptions of the aims of the therapy. Patients who correctly understood the aims of the therapy appeared to demonstrate a greater flexibility in their reaction to worry. All patients who completed the intervention were positive about the techniques introduced in MCT. Most patients were able to use these techniques in a manner consistent with the therapy model. However, in some cases techniques were used in a manner not consistent with the model. The results suggest that the techniques used in Group-MCT are largely understood and beneficial; however, therapists should be mindful of patients’ potential misinterpretation and inappropriate use of the techniques based on these narratives.
Practitioners experience of training and delivery of Group-MCT
Aim
To assess CR staff’s experience of learning and delivering MCT.
Methodology
Cardiac rehabilitation staff were interviewed at three time points: before Group-MCT training, in the middle of training, and after delivering Group-MCT. A topic guide was developed and used to guide the interviews.
Nine CR staff delivering MCT were interviewed about their experiences of training in and delivery of Group-MCT. See Appendix 1, Table 4, for an overview of therapist demographic characteristics.
For the results, see Appendix 1.
Cost-effectiveness of group-metacognitive therapy
Aims
A within-trial economic evaluation aimed to compare the cost-effectiveness of MCT plus usual care (CR) with that of CR alone from a health and social care perspective in the UK. The protocol of the cost-effectiveness evaluation has been published21 and details of the findings of the evaluation are reported in Appendix 2.
Methods
The economic evaluation used intention-to-treat and estimates total costs and quality-adjusted life-years (QALYs) for the trial follow-up (cost–utility analysis). An NHS and social care perspective was taken. The time horizon of the primary analysis was 12 months to incorporate sufficient time for any impact of MCT on service use and health status. Unit costs are reported in Appendix 3.
Analysis
Quality-adjusted life-years were estimated using the EQ-5D-5L, which was collected at baseline and at 4- and 12-month follow-up. Data on health and social care use were collected using an economic patient questionnaire adapted from other trials. This captured secondary, primary, community and social care use. Unit costs of NHS and social care services were taken from national average unit cost data, and the price year was 2019.45,46 Single imputation was used to impute missing baseline variables, with MI used to impute values missing at follow-up. Costs were imputed by category and utility by individual EQ-5D-5L domain to use all available data. The primary outcome of interest is the incremental cost-effectiveness ratio (ICER). Regression analysis was used to estimate net costs and net QALYs and these estimates were bootstrapped to generate 10,000 net pairs of costs and QALYs to inform the probability of cost-effectiveness. Sensitivity analyses tested the impact of the study design and assumptions on the ICER and cost-effectiveness acceptability analysis. See Appendix 2, Table 5, for the sensitivity analysis rationale.
Results
In the primary cost-effectiveness analysis, the MCT intervention is dominant, meaning it is both cost saving (net cost −£219, 95% CI −£1446 to £1007) and health increasing (net QALY 0.015, 95% CI −0.015 to 0.045). However, the CIs are wide and overlap zero, indicating high level of variability in the data and uncertainty in the estimates. The primary analysis found that at a willingness-to-pay threshold (WTPT) of £30,000 per QALY the MCT intervention is around 76% likely to be cost-effective, again reflecting uncertainty. See Appendix 2, Tables 5–7, for further details; see Appendix 2, Figure 9, for the cost-effectiveness plane; and see Appendix 2, Figure 10, for the cost-effectiveness acceptability curve. Sensitivity analysis demonstrated that the results at 4-month follow-up were similar to those of the primary analysis, and the complete-case analysis or the use of different assumptions around the cost of MCT did not affect conclusions. In these sensitivity analyses MCT remained dominant but with CIs wide and overlapping zero, demonstrating significant uncertainty.
Conclusion
Although the primary cost-effectiveness analysis and the majority of sensitivity analysis indicate that the MCT intervention may be cost saving and health increasing, or below typically accepted thresholds, the wide CIs that overlap zero indicate a high level of variability and uncertainty in the estimates. Further research should aim to reduce the uncertainty in the findings, for example with larger sample sizes and alternative measures used to produce utilities. In addition, research should explore how cost-effectiveness differs according to the implementation of MCT within CR.
Stated preferences survey
Aims
A discrete choice experiment (DCE) aimed to explore preferences for different characteristics of a clinic-based psychological intervention added to CR. The PPI work to develop the stated preference study has been published as a case study,20 and the analysis of the survey has been published.33 An example of survey materials is included in Appendix 4.
Methods
A DCE was conducted and recruited a general population sample and a trial sample. DCE attributes included the modality (group or individual), the healthcare professional providing care, information provided prior to therapy, the location, and the cost to the NHS. Participants were asked to choose between two hypothetical designs of therapy, with a separate opt-out included. A mixed logit model was used to analyse preferences. The cost to the NHS was used to estimate the WTP for aspects of the intervention design/delivery. The study recruited a range of participants, including members of the UK general public aged ≥ 18 years, recruited via a commercial survey sample provider, as well as Group-MCT trial participants.
Analysis
The DCE was analysed using individual choice responses as the dependent variable in the model.47 Owing to the presence of potential scale and preference heterogeneity (confirmed by a Swait and Louviere plot) a mixed logit model was used for analysis; this model assumes that parameters vary between individuals and accounts for heterogeneity across samples. Random utility theory assumes that a participant chooses between two options by interpreting the information described as a set of characteristics and selecting the one that provides the highest overall utility or value to them. Therefore, characteristic coefficients indicate the direction of preference. Marginal rates of substitution for each attribute were estimated by dividing the coefficient for that characteristic by the inverse of the NHS cost coefficient.
Results
Three hundred and four participants completed the DCE, the majority of whom were a sample of the general public (n = 262). The general population appeared to favour individual therapy (WTP £213, 95% CI £160 to £266) delivered by a CR professional (WTP £48, 95% CI £4 to £93) and at a lower cost to the NHS (β = −0.002; p = 0.000). Participants preferred to avoid options where no information was received prior to starting therapy (WTP −£106, 95% CI −£153 to −£59). The results for the location attribute were variable and challenging to interpret.
Conclusion
The study demonstrates a preference for psychological therapy as part of a programme of CR, as participants were more likely to opt in to therapy than they were to opt out. The results indicate that some aspects of the delivery that may be important to participants can be used to design a tailored psychological therapy that reflects preferences. However, preference heterogeneity is an issue that may prevent a ‘one-size-fits-all’ approach to psychological therapy delivery in CR. The COVID-19 pandemic (during which recruitment took place) is likely to have affected the DCE.