Statistical analyses plan
Analyses followed the formal SAP for CARMS which was written by the trial statistician (Richard Emsley) and agreed by the co-PIs prior to approval by the CARMS TSC and DMEC. The SAP was approved prior to data entry, validation and quality assurance checks being completed, and prior to data-lock being instigated. Participant flow at each stage of the trial was reported in accordance with the CONSORT statement for non-pharmacological treatments.120 All statistical analyses were conducted with Stata® version 16 (StataCorp LP, College Station, TX, USA) or later. The SAP is available from the corresponding author on request.
Missing data
Missing data on individual measures were pro-rated if more than 90% of the items were completed, unless a measure-specific threshold was otherwise specified in the manual (or equivalent publication) for that measure. If this pro-rating procedure was not possible, then the item was considered as missing. We allowed for the presence of missing data under the assumption that the data were ‘missing at random’, conditional on the covariates in the model.
Analysis of the randomised controlled trial data to assess efficacy
All analyses and summary statistics were conducted using intention-to-treat (ITT) principles. The ITT population is defined as all participants randomised regardless of completion of therapy or withdrawal from the study. Analyses were conducted after all 12-month follow-up outcome assessment measures had been completed. No interim analyses were carried out.
Baseline data
Analyses of baseline data were restricted to descriptive statistics for the treatment and control groups. Counts and percentages for binary and categorical variables, and means and standard deviations, or medians, with lower and upper quartiles, for continuous variables, along with minimum and maximum values and counts of missing values were generated. No tests of statistical significance or confidence intervals (CIs) were performed to examine differences between the two trial groups on baseline data.
Testing the efficacy of the suicide-focused intervention Cognitive–Behavioural Suicide Prevention for psychosis from the randomised controlled trial data
Linear, mixed regression models121 were fitted to the primary suicidal ideation severity, ASIQ, scores to assess efficacy effects of the suicide-focused intervention. ASIQ scores at 6 and 12 months were repeated measures. Fixed effects were randomisation group (treatment vs. control), NHS site, antidepressant use at baseline (yes/no), baseline assessment scores for the ASIQ, allocation group (treatment vs. control), time and time-by-allocation group interaction effects. Hence, stratifying factors were NHS site and antidepressant use at baseline. Participant was included as a random intercept to account for repeated measures. The model included a random intercept for therapist in the treatment group. Participants in the control group were considered as clusters of size 1.65
The treatment effect was the between-group adjusted mean difference. Marginal treatment effects122,123 were estimated for the primary suicide outcome measure (ASIQ score) and for the secondary suicide outcome measures. The 6-month time point was critical. However, analyses also included examining treatment effects at the 12-month follow-up time point. Effects were reported as adjusted mean differences in scores between the RCT groups (treatment + TAU vs. TAU) with 95% CIs and two-sided p-values. Cohen’s d effect sizes were calculated as the adjusted mean difference of the outcome divided by the sample standard deviation of the outcome at baseline. The same modelling procedure was applied to the mechanistic and clinical variables. Linear mixed models were used for continuous outcomes, and logistic mixed models for binary outcomes.65
Analysis of the randomised controlled trial data to assess psychological suicide mechanisms
Mediation analyses were performed to examine potential mechanisms underlying the effect of the treatment group (therapy plus TAU) compared to the control group (TAU alone) on suicidal ideation severity (ASIQ). The analysis used causal mediation analysis based on parametric regression models.124 The ASIQ suicide measure at 6 months was the outcome variable, and the allocated group (treatment vs. control) was the predictor variable. Scores at 6 months were used for the mediator variables. Baseline scores were control variables. Hence, analyses comprised a linear regression model for each mediator with random allocation group, baseline outcome, baseline mediator and site as covariates, together with separately estimating a linear model for suicidal ideation with the mediator, random allocation group, baseline outcome, baseline mediator and site as covariates.124,125
The effect of random assignment on the mediator was multiplied by the effect of the mediator on suicidal ideation to estimate the indirect effect, and the effect of random assignment on suicidal ideation in the model is an estimate of the direct effect.65 The addition of the indirect and direct effects equate to the total effect. The proportion mediated is the indirect effect divided by the total effect.124 Bootstrapping was used to obtain valid standard errors (SEs) for the causal effects. Ninety-five per cent CIs were based on the percentile of the bootstrap distribution.
Compliance-adjusted analyses
To account for departures from random allocation in the treatment (therapy) group, two compliance-adjusted analyses were performed:126 (1) a binary compliance measure and (2) a continuous measure of number of therapy sessions received. The first measure of compliance included anyone who received at least one session of therapy, but excluded anyone assigned to the treatment group who had no therapy sessions. The treatment effect is interpreted as the complier-average causal effect, where complier is defined as those participants randomised to the treatment (therapy) group who received at least one session of therapy and those participants randomised to TAU who would have received at least one session of therapy had they been randomised to CARMS (a counterfactual, based on predictions from a model). The treatment effect is the adjusted mean difference between randomised groups within this subgroup of compliers.
The second measure of compliance was the number of sessions of therapy attended. This is observed for all participants in the treatment (therapy) group and is fixed by design at zero in the TAU group. The treatment effect is the effect of one additional session of therapy on the outcome variable, assuming a linear effect, for example, going from s sessions to s + 1 sessions for any s between zero and the maximum number of sessions.
Both analyses used a two-stage instrumental variables approach. In stage one, the treatment receipt measure is regressed on random assignment, baseline primary suicide outcome variable (ASIQ) and NHS site. In the second stage, this predicted value is included in the primary analysis models in place of the randomisation variable. Both models are estimated in a single bootstrap procedure to produce valid SEs for the effect of treatment received, with resampling at the level of participant.127
Health economics analyses plan: exploratory
It should be noted that this analysis was exploratory. The cost-effectiveness of the treatment intervention was compared with TAU using an ITT approach and from an NHS and social care perspective. The costs for the economic evaluation included the costs associated with NHS and social care resources used by participants in the post-intervention period (between months 6 and 12 of the follow-up period) and the direct costs associated with delivering the intervention during the baseline to 6 months’ period. Resource data collected 6 months prior to baseline assessments were included as a covariate in the regression modelling analyses to adjust for prior differences in resource use.128 The intervention-related costs covered training for staff and time spent providing therapy sessions. Regression models, based on complete cases, were used to estimate net costs (generalised linear model with gamma family and log-link) and quality-adjusted life-years (QALYs) (linear model) for the treatment group compared with the TAU group. Models were adjusted for study site, antidepressant medication use and baseline values (utility value or costs, respectively) and incorporated clustering by therapist. The measure of health benefit was utility, derived from EQ-5D-5L baseline and the 12-month assessment using the mapping approach as currently recommended by the National Institute for Health and Care Excellence (NICE).129 QALYs were calculated from these utility values using an area-under-the-curve approach.