Methods

Parts of this chapter have been reproduced from Gooding et al.65,69 These are Open Access articles distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) licence, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The text below includes minor additions and formatting changes to the original text.

Climate, health and sustainability

Throughout the CARMS project, we took measures to (1) reduce paper usage with online notes, agendas and meeting minutes; (2) hold meetings, for example, Trial Steering Committee (TSC), Data Monitoring and Ethics Committee (DMEC), CARMers online as opposed to in person, which removed the need for travel and (3) as a consequence of the pandemic lockdowns learnt that online assessments were often preferred by participants, which had the additional advantages of reducing paper usage and car travel. In addition, the CARMS project made no use of generative Artificial Intelligence (AI) which has a high water usage, with data centres often in places facing water shortages.

Patient and public involvement and engagement and equality, diversity, inclusion and antiracist approach

A bespoke group of Experts-By-Experience (EBE) provided input to the CARMS project in the early stages of development of the Efficacy Mechanism and Evaluation (EME) application for funding in 2014. Many of these individuals also continued to work with CARMS after funding had been secured and were joined by new members to form a Service User Reference Group (SURG), self-named the CARMers, comprising around 8–10 members. The CARMers represented a diverse array of experiences, including mental health problems (e.g. suicidal thoughts and attempts, psychosis, depression, anxiety, substance use); ethnicity; age; gender and sexual orientation. Over the course of the CARMS project, the CARMers met approximately monthly, with the frequency of meetings being increased or decreased depending on the stage of CARMS and the contribution needed.

The CARMers provided input into all stages of the research process, with the following being indicative but not exhaustive: developing the research questions; choice of outcome measures; advising on ethical issues and applying for Research Ethics Committee (REC) and Health Research Authority (HRA) approvals, including the layout and wording of participant/patient information sheets (PISs) and consent forms; developing and evaluating recruitment and retention strategies (e.g. which mental health teams to target, and how to approach them); designing recruitment materials (e.g. information leaflets for participants and mental health professionals); advising on communication strategies (e.g. newsletters, providing workshops to NHS Trusts); designing the CARMS website; giving conference presentations; giving feedback on therapy content and processes; training research staff; providing input into aspects of the analysis and interpretation of qualitative findings; and developing the dissemination strategy. The CARMers were paid for their time and were reimbursed for travel and information technology expenses in accord with NIHR guidelines.

Design

The design of CARMS was developed with reference to Consolidated Standards of Reporting Trials (CONSORT)70 and Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guidelines,71 and the Template for Intervention Description and Replication (TIDieR) checklist and guide.72 The RCT component of CARMS was single-masked with two parallel arms, namely the suicide-focused psychological intervention for psychosis (CBSPp) plus TAU (treatment group) versus TAU only (control group). Participants were randomised to one of the two trial groups, that is, either the treatment or the control group with stratification based on NHS recruitment site and the use of antidepressant medication (yes/no). There were three assessment time points of baseline and 6 and 12 months. Outcomes were assessed at all three time points. The primary suicide outcome variable measured at 6 months was used to assess efficacy. Four qualitative and mixed-methods workstreams were nested within the RCT design, which examined acceptability, implementation, psychological mechanisms and the experiences of participants involved in a suicide trial.

Settings

Sites

Cognitive AppRoaches to coMbatting Suicidality was positioned across four NHS Trust sites located in the North West (NW) of England, UK, which were Greater Manchester Mental Health NHS Foundation Trust (GMMH), Pennine Care NHS Foundation Trust (PCFT), Lancashire and South Cumbria NHS Foundation Trust (LSCFT) and Mersey Care NHS Foundation Trust [North West Boroughs Healthcare NHS Foundation Trust (NWBHT) at the time of recruitment inception]. This last trust was able to support CARMS only until October 2018. These sites were chosen to facilitate recruitment from urban and rural UK settings. Outcome assessments, therapy sessions and qualitative interviews took place in the homes of participants, on NHS premises or by telephone, with the latter being a more frequently chosen option during the COVID-19 pandemic.

Clinical Trials Unit

The Manchester Academic Health Science Centre (MAHSC) Clinical Trials Unit (CTU) (MCTU) conducted trial procedures, including carrying out the randomisation procedures; case report form (CRF) development; designing and building databases; classifying and recording adverse events (AEs) and serious adverse events (SAEs); data entry; and data quality control procedures.

Study population

The inclusion criteria were: (1) meeting International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) criteria for a non-affective psychosis (F20–F29),73 (2) self-reported suicidal experiences (e.g. thoughts, urges, compulsions, images, plans, acts/attempts) in the 3 months prior to recruitment into CARMS, (3) in contact with NHS mental health services and under the care of a mental health services team (e.g. community mental health, early intervention, home treatment, psychiatric inpatient mental health care), (4) aged 18 or over, (5) native English speaking or having sufficient competency of the English language not to need an interpreter to participate and (6) able to give informed consent as assessed by either a responsible clinician or by CARMS research staff following the guidelines of the British Psychological Society (BPS) on gaining informed consent (BPS Code of Human Research Ethics).65

The exclusion criteria were: (1) dementia or organic brain disorder as defined by ICD-10, (2) unable to complete assessments due to language difficulties and (3) currently participating in a psychological intervention clinical trial.

The withdrawal criteria were: (1) the participant decides to withdraw their data from the trial for any or no reason and (2) the participant is lost to follow-up. However, data were intended to be included in the analyses up to the point of withdrawal with appropriate consent.65

Sample sizes

Efficacy sample size

Based on a power of 80%, the target sample size was 250 participants in total, with 125 in each of the treatment and control groups, for the primary analyses at 6 months across recruitment sites. A clinically meaningful difference in the primary suicide outcome variable Adult Suicidal Ideation Questionnaire (ASIQ) from baseline to the 6-month follow-up time point was estimated as a 16-point reduction, which corresponds to an effect size of 0.42. We accounted for clustering in the treatment group with an intraclass coefficient (ICC) = 0.02, with 6 therapists seeing an average of 21 clients/participants and a variance of 21 participants (assuming a Poisson process and with no clustering in the TAU arm); 1 : 1 allocation; and a 0.05 significance level. An attrition rate of 25% was initially estimated from baseline to the 6-month follow-up time point, meaning a total recruitment of 333 was required. However, retention levels consistently exceeded 75%. Consequently, the target recruitment sample size was adjusted to 295, with the aim of retaining 250 participants at both 6- and 12-month follow-up time points.

Mechanism sample size

A sample size of over 250 was calculated to have > 80% power to detect a proportion mediated of 35%, and over 70% power to detect a proportion mediated of 30% [(using the powerMediation package in R. R Foundation for Statistical Computing, Vienna, Austria)]. Hence, the target sample size was considered to have sufficient power for the proposed mediation analysis.

Qualitative workstreams: sample sizes

Based on previous work with participants with severe mental health problems and suicidal experiences,2,61,74–76 it was anticipated that sample sizes of 20–25 participants would suffice. Sampling was planned to occur in parallel with data analysis and cease at the point at which new data appeared redundant.77

Methods to examine and protect against sources of bias in the randomised controlled trial

The following measures were put in place in an attempt to mitigate potential sources of bias.78,79

The randomisation procedure

Random allocation to groups was at the level of the individual participant. Participants were randomised to one of two trial groups, with stratification based on use of antidepressant medication (yes/no) and NHS site. Randomisation took place using an online Sealed Envelope [MCTU: MAHSC (Manchester Academic Health Science Centre), Clinical Trials Unit, Manchester, UK. https://www.sealedenvelope.com/] randomisation procedure conducted and overseen by the CTU. This system was accessed only by specially allocated staff (e.g. the CARMS Trial Manager, the CTU Trial Manager). When the system randomised a participant, an unmasked e-mail confirmation was sent to the unmasked co-principal investigator (co-PI) and the CARMS Trial Manager (also unmasked). Following randomisation, the participant was given a unique trial identifier (ID) which was used on all trial documents (e.g. CRF, SAE forms and so on). This ID was sequential across all NHS sites.

Protecting against selection bias

1. The CTU (MCTU) performed the randomisation procedures.
2. Randomisation took place after potential participants had been screened for eligibility, recruited, that is, had consented to participate in the RCT, and had completed the baseline outcome assessments.
3. Participants were given their randomised allocation group (treatment or control) by one of the trial therapists or the Trial Manager.

Protecting against detection bias

1. Care co-ordinators were informed of a participant’s allocation group upon randomisation. Although this is a form of bias, it was unavoidable because therapists needed to pass on information about their clients to the participant’s care team.
2. Masking/concealment procedures were as follows:
   1. Research assistants (RAs) performing the outcome assessments, the trial statistician, MCTU personnel handling data and one of the co-PIs were masked with respect to the randomisation group of participant allocation.
   2. All instances of unmasking were recorded.
   3. Follow-up appointments were arranged by unmasked staff (e.g. the CARMS Trial Manager), during which participants were asked not to reveal their allocation (treatment vs. control) to RAs during the 6- and 12-month outcome assessment sessions. In addition, RAs carried a large reminder card, asking participants not to reveal their allocation, which was used during face-to-face appointments.
   4. Masked RAs were housed separately from unmasked CARMS staff (e.g. the CARMS Trial Manager, the therapists), where conversations about participants could not be overheard, including telephone conversations.
   5. CARMS procedures were developed and instituted for situations in which a RA performing the outcome assessments became unmasked. As soon as a RA became unmasked, they stopped doing the assessments with the participant and informed the Trial Manager at the earliest opportunity. A back-up, masked CARMS RA, or a researcher independent of CARMS, then took over doing the assessments at the next appointment with the participant. Replacement of RAs doing assessments were arranged according to a random schedule even when there had not been a breach, so that RAs could not easily deduce that a breach had actually occurred.

Examining possible attrition biases

1. Participant throughput was documented, and, where possible, reasons were recorded about why participants/potential participants opted not to continue.

Protecting against reporting biases

1. A statistical analysis plan (SAP) was approved by the CARMS co-PIs, the TSC and the DMEC in advance of data collection completion, data lockdown and the quantitative analyses ensuing.
2. The MCTU developed and implemented procedures for database build, data input and data validation.

Measures

Routine National Health Service data

Participant data routinely collected by the NHS were not analysed as outcome variables by the CARMS project. However, we did have access to information contained in clinical notes which we could add to as necessary (e.g. risk information).

Cognitive AppRoaches to coMbatting Suicidality screening

A screening checklist was used to establish if a potential participant met eligibility criteria. Information from participant’s self-reports during a screening appointment was collated together with information from their care co-ordinator. The checklist required information to be recorded about any suicidal experiences in the past 3 months and information relating to diagnosis.

Assessment of mood

A visual analogue scale (VAS), ranging from 0 (worst mood) to 100 (best mood), of mood provides a very fast indication of current mood states and was used prior to and on completion of all assessment sessions/appointments and qualitative interviews. This was used to assess the impact of assessments on the mood states of participants and take action as appropriate.

Demographic information

Self-reported ethnicity (e.g. Asian, Caucasian), employment status (e.g. paid employment, volunteer, exempt due to disability), relationship type (e.g. single, divorced/separated), dependants, living arrangements (e.g. alone, with partner/spouse), highest educational attainment/qualification (e.g. tertiary, secondary) and any current engagement in other psychological therapies (e.g. CBT, dialectical behaviour therapy, motivational interviewing, emotion-focused therapy, family therapy, psychodynamic therapy, supportive counselling) were recorded.

Primary outcome suicide measure

The primary outcome measure was the ASIQ,80 which is a self-report 25-item questionnaire. The questionnaire is designed to measure a range of suicidal thoughts from mild to severe, with severe thoughts being indicative of high lethality. It includes cognitions reflecting a desire not to be alive (e.g. ‘I thought it would be better if I was not alive’; ‘I thought about having a bad accident’), in addition to more specific thoughts about intent to die by suicide (e.g. ‘I thought about killing myself’; ‘I thought about what to write in a suicide note’) and the role of other people (e.g. ‘I thought that others would be better off if I was dead’; ‘I thought that no one cared if I lived or died’). Participants are asked to report the frequency of thoughts about death and suicide in the last month using a seven-point Likert scale (possible range 25–150). Internal consistency was documented as 0.97 for individuals attending psychiatric outpatient clinics with a history of suicide attempts. Test–retest reliability was also reported to be high (r = 0.95).80 For the current data set, the alpha baseline reliability was 0.96.

Secondary suicide outcome measures

1. The Suicide Probability Scale (SPS)81 has 36 self-report items. It uses a binary response format (true/false) to measure suicidal ideation (e.g. ‘I think of suicide’), hopelessness (e.g. ‘I feel the world is not worth continuing to live in’), negative self-evaluations (e.g. ‘I feel I am not able to do many things well’) and hostility (e.g. ‘I feel hostile towards others’). The internal reliability for a sample, including psychiatric inpatients, was 0.93.81 In the current study, this value was 0.89.
2. The Beck Scale for Suicidal Ideation (BSSI)82 is self-administered and measures recent suicidal thoughts, plans and intent over the past week (19 items) (e.g. ‘Do you have strong, moderate or weak wish to die?’; ‘Do you have a plan?’), together with previous attempt history and severity of the last attempt (two items). Each item is scored using a scale from 0 to 2 (possible range 0–38 for ideation; 0.42 for all items). An alpha coefficient of 0.87 has been previously reported in a sample of psychiatric outpatients,82 and 0.96 with those with non-affective psychosis.83 The alpha coefficient in the current study was 0.90 for the 19 ideation items, and 0.88 for the total 21 items.
3. Timeline Followback (TLFB) for suicidal experiences84,85 is a calendar method adapted from substance use research in which participants are asked to estimate their frequency of (1) suicide attempts (TLFB A), (2) serious self-harm incidents requiring first-aid (TLFB B), (3) concrete suicide plans (TLFB C) and (4) suicidal thoughts where they had spoken with someone about those thoughts and/or contacted mental health charities because of their suicidal experiences (e.g. Samaritans) in the previous 6 months (TLFB D).

Mechanistic outcome measures

There were six mechanistic outcome questionnaire measures, all of which were self-report.

1. The Difficulties in Emotional Regulation Scale (DERS)86 measures perceptions of emotional control, including reflective capacity and/or understanding; acceptance of emotionally related thoughts and feelings; having control of emotional reactions; and feeling able to use effective emotional strategies. Internal reliability was documented as 0.97 in a sample of psychiatric outpatients.87 In the current study, this value was 0.85.
2. The Social Problem-Solving Inventory (SPSI)88 captures appraisals of social problem-solving using 25 items. The SPSI provides a total score in addition to five subscales of positive problem orientation (SPSI-PPO), negative problem orientation (SPSI-NPO), rational problem-solving (SPSI-RPS), impulsive problem-solving (SPSI-IPS) and avoidant problem-solving styles (SPSI-APS). Alpha reliabilities have been reported as ranging between 0.85 and 0.96.88 Cronbach’s alpha for the total score for this data set was 0.88.
3. The social support appraisals (SS-A) scale89 uses 23 items to assess perceptions of available social support from friends (e.g. ‘I feel a strong bond with my friends’), family (e.g. ‘My family cares for me very much’) and other people more generally (e.g. ‘People admire me’). Cronbach’s alpha reliability values of 0.80 and 0.9089 have been documented. This value was 0.93 for the current data set.
4. The Beck Hopelessness Scale90 measures perceptions of having a negative future with 20, binary choice (yes/no) response options. Reliability estimates have exceeded 0.88 in samples, including those with suicidal thoughts and behaviours and severe mental health problems.90–92 Cronbach’s alpha for this data set was 0.84.
5. The Defeat Scale29 uses 16 items to assess perceptions of feeling defeated (e.g. ‘I feel that I have given up’). In people with non-affective psychosis who also experienced some level of suicidal thoughts and behaviours in the past or currently, the Cronbach’s alpha coefficient was reported as 0.86. Cronbach’s alpha coefficient was 0.94 for the current data set.
6. The Entrapment Scale29 measures perceptions of being trapped (e.g. ‘I can see no way out of my current situation’), again, with 16 items. Previous work with people with psychosis reported a Cronbach’s alpha of 0.95, and was also 0.95 in the current study.

Other mental health symptom measures

There were four symptom measures administered as part of a semistructured clinical interview:

1. Positive and Negative Syndrome Scale (PANSS)93 is a semistructured clinical interview recording severity of positive psychotic symptoms (7 items), negative psychotic symptoms (7 items) and general symptoms of mental health problems (16 items). Inter-rater reliability figures were reported as 0.87, 0.79 and 0.69 for PANSS positive, negative and general subscales in the initial study.93
2. The PANSS was administered by telephone for a subsample of 45 participants due to UK COVID-19 lockdown visiting restrictions. This meant that Items P4 (excitement), N1 (blunted affect), G4 (tension), G5 (mannerisms and posturing), G7 (motor slowing), G11 (poor attention), G13 (disturbance of volition) and G14 (poor impulse control) were missing for these participants due to the need for observation of the participant.
3. In the current study, alpha coefficient scores were 0.64 (0.66 with P4 excluded), 0.66 (0.58 with N1 excluded), 0.62 (0.59, excluding G4, G5, G7, G11, G13 and G14) for the positive, negative and general items, respectively. Only data for complete cases have been documented.
4. The Psychotic Symptom Rating Scales (PSYRATS)94 measure a range of dimensions associated with hallucinations and delusions. It has auditory hallucination (11 items) and delusion (6 items) subscales. The alpha coefficients were high in the original study (> 0.90 for both dimensions).
5. The Personal and Social Performance Scale (PSP)95 measures domains of (1) socially useful activities (e.g. housework, volunteering, studying), (2) personal and social relationships [e.g. family, friends, partner(s)], (3) self-care and personal hygiene and (4) disturbing and/or aggressive behaviour. The PSP has excellent face validity and reliability, with the majority of ratings taking place in those with schizophrenia.95
6. The Calgary Depression Scale (CDS)96 was specifically designed for people with schizophrenia and was reported as having excellent reliability estimates.96

Reliability of clinical measures for psychosis

Upon completion of training, RAs rated 10 clinical interviews to assess their reliability for rating the clinical measures compared with the Mean Gold Standard. Table 1 shows the reliability ratings after training.

The average ICCs calculated for the group of 11 RAs upon completion of training was 0.93, and 0.90 for the PSYRATS delusion and hallucination scales.

Self-report measures for substance use and sleep quality

There were four additional clinically relevant measures administered:

1. The Alcohol Use Disorders Identification Test97 is self-administered and uses 10 items scored from 0 to 4 to measure harm due to drinking alcohol.
2. The Drug Abuse Screening Test (DAST)98 20-item version assesses problems arising due to drug use with binary (0 or 1) response items.
3. TLFB99 for substance use asks participants to estimate the number of units of alcohol consumed in the last 3 months, the number of days abstinent in the last 3 months, the number of days abstinent from the most frequently used drug and the number of days abstinent from all drugs.
4. Reasons for substance use in schizophrenia scale100 has three subscales: (1) coping with distressing emotions and symptoms (18 items), (2) social enhancement and intoxication (11 items) and (3) individual enhancement (9 items).
5. The Sleep Condition Indicator101 uses eight items to measure sleep problems based on the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition criteria for insomnia disorder.

Measures added as a consequence of COVID-19

Information relating to the impact of COVID-19 was collected from participants in three ways after UK lockdown restrictions were enforced in March 2020:

1. During the structured clinical interviews and in the therapy sessions, participants were asked if they had any COVID-19 symptoms, if they had been self-isolating, and, if so, the reason for self-isolating (e.g. on government advice as they are in the at-risk group; others in the household have COVID-19 symptoms).
2. Participants were asked to complete the Impact of Events Scale-Revised (IES-R),102 which assesses current personal distress related to a stressful life event, for example, the pandemic.
3. During clinical interview measures (PANSS, PSYRATS, PSP, TLFB for suicide and TLFB for substance use), in addition to documenting their current thoughts, feelings and behaviours, participants were asked to imagine, and rate, how they would have been feeling if COVID-19 was not present.

Therapy process measures of therapeutic alliance, engagement and adherence

The Working Alliance Inventory (WAI) – short form103 completed by the therapist and the participant was administered after approximately four sessions and at the end of therapy, where possible. The higher the score, the more positive the rating of the therapeutic alliance.

Therapy process measures

Both therapist and client were invited to complete a brief rating scale at each session. The therapist sessional scale comprised ratings of (1) the number of sessions attended; (2) the duration of each session in minutes; (3) the main foci within each session (e.g. suicidal experiences, psychotic experiences, emotional processing, interpersonal relationships, problem-solving, defeat, entrapment and/or hopelessness); (4) the therapist’s perception of the stage of therapy the participant had achieved at the session, from a choice of 12 overlapping stages, including ‘building engagement’, ‘establishing a shared understanding’, delivery of an intervention technique, such as cognitive or behavioural strategies, through to ‘developing a staying-well plan’ indicated towards the end of the therapy course and (5) their perception of the level of CBT formulation achieved using a 5-point scale, with 1 indicating that it has not been possible to achieve any shared understanding of the client’s problems to 5 indicating that a comprehensive, shared formulation had been agreed between the client and therapist.104 Therapists also recorded the mode of delivery of the session – that is, face-to-face, telephone, web platform. With a participant’s consent, therapy sessions were audio-recorded. Consent for audio-recording was taken at each session. Finally, clients were invited to complete a brief rating scale at each session, detailing how much they had experienced distress associated with the mechanistic outcomes (interpersonal relationships, emotional difficulties, problem-solving, defeat, entrapment, hopelessness) as well as thoughts about self-harm and/or suicide.

Health economics measures used to address an exploratory aim

Two health economic measures were collected at baseline and at the 12-month follow-up time point. Both sampled self-reported service use in the previous 6 months, that is the 6 months prior to baseline and the 6 months prior to the 12-month follow-up assessment. No data were collected between baseline and the 6-month follow-up point (end of treatment).

The Client Service Receipt Inventory (CSRI)105 retrospectively collects self-reported health and social care use of resources in addition to factors related to participant’s use of accommodation, living situation, income, employment and benefits. The CSRI was modified to reflect the care pathways specific to the participants in the current study, for example, the use of mental healthcare services (community teams, crisis teams, inpatient psychiatric admissions).

EuroQol-5 Dimensions, five-level version

The EuroQol-5 Dimensions, five-level version (EQ-5D-5L) is a generic, preference-based utility measure consisting of five domains: mobility, pain/discomfort, self-care, usual activities and anxiety/depression, and five corresponding levels of severity within each domain. Participants are asked to rate their health over the last week on each domain on a scale of 1 (best) to 5 (worst). The EQ-5D-5L106 is considered an improvement on the earlier version (which used a scale of 1–3), because it is less prone to ceiling effects across a range of health problems.107–110

Procedures: randomised controlled trial and mechanism workstreams

Data management

All trial data were anonymised, with each participant being assigned a unique RCT ID. A Trial Master File (electronic and hard copy) was held by the CARMS Trial Manager at the sponsor NHS site, which was GMMH. An electronic file linking the unique RCT IDs with personal participant details was password protected and encrypted. Trial data were entered into databases by MCTU staff, who were masked to the randomised allocation of participants. Appropriate quality control procedures were carried out by the MCTU during the trial and prior to data-lock, which included a 100% accuracy check on the primary suicide outcome assessment and a 10% check on the ‘non-critical’ data points. Additional measures which were collected during the pandemic (specifically, COVID symptoms, isolation measures and the IES-R) were collated by the CARMS Trial Manager and stored at the host NHS site.

Qualitative interview audio-recordings were stored on University of Manchester encrypted, password-protected shared drives. Once transcribed, and checked, audio-recordings were destroyed. Transcriptions were pseudo-anonymised and stored on University of Manchester encrypted, password-protected shared drives and at the host NHS site.

Therapy process data were collated and held by the CARMS Trial Manager for participants assigned to the treatment group. For each therapy session offered, the following information was provided by the therapist, where possible: date, session number, attendance (e.g. attended, did not attend, cancelled), duration (minutes), stage of therapy, level of CBT formulation, session details/foci, mode of delivery (e.g. face-to-face, telephone, remote video link) and audio-recording availability/consent. Additionally, clients were invited to provide ratings on a brief range of potential factors and psychological processes associated with suicidal outcomes, such as social support, problem-solving, emotional processing, defeat, entrapment and hopelessness. Participants could choose whether any part of their therapy sessions were/were not recorded. Other therapy data, for example, WAI, were collected and stored by the CARMS Trial Manager at the host NHS site. Sessions were recorded on encrypted audio devices and stored securely with access only from unmasked research team staff. (It should be noted that therapy recordings and WAI data were sometimes stored temporarily on the University of Manchester secure server).

Recruitment

Recruitment took place across NHS Trust sites from June 2017 to November 2020, with the last person being randomised in January 2021. The 12-month follow-up assessments were completed in January 2022. Potentially eligible participants were mainly identified by NHS mental health team members working in, for instance, home treatment teams, community mental health teams, early intervention teams, crisis teams and inpatient psychiatric wards. However, potential participants could also be referred to CARMS from mental health charities and by self-referral. Information in the form of flyers, leaflets and posters were distributed to mental healthcare teams and within the community. In addition, information leaflets were designed for, and distributed to, interested potential participants.

There were several recruitment stages: (1) identification of potential participants, usually by care co-ordinators in mental healthcare teams, (2) establishing consent to contact (by the potential participant’s care co-ordinator), (3) referral to the CARMS team (usually by the potential participant’s care co-ordinator), (4) screening for eligibility by CARMS research staff, (5) if eligible, initiating and completing PIS and consent procedures and (6) baseline assessments commencing upon consent. It should be noted that at least 24 hours was left by CARMS research staff between providing and going through the PIS and taking consent.

To assess how information about CARMS had reached participants, and what the potential sources of recruitment were, participants were asked, ‘How did you find out about CARMS?’ Responses were: care co-ordinator = 219 (77.7%); other health professionals = 24 (8.5%); self or other referral (e.g. via newsletters/posters/leaflets, contact with a CARMS team member, referral from other studies) = 23 (8.2%); depot clinics = 3 (1.1%); community support officer on ward = 2 (0.7%); support worker = 2 (0.7%); care worker in supported accommodation = 1 (0.4%) and a CBT therapist = 1 (0.4%).

Mode of assessments

Assessments were carried out by CARMS research staff in the participants’ homes or on NHS premises, dependent on the preferences of participants and/or risk information. Where there were no risks to research staff, assessments were carried out in one-to-one, face-to-face appointments. In instances where risk to research staff was identified, such as potentially unsafe geographical locations due to crime, this was mitigated, where possible, by two members of the research team being present at the appointment. Some participants felt more comfortable if assessments were carried out by CARMS research staff in the presence of their care co-ordinator which was readily accommodated. During the COVID-19 UK lockdown restrictions, assessments were carried out by telephone or using remote digital platforms, for which ethical approval was obtained (see Appendix 1).

Timing of assessments and retention procedures

Participants were offered as many appointments as they felt were necessary in order to complete all of the assessments. Baseline assessments were started as soon as practicable after PIS procedures had been completed and consent had been taken. Follow-up assessment appointments at 6 and 12 months were arranged as soon as possible, but these were endeavoured to be completed within 4 months of the date that the assessment was due. Retention over time between appointments was optimised by providing CARMS newsletters/updates to participants and nearer to the time of the follow-up appointments, with reminder phone calls and/or texts to both the participants and to their care co-ordinators.

As far as possible, assessments commenced with the semistructured clinical interviews (e.g. PANSS, PSYRATS, CDS), because they encouraged participants to talk freely about their mental health problems, which, from experience, was considered to be personally engaging. Suicide questionnaires were spaced across questionnaire measures, with the primary outcome measure being prioritised. Participants were paid £10 on completion of assessments at baseline and at 6- and 12-month follow-up time points (i.e. a total of £30 in payment was possible).

Therapists were provided with copies of completed baseline measures and a summary of key clinical details regarding symptoms, risk and any relevant contact issues prepared by the CARMS staff for those participants in the treatment group prior to therapy sessions ensuing.

Treatment as usual and the treatment suicide-focused therapy intervention Cognitive–Behavioural Suicide Prevention for psychosis

Control group: treatment as usual only

Treatment as usual was in the form of usual clinical care. This was most frequently care from a multidisciplinary mental health team as an outpatient with a named care co-ordinator. It could include assessments of mental health, prescriptions for psychiatric medication, regular monitoring of psychiatric symptoms and inpatient psychiatric care. Where relevant, signposting towards practical resources; support for making contact with other organisations, such as mental health charities; help with joining/exploring social groups and support accessing online resources might have been offered by an individual’s mental healthcare team. It should be noted that we acknowledge the multiple skills of care co-ordinators working in NHS mental healthcare services which can include provision of multiple sources of support, such as psychosocial interventions, and, in some cases, specific therapeutic skills, such as CBT and DBT. That said, < 10% of our participants (see Table 3) were receiving psychological therapy in addition to that offered by the CARMS project, and as is usual within many services, this was provided by specialist practitioners rather than care co-ordinators.

Treatment group: the suicide-focused Cognitive–Behavioural Suicide Prevention for psychosis talking therapy plus treatment as usual

The intervention is based on cognitive–behavioural principles which have been advanced to address suicidal experiences and documented in a detailed, evidenced-based, treatment manual.1 Up to 24 sessions of the suicide-focused therapy (CBSPp) were offered to each participant assigned to the treatment group of the trial, with each session usually being scheduled weekly for approximately 50–60 minutes. However, there was maximal flexibility around both the number of sessions and the duration of each session dependent on each participant’s needs and preferences. This meant that if participants wished, they could pace therapy sessions to be aligned with their personal circumstances, and, for instance, schedule sessions to be spaced less or more frequently than weekly. That said, the routine of a weekly schedule seemed to work well for most participants. Therapy sessions were audio-recorded with a participant’s permission. Participants continued with any TAU (e.g. medication, care co-ordination, social support) while also receiving the suicide-focused therapy (CBSPp).

The CBSPp intervention is delivered in a one-to-one (therapist–client) format using a collaborative, shared formulation-based approach.1,64,111 Initial sessions concentrate on engaging the individual in exploring their mental health and, in particular, their suicidal experiences. Ways in which different types of suicidal thoughts and behaviours interact with a client’s experiences of psychosis, life circumstances and personal history is also explored to develop a shared, albeit evolving and fluctuating, understanding of the dynamics contributing to their suicidal experiences. This collaboratively acquired understanding has the potential to generate different ways of thinking which are considered likely to modify numerous negative appraisals, self-evaluations and self-perceptions. Based on recent psychological models of suicidal experiences,22,54,112 four domains of thoughts, feelings and perceptions are used as a guide, albeit not exclusively, and with a focus on the specific needs of each participant. The four domains are (1) emotionally difficult, overwhelming and/or numbing experiences, (2) perceptions of social isolation and disconnection, (3) perceptions of poor interpersonal problem-solving and (4) feelings of being defeated, humiliated, trapped and hopeless.

Therapist qualifications, characteristics, caseload, training and supervision

Qualifications

In total, 12 therapists worked on CARMS, all of whom were qualified clinical psychologists or cognitive–behaviour therapists meeting the British Association of Behavioural and Cognitive Psychotherapies (BABCP) minimum standards for CBT practice. They were all registered with the Health and Care Professions Council.

Characteristics

All therapists were White/Caucasian; seven were male, and five were female. Length of clinical experience post qualification ranged from 0.5 year to 17 years. Their ages when they commenced working on CARMS ranged from 34 to 60.

Caseload, training, supervision and treatment fidelity

All therapists worked part-time on CARMS (e.g. 3 days a week) and were employed or held an honorary contract with one of the participating NHS Trusts. Across therapists, they had a caseload which ranged from 1 to 10 at any one time. Four [Gillian Haddock (GH), Steven Jones, Fiona Lobban, Daniel Pratt (DP)] of the CARMS coinvestigators were qualified clinical psychologists, with extensive experience of working clinically in NHS mental health services and working with people with psychotic and suicidal experiences. Two coinvestigators (GH and DP) provided training in the suicide-focused therapy (CBSPp) and provided group therapy supervision – usually, weekly – during which therapy sessions were discussed (with the participant’s consent), including focused reflections upon excerpts from therapy session recordings. The therapists also took part in regular peer supervision. In addition, individual supervisory sessions took place with GH or DP, when required. Hence, treatment fidelity was maintained through adherence to the treatment manual,1 listening to, and reflecting on, audio-recorded therapy sessions, and, in addition, some therapy sessions were rated by supervisors using the Cognitive Therapy Scale for Psychosis113 to ensure fidelity to the cognitive–behavioural approach.

Safety procedures

There were three domains where safety was important to consider in the context of working with people who had severe mental health problems, including psychotic and suicidal experiences, which were: (1) identification and recording of AEs and SAEs, (2) duty of care of participants and (3) duty of care of research staff. Tailored CARMS procedures were developed in each of these domains in accordance with Department of Health Research Governance Framework for Health and Social Care 2005, which requires that the principles of good clinical practice (GCP) are applied to all NHS research involving patients/service users (SUs) and that the safety of research participants is given priority at all times.

Identification and recording of adverse events and serious adverse events

All of the University of Manchester’s sponsor responsibilities in relation to AE and SAE identifying, recording, reporting and reviewing were formally delegated to the coinvestigator and co-PIs.

Adverse events were defined as harm, or deterioration in health, occurring to a participant during a trial.114 SAEs were defined as ‘reactions which, in their most severe forms, threaten life or function’,114 (p. 782). In CARMS, a SAE which involved life-threatening self-harm or a suicide attempt were considered, a priori, to be expected SAEs. AEs were categorised in terms of severity (mild, moderate, severe) and seriousness (yes, no). SAEs were qualitatively categorised in terms of (1) self-harm/suicide (self-harm, suicide attempt, other) (2) SAE consequence (death, life-threatening, disability/incapacity, hospitalisation); severity (mild, moderate, severe) and expectedness (expected, unexpected). Adverse reactions included any AE that was probably, possibly or certainly related to the therapeutic intervention and/or research procedures. Similarly, serious adverse reactions were SAEs which were probably, possibly or certainly related to the therapeutic intervention and/or research procedures. All events were further classified in terms of whether they were or were not unexpected, with expected events being predefined. Categorisations of AEs and SAEs took place via discussions with members of the CARMS team (coinvestigators) and with members of the MCTU, with the requirement that at least two members of the CARMS team were clinically qualified and experienced in NHS mental health service settings. Where there was any doubt as to the nature of a classification of an event, the Chair of the TSC provided guidance.

All CARMS staff were trained in procedures relating to the identification and recording of AEs and reactions, with possible events being discussed extensively in supervision. AEs and SAEs were monitored and recorded by the CARMS Trial Manager, research staff and the trial therapists. All AEs and SAEs were reported to the MCTU, to the host NHS site and to the TSC and the DMEC. Incidents of all events were communicated to these bodies via a written report which provided details of the nature of the events.

Duty of care to participants

Identifying and managing risk issues prior to an appointment

It was important to identify risk pertaining to participants on a case-by-case basis, encompassing (1) suicidal thoughts and/or acts, (2) self-harm (with or without intent to die by suicide), (3) potential harm by a participant to other people and (4) potential harm to a participant by other people. Standard operating procedures (SOPs) from site NHS Trusts were adapted for use by CARMS research staff (CARMS SOPs are available from the co-PIs, Gooding and Haddock, on request). A ‘risk checklist’ was used to identify risk issues with a potential participant’s care co-ordinator approximately 24 hours prior to a scheduled appointment. Specific issues discussed included accident and emergency (A&E) attendance for self-harm/suicide attempts; knowledge of suicide attempts where the participant had not gone to A&E; disclosure of suicide plans; knowledge of frequency of occurrence of suicidal thoughts, plans and attempts; information about severity and/or worsening of mental health problems, including any new mental health problems; and potential environmental risks to the RAs. When risk issues arose, research staff discussed with care co-ordinators whether there were safe ways of managing the risk. In addition, risk issues were discussed with research staff in weekly supervision sessions by the co-PIs, where the decision was made to (1) proceed with the appointment, (2) delay or defer the appointment (e.g. until more information could be ascertained about the participant) or (3) not to continue with the appointment. RAs were required to arrange ‘clinical cover’ prior to an appointment with a participant, that is, ensure that a designated clinical member of the CARMS team was available during the appointment should risk issues arise, and that they were available at the end of the session for debriefing.

Identifying and managing risk issues during and at the end of an appointment

Before starting an appointment with a participant, CARMS staff reminded participants of the conditions of confidentiality regarding harm to self and/or other people. Prior to going through assessments, participants were asked to complete a brief VAS of mood.115 Throughout appointments, at regular intervals, CARMS staff explicitly ascertained the participant’s emotional and cognitive state and used frequent breaks as requested by participants and as judged to be helpful by the RA. CARMS procedures2 were used as appropriate, which gave detailed guidance covering how to (1) prevent distress in participants, (2) manage situations in which participants did become distressed and (3) work with situations in which participants disclosed risk, including suicide risk.

At the end of every appointment with a participant, a CARMS debriefing protocol was followed: (1) check with the participant the extent to which they ‘could keep themselves safe’, (2) ask the participant to complete a brief VAS of mood and an adjective checklist, (3) ask the participant about their experience of the session, in particular, talking about mental health problems and suicidal thoughts, plans, acts115 and (4) engage participants in conversation about a non-distressing topic, for example, recent achievements, things they value and/or are proud of or events they are looking forward to.

Passing on risk information

At the end of every appointment with a participant, CARMS staff were required to (1) contact the participant’s care co-ordinator to pass on details about the conduct of the session, including any risk information (if the care co-ordinator was unavailable, then risk information was passed on to the mental healthcare team duty officer) and (2) contact a designated member of the CARMS team to debrief about the session during which risk issues were discussed.

Duty of care to research staff

Ensuring that duty of care to CARMS staff (e.g. RAs) was realised by (1) implementing university and NHS Trust lone worker policies, (2) developing and rigorously following NHS Trust and CARMS safety checking procedures, (3) ensuring that CARMS staff engaged with a debriefing procedure with clinically trained members of the CARMS team or with a CARMS therapist after every assessment appointment or qualitative interview with participants, (4) providing weekly group supervision, (5) providing regular individual supervision, (6) ensuring that CARMS staff completed and passed mandatory GCP modules and (7) training staff extensively in CARMS procedures relevant for working with participants who had psychotic and suicidal experiences.2 Training sessions were ongoing, with the CARMers playing a central role.

Group supervision with the co-PIs took place face to face weekly for CARMS staff doing assessments and interviews, which was increased to remote supervision three times a week during periods of lockdown throughout the COVID-19 pandemic. Group supervision was supplemented with peer-led supervision approximately once a month. In addition, CARMS staff completed formal individual performance reviews and were also offered regular informal individual supervision sessions with the co-PIs. With support from the co-PIs, CARMS RAs developed their own extensive guidelines for ‘staying well’ with respect to both mental and physical health during lockdown home working periods.

Procedures specific to the qualitative and mixed-methods workstreams

Four qualitative research questions were formulated as follows

Efficacy qualitative research questions: (1) what is the acceptability of the suicide-focused therapy, CBSPp, to participants,116 and (2) what are the barriers and facilitators of implementing suicide-focused therapy in NHS mental health services to participants and mental health professionals working in the NHS?117

Mechanistic qualitative research question: (3) what are the pathways underpinning suicidal experiences from the participant’s perspectives of living with psychosis and suicidal thoughts, urges and attempts?46

Clinical qualitative and quantitative mixed-methods research question: (4) to what extent does participating in a research project which was focused on suicidal thoughts and behaviours result in negative experiences and negative emotional reactions?115

At the time of writing, four qualitative papers which address the efficacy and clinical research questions have been published.46,115–117 Procedural details are documented in each of those papers.

In sum:

1. There was a RA and research fellow who were assigned specifically to the qualitative CARMS workstreams.
2. The CARMers fed into many aspects of the qualitative workstreams, including designing information leaflets, designing the PISs and consent forms; developing the initial topic guides; training qualitative interviewers and giving advice on interpretation of the data.
3. Where possible, sampling was purposive using a sampling matrix,118 with the goal of maximising diversity of participant characteristics and experiences. All SU participants had lived experiences of psychosis and suicidal thoughts and behaviours, together with experiences of the intervention for the workstream assessing the acceptability of suicide-focused therapy, CBSPp.116 Health professional participants had a range of experiences of working within relevant mental and social care services. Hence participants across the studies yielded a high level of information power.119
4. Participants were provided with PISs specific to the qualitative research question being addressed and also asked to sign a specific consent form which was in addition to the main consent form used for the CARMS RCT.
5. Qualitative interviews were scheduled so that participants were interviewed after assessments had been completed at baseline or at 6- or 12-month follow-up time points. Interviews were recorded using an encrypted device either in person or, following COVID, by telephone; audio-recordings were transcribed verbatim by a sponsor-approved professional transcription company; a pseudo-anonymisation procedure for the transcripts was followed; and transcripts were checked for accuracy, where possible, by the person carrying out the interviews.
6. Topic guides were based on (1) the lived experiences of the CARMers, (2) previous experiences of the CARMS team working with people with psychosis and suicidal thoughts, feelings and acts and (3) existing literature, especially with respect to identifying gaps which the topic guides could address. Topic guides evolved as interviews progressed.
7. In addressing duty of care to, and safety of, participants, the CARMS team was particularly vigilant of potential amplified distress arising from the suicide- and psychosis-related narratives inherent in the qualitative interviews.
8. Methods to ensure trustworthiness of the final analyses, for example, an audit trail, patient and public involvement and engagement (PPIE) involvement, researcher triangulation,118 were used throughout the analyses.

Procedural changes as a consequence of the COVID-19 pandemic and associated United kingdom lockdown restrictions

Lockdown restrictions due to COVID-19 ensued from 21 March 2020 in the UK. This meant that procedural changes were necessary, which included (1) working with the CARMers, (2) working with mental health professionals, (3) duty of care when working with people who have suicidal experiences, (4) identifying SAEs, (5) training and supervision of research staff and therapists, (6) recruitment and (7) retention. It should be noted that all changes to CARMS protocols and procedures were approved by an NHS ethical committee. Please see Appendix 1 for REC-approved non-substantial amendments in Table 8, and substantial amendments in Table 9, which include amendments necessitated by the pandemic. The changes to key CARMS procedures necessitated by lockdown are provided in Appendix 2, Table 10.