Study design
This was an external pilot two-arm parallel-group individually randomised controlled trial with a nested qualitative process evaluation and economic evaluation.
A number of modifications were made throughout the delivery of the pilot RCT (as detailed in Chapter 7). The majority of these were considered minor and did not involve changes to key aspects of the study design. The analysis of the qualitative data was initially intended to involve both a thematic analysis and a NPT framework analysis; however, a thematic analysis was not conducted as the NPT framework45 appeared to capture the interview data sufficiently well (see Chapter 9, Data analysis, for a detailed justification).
Study approvals
Study approvals are detailed in Chapter 2.
Study sites
Fifteen CPs in the north of England took part. Thirteen CPs were involved from the commencement of the study and two additional CPs were recruited during the recruitment period. Six CPs were involved in both the feasibility study and the pilot RCT.
Participant eligibility
The inclusion and exclusion criteria were as described for the feasibility study (see Chapter 2).
Participant recruitment
Four CP-based recruitment methods and one GP-based recruitment method were employed for the pilot RCT. These are described in Chapter 2 (see Report Supplementary Material 7 for the participant study information pack for the pilot RCT). Refinements were made following the feasibility study, including removing the use of the BIS (see Chapter 5).
Screening for eligibility
The process of screening for participant eligibility was as detailed in Chapter 2.
Baseline assessment
The process for participant baseline assessment was as described in Chapter 2, with the exception that participants could opt to complete the baseline questionnaire over the telephone with a study researcher or complete and return the questionnaire via post (see Chapter 5; see Appendix 4 for non-validated baseline pilot RCT questionnaires).
Randomisation
Individual randomisation was employed and carried out by the York Trials Unit online randomisation service, independent of the study team. Following completion of the baseline questionnaire, participants were randomised on a 1 : 1 basis to either the ESI arm or the usual-care arm (see Intervention and Comparator).
Following randomisation, participants were contacted by the ESI clinical supervisor (or a researcher not involved in follow-up assessments) to advise them of their treatment arm allocation. Participants randomised to the ESI arm were advised that a ‘Healthy Living Advisor’ from the local participating CP would contact them within the next few days to arrange their first pharmacy support session (ESI session). All participants (and their GPs) received a letter confirming their treatment arm allocation; those randomised to the ESI arm also received copies of the ESI materials.
Owing to the nature of the intervention it was not possible to blind participants, ESI facilitators or GPs to participant treatment arm allocation. Study researchers were blinded to participant treatment arm allocation, where possible.
Sample size
To estimate recruitment and attrition, and the standard deviation of the PHQ-9 (the intended primary outcome to be used in a main trial), a sample size of 100 participants was used. Allowing for 20% attrition, with 100 participants, the 95% confidence interval (CI) for this level of attrition would be 8 percentage points and would provide robust estimates of recruitment and follow-up. Equally, this sample size would allow for robust estimates of the standard deviation of the intended primary outcome in this patient population, to inform a future sample size for a main trial.
Intervention
The ESI is described in detail in Chapter 2.
A number of minor adaptations were made to the ESI (as used in the feasibility study) for implementation in the pilot RCT. These are described in detail in Chapter 5.
Comparator
The control arm (‘usual-care’ arm) was the usual primary care management of subthreshold depression offered by the participant’s GP and any other local community provision.
Follow-up
The process for participant follow-up assessment was as described in Chapter 2, with the exception that participants could opt to complete the follow-up questionnaire over the telephone with a study researcher or complete and return the questionnaire via post (see Chapter 5; see Appendix 5 for non-validated pilot RCT follow-up questionnaires). Telephone follow-ups and participant contacts to obtain missing data following receipt of postal questionnaires were conducted by researchers blinded to participant treatment arm allocation, where this was feasible.
Outcome measures
The intended primary and secondary outcome measures were the same as those described in Chapter 2 with the exception of the PHQ-1566 (physical/somatic health problems), which was removed from the pilot RCT (see Chapter 5).
In addition, as part of the process evaluation (see Qualitative study), participants were invited to complete and return (using a pre-paid envelope) the 9-item BADS62 at 5 weeks post randomisation in order to assess response rates (to inform a future definitive RCT).
Study completion and participant withdrawal
The criteria for participant exit from the pilot RCT and the criteria and process for participant withdrawal were the same as reported in Chapter 2.
Clinical data analysis
All planned analysis was detailed in a statistical analysis plan and was agreed with the TSC prior to completion of data collection. A single analysis was performed at the end of the pilot RCT using Stata® v15. Because this was a pilot RCT with a small sample size, the planned analysis was mainly descriptive. The flow of participants through the pilot RCT will be reported. All data are presented descriptively by treatment arm and overall. Continuous data are reported using means, standard deviations, medians, the upper and lower quartiles, and the minimum and maximum. Categorical data are reported as counts and percentages. Baseline data are summarised descriptively, with no formal statistical comparisons undertaken.
Serious adverse events will be detailed by treatment arm.
Primary analysis
The intended primary outcome for the pilot RCT was the PHQ-963 at 4 months post randomisation. This will be summarised descriptively overall and by treatment arm. To explore any potential effect from the ESI, a linear regression model, adjusted for age, gender and baseline PHQ-9 score, will be undertaken. Given the small sample size, the results will be exploratory and should be interpreted with caution.
Secondary outcomes
All secondary outcomes will be reported descriptively, as detailed above. No formal statistical comparisons of these measures will be undertaken.
In addition, the pharmacy effect will be explored using the intracluster correlation coefficient and the associated 95% CI to determine whether or not the effect of the intervention differed across the recruiting CPs.
The average caseload per ESI facilitator will be detailed.
Quantitative outcomes associated with the process evaluation (see Qualitative study) will be reported descriptively.
Interim analysis and stopping rules
No interim analysis was planned for this study and there were no stopping rules in place.
Qualitative study
The pilot RCT included a qualitative process evaluation utilising semistructured interviews with study participants, pharmacy staff (including ESI facilitators) and GPs. The aim of the process evaluation was to explore the acceptability of the ESI within CPs, the elements of the intervention that were considered useful (or not) and the appropriateness of study procedures.
Participants
The aim was to purposively sample up to 15 study participants who completed the ESI to explore acceptability and important elements of the ESI, and up to 15 study participants who started the ESI but then dropped out (had less than two sessions) to explore their reasons for disengagement. All study participant interviews were conducted after the proposed primary outcome measure had been collected (4 months post randomisation) and at a time and location convenient to the participant (e.g. at home, in the pharmacy or by telephone).
The aim was to interview up to 20 pharmacy staff (ESI facilitators, general pharmacy staff and pharmacists) to explore important aspects of the ESI, ESI training and clinical supervision, and barriers to/facilitators of study participation and implementation. Interviews were conducted in person in a private room at the pharmacy or over the telephone, according to participant preference, at a time convenient to them.
To ensure confidentiality and to encourage interviewees to speak freely, interviews with ESI facilitators were not conducted with those study researchers who had been involved in their training (either ESI training or general recruitment training). Interviews with general pharmacy staff were conducted by a study researcher who had not previously been in contact with them as part of the study (i.e. had not been involved in their training and was not their CP’s main contact for the study). Study researchers who had had previous contact with interviewees were able to access anonymised interview transcripts only.
Interviews were also planned with up to eight GPs to explore their views and knowledge of the study, the ESI, how this may impact on routine primary care and the barriers to/facilitators of implementation. Interviews took place at a time and location according to GP preference (e.g. in the general practice, over the telephone).
Interview topic guides varied between participant treatment arms, but incorporated core questions used in the feasibility study (see Report Supplementary Material 8). These were developed by the research team and modified iteratively as data generation and analysis proceeded.
Recruitment
Study participants were recruited as described in Chapter 2 (see Report Supplementary Material 7).
Enhanced support intervention facilitators were provided with a study information pack (containing an invite letter, PIS, consent form and prepaid envelope) following completion of the ESI training (see Report Supplementary Material 9 for examples). The remaining pharmacy staff from the recruiting CPs (including general pharmacy staff and pharmacists) were posted a study information pack by the study team (see Report Supplementary Material 9). Interested pharmacy staff were asked to complete and return the consent form to the study team. A reminder letter was sent to all pharmacy staff once ESI delivery had ceased at each pharmacy. This process was carried out independently of the pharmacy to ensure that it was free from pressure from managers or co-workers.
General practitioners of recruited participants and GPs from those practices involved in the identification of potential participants were posted the study information pack (see Report Supplementary Material 10). Interested GPs were asked to complete and return the consent form to the study team.
Analysis
Interviews were recorded, with participant consent, using an encrypted digital audio-recorder and transcribed verbatim independently of the study team. Transcripts were then anonymised by members of the research team to ensure the confidentiality of participants. A framework analysis was conducted using NPT.45 NPT supports the identification of key factors that enable or constrain the successful normalisation of an intervention into routine working practices.77 By applying the NPT domains, the pilot RCT process evaluation explored how participant groups made sense of (coherence) and engaged with (cognitive participation) the ESI. In addition, the work completed and needed to facilitate the intervention (collective action), as well as participant’s appraisal (reflexive monitoring) of the intervention, was explored.
Data were coded directly to the NPT domains (coherence, cognitive participation, collective action and reflexive monitoring) by three researchers (CS, LN and AP). These researchers were from a variety of professional backgrounds to increase the reliability of the analysis,70 and all researchers received supervision from a researcher experienced in the use of NPT (CCG).
Before formal data coding began, general reliability for coding data to the NPT domains was established. This involved researchers independently coding transcripts and then subsequently meeting to discuss and agree coding at a series of meetings supported by CCG. Although it is noted that different interpretations of the NPT domains can occur, this exercise was completed to begin to immerse researchers in the NPT framework and data engagement, as well as to establish a general agreement on coding. The coding process was supported by regular memo taking, with the purpose of recording emergent understanding of the data and to inform discussion and analysis between researchers.
Following the coding process, the analysis drew on principles of the constant comparison method.69 This process involved comparing and contrasting coding both within and across participant groups under each NPT domain, with the purpose to explicate phenomena within NPT categories. This was an iterative process that also involved moving back and forth between the meaning of the NPT categories described by Murray et al.77 and the emergent phenomena observed in the data. Deviant case analysis was sought throughout the process. Subsequently derived categories and phenomena with NPT domains were discussed and agreed by researchers (CS and CCG) and are presented in Chapter 9.
A future definitive RCT would involve a mixed methods process evaluation incorporating mediational and moderational analyses to investigate mechanisms of change.78 Mediational analysis would examine outcomes such as number of ESI sessions, content of ESI sessions and changes in activation (using the BADS). Moderational analyses would examine baseline variables that may moderate outcomes such as age, depression onset age, number of episodes and socioeconomic status. Although the current pilot RCT would be underpowered to undertake these analyses, data will be reported descriptively, where feasible.
Health economic analysis
The health economic analysis aimed to collect resource use and quality-of-life data to estimate levels and changes in costs and health outcomes over the study period to understand the potential economic impact of the ESI in a RCT. Resource use data focused on primary and secondary care services, including mental health services; service use data were multiplied by unit costs to arrive at the total cost in each treatment arm. Unit costs of health service use were obtained from the UK national database of reference costs, the Department of Health and Social Care,79 and the Unit Costs of Health and Social Care report produced by the Personal and Social Services Resource Unit.80 All costs are reported for year 2018–19 using Great British pounds. Quality-of-life data were collected using the standard EQ-5D-3L and SF-12v267,73 questionnaires; these were then used to estimate health-related quality of life (i.e. utility) using the UK tariff.73 The pilot RCT analysis was not intended to provide a definite estimate of relative cost-effectiveness but to inform a larger RCT.
The pilot RCT used the same bespoke resource use, EQ-5D and SF-12v2 questionnaires, as the feasibility study (except for the addition of a question on the number of ‘pharmacy visits’ in the resource use questionnaire; this was based on feedback from the feasibility study). Individual-level data were collected on health service resource use (including primary, community-based and secondary care services, including mental health consultations), as well as quality of life. In addition, intervention-related data were collected in terms of the time spent by the ESI facilitator with each participant to deliver the ESI (i.e. number of ESI sessions multiplied by the duration of each session + total administration time and clinical supervision time to deliver the ESI). Resource use was multiplied by unit costs to arrive at the total cost in each treatment arm. Participants were offered up to six sessions with an ESI facilitator. ESI sessions were delivered at the individual level and the length of each session was recorded. The time spent by the ESI facilitator delivering the session was multiplied by their hourly rate.
The analysis was conducted from the UK health services perspective. Data were collected at baseline and at 4 months post randomisation. The analysis of resource use and health-related quality-of-life data was conducted in terms of the overall response rate for each questionnaire, the rate of missing items within each questionnaire and the level of health service resource use and health-related quality of life (by item/domain) at baseline and at the 4-month follow-up.
Serious adverse events
The process for the identification, reporting and reviewing of suspected SAEs was the same as that detailed in Chapter 2.
Risk management
The same study-specific risk protocols for the identification, assessment and reporting of suicide or self-harm were followed as those described in Chapter 2.
Study training
The training of study researchers and ESI facilitators was as described in Chapter 2.
Study researchers were informed of any changes to study procedures from the feasibility study, and any study researchers who had not worked on the feasibility study received training on all study procedures.
Enhanced support intervention facilitators who had not taken part in the feasibility study completed the 2-day ESI training workshop (see Chapter 5, see Intervention and see Tables 14 and 15 for a description of the adaptions made to the pilot RCT ESI training workshop). Those ESI facilitators who had taken part in the feasibility study and were continuing in that role in the pilot RCT were offered telephone-based refresher training.
Study researchers delivered face-to-face general study and recruitment training to all those recruiting CPs who had not taken part in the feasibility study. Study training was adapted following the feasibility study and was streamlined and delivered over two shorter sessions to aid understanding and to encourage discussion regarding potential challenges to study delivery (see Chapter 5 for a detailed description of these study training adaptations).
Those CPs that were involved in the feasibility study and that were continuing to the pilot RCT were given face-to-face refresher training lasting around 1 hour to update CP staff on the changes made to the study and recruitment procedures from the feasibility study.
Patient and public involvement
There was PPI throughout the duration of CHEMIST (feasibility study and pilot RCT). This is described in detail in Chapter 2.