This process evaluation protocol and results are available in Appendices 17 and 18,69,70 and as an archived full report available in Report Supplementary Material 15, prepared ahead of the main results being available. For the process evaluation we have included all 736 randomised participants.
The aims of the process evaluation of the main trial were to:
- assist in the interpretation of the results of the main effectiveness trial
- develop a set of transferable principles regarding the intervention to inform its implementation on a wider scale, if the intervention proves to be effective.
Methods
We used a mixed-methods approach. Quantitatively, we described reach/context, recruitment, dose delivered, dose received and fidelity. Qualitatively, we explored the experiences of participants, intervention facilitators and GPs about their involvement in the trial (see Table 4).
| Key process evaluation components | Source of data | Type of data |
|---|---|---|
| Reach and context | NHS GP practice data and trial data | Practice numbers and location. Census and national statistics |
| Recruitment | Trial recruitment data | Routine trial dataSample of expression of interest forms from those who declined to participate |
| Dose delivered | Trial intervention delivery records | Groups delivered/not delivered and whyLocation of groups |
| Dose received | Trial intervention attendance sheetsTrial data | Attendance dataReasons given for not attending |
| Fidelity | Intervention group audio-recordingsParticipants one-to-one consultation forms GP feedback forms | Audio-recording data10% form completion check for adherence |
| Impact of intervention | Participant interviews | Interview transcripts |
| Experience of participating in the trial | Staff interview/focus groups Participant interviews Participant feedback forms GP feedback forms | Intervention staff focus group notes and recordings Patient interview recordings/transcripts Participant feedback GP feedback |
Results
Reach/context
The intervention team delivered 42 (2-day) group sessions. Of the 380 (migraine plus tension-type headache) participants allocated to the 2-day group sessions, 288 (76%) attended at least part of the 2-day course, and 92 (24%) were not exposed to the CHESS intervention at all. Of the 288 who did attend the group sessions, 227 (79%) attended both days, whereas 61 (21%) only attended day 1. Of the 288 who took part in at least one session, 261 (91%) had a one–to-one interaction with the nurse. Overall, 261 out of 380 (69%) participants achieved the predefined minimum dose (attended at least some of the course and the one-to-one discussion with the nurse). Only 217 (57%) fully adhered to the intervention. Intervention fidelity was good, with adherence being slightly better than competence [adherence, 83% (IQR 67–100%); competence, 70% (IQR 50–90%)].
Assessment of a 10% (n = 27) random sample of the case report forms completed during the one-to-one sessions found that the sessions were fully completed as required by the protocol.
Written feedback on the 2-day group session and the one-to-one session with the nurse was provided by 117 participants. There were high levels of satisfaction with the course overall and with the facilitators, although there were lower satisfaction scores for the venues, the relaxation and taster sessions, and the mindfulness session. Similar views were expressed during interviews.
During focus groups, intervention facilitators reported that they found some sessions more challenging to deliver than others, notably sessions on acceptance, impact of thoughts mood and emotions on headaches, mindfulness and relaxation for headaches, medication management and managing setbacks.
Twenty-eight participants took part in the interview study exploring their experience of the trial (self-management support group, n = 17; standard-care group, n = 9; randomised to intervention but did not attend any sessions, n = 2) soon after receiving the intervention and 12 months after randomisation. In the first interview, all participants were generally positive about the intervention they experienced. Group participants liked the group format and valued meeting with others to share information. Those with advanced headache knowledge found the group confirmatory, but they were positive about the opportunity to discuss experiences with others. Some interviewees felt that people earlier in their headache trajectory might have more to gain from the intervention. There were people, all with high levels of knowledge about headache, who would have liked a session on where to find information about cutting-edge treatments. The most popular sessions were on lifestyle, stress and anxiety, and sleep, as interviewees felt that they had gained understanding of how these may affect headaches. Many participants did not like the sessions on mindfulness and relaxation for headaches and on managing setbacks. Participants also valued being able to review their headaches and management at the one-to-one sessions with the nurse.
Interview data at 12 months indicated that, although some participants had made changes to how they managed their headaches, such as changing medications and recognising triggers, most had changed very little. Among those who had made little change, there were some for whom life had become more problematic with family issues, work and other health issues.
Conclusion of process evaluation
The process evaluation suggests that CHESS reached a diverse population across different geographical settings. Attendance reached our predefined dose, but many participants were not exposed to the intervention. The interventions were delivered with fidelity and, although generally well received, with some sessions liked more than others, both intervention facilitators and participants had reservations about several components of the course. There is evidence that participants valued the group and one-to-one aspects of the intervention giving them the opportunity to explore and review their headaches and its management. The process evaluation provides no clear explanation as to why the CHESS intervention appears ineffective.