The CHESS programme has been a substantial amount of work over several years. We have advanced our understanding of the challenges of living with chronic headaches and made some progress in developing the methodology for running RCTs of complex interventions for people living with chronic headaches. Specifically, we have:
- reviewed and added to the qualitative literature on the experience of living with chronic headaches
- demonstrated that we can recruit people from the community for trials of headache treatments
- developed an approach to classifying headache disorders using a nurse telephone interview
- reviewed the existing literature on quality-of-life assessment for people with chronic headache
- developed a core outcome set for people living with migraine
- developed a mapping algorithm to improve health economics analyses in headache studies
- reviewed the literature on prognostic factors for people living with chronic headaches
- validated a patient-centred outcome measure for studies of people living with chronic headaches, as opposed to people living with migraine.
Nevertheless, our intervention had no meaningful impact on headache-related quality of life or headache days. This is intensely disappointing for the research team and, more importantly, we do not have anything new to offer people living with what can be a profoundly debilitating, and poorly understood, disorder.
In phase 1 we did all our preparatory work for the planned RCT. This can be broadly divided into new fieldwork and systematic reviews. The fieldwork demonstrated that is possible to recruit people with headache disorders from general practice, providing participants to evaluate our classification interviews, our package of outcome measures and pilot our intervention ahead of the main RCT.
We found in our systematic reviews ample evidence that the disability caused by chronic headaches fits within the same broad theoretical framework used to approach other chronic painful disorders. Although the available literature was sparse, our review of qualitative studies had resonance with that found in other chronic pain disorders such as LBP. It is perhaps surprising that we identified only four qualitative studies of the experience of living with chronic headaches. This contrasts with 187 such studies, largely of chronic musculoskeletal pain, found in a review of reviews of the experience of living with chronic pain.74
There is a need for further work to explore the experience of living with, and receiving care for, chronic headaches. In our review of prognostic factors, we identified potentially modifiable prognostic factors that are common to the wider pain literature: anxiety, depression, poor sleep, stress and poor self-efficacy.21,23 Further evidence supporting the relationship between chronic headaches and chronic musculoskeletal pain comes from our review demonstrating the association between primary headache disorders and persistent LBP.42 Looking more widely at the headache literature, including episodic and chronic migraine and tension-type headache, we found evidence that self-management interventions could have positive effects on headache-related disability, mood and pain intensity, but not on headache frequency.22 A subsequent Cochrane systematic review75 of psychological therapies for episodic or chronic migraine, however, concluded that high-quality research to determine if psychological therapies were effective was absent. The evidence for the use of self-management support programmes for chronic musculoskeletal pain is mixed.25,76 Nevertheless, we had sufficient evidence to consider that a self-management support intervention based on psychological principles might be effective for our population of interest. We were also anticipating that the provision of a diagnostic classification, advice on specific drug treatments for migraine and addressing medication overuse would enhance the overall effect.
Our review of patient-reported outcomes identified a limited evidence base for patient-reported outcomes for use in headache studies.26 Only the HIT-6 had sufficient evidence to support its use in a mixed headache population. We therefore adapted the MSQ v2.1 as a new measure, the CHQLQ, that is about headaches in general. We demonstrated that it had good measurement properties. This may be a better measure than the HIT-6 for trials for studies of people with chronic headaches.
Our work on COSMIG identified that headaches days and headache-related quality of life should be given equal standing.39 For these reasons, it was important to look again at how to measure patient-centred outcomes for our trial. When this work was done, an additional migraine-specific measure became available: the Migraine Functional Impact Questionnaire. This has superior measurement properties to the MSQ v2.1 and was also seen to have greater face validity by our patient partners.77 For these reasons we are recommending this as part of the core outcome set for migraine studies.
As is usual in NIHR trials we included the EuroQoL-5 Dimensions (EQ-5D) to allow the calculation of health utilities for our health economics analysis.44 We had concerns about the use of this in our population because of the unpredictable and rapidly changing pattern of headache disorders. The EQ-5D measures health state ‘today’ and does not take into account the, sometimes large, differences in health states within, or between, days. In our systematic review of PROMs we found only limited evidence to support the use of EQ-5D in headache disorders.26 We did find acceptable evidence of construct validity for the Short Form questionnaire-36 items78 and some limited evidence for the SF-12.46 The longer measurement period of the short-form suite of measures may make them a more appropriate outcome measure. In the light of our concerns about the EQ-5D we included the SF-12 in our final package of outcome measures to also allow us to calculate health utility from the SF-6D. In our mapping study we were to show that health utility values for economic evaluations could be predicted from the HIT-6 or the CHQLQ.
Overall, our work on outcome measures for headache studies has informed the approaches we should use to measure outcomes in headache trials. Importantly we have shown that patient-centred outcomes that assess quality of life need to be given equal weighting with headaches days in future trials of interventions for headaches/migraine. We have identified, and for chronic headaches validated, appropriate outcomes that can be used in future studies. By involving patients in all stages of this work we have been able to ensure its relevance to our population of interest.
The core of phase 2 was our RCT. Because we exceeded our planned sample size, and the effects of clustering in the intervention arm were negligible, we had ample statistical power to identify any clinically important between-group differences. There was no evidence of any positive effect on clinically relevant outcomes at 12 months, when we measured our primary outcome, or at 8 months, with the limits of the 95% CI excluding our target (worthwhile) effect size. Others, in a study of episodic migraine, have suggested that the minimal clinically important between-group difference for the HIT-6 should be 1.5 points.79 Even against these stricter criteria, not directly applicable to our population, we have excluded a worthwhile benefit at 12 months. Neither did any of our sensitivity, or pre-planned subgroup, analyses at 12 months show a possible benefit. However, there was a small beneficial effect on HIT-6 at 4 months (shortly after the individualised one-to-one nurse consultation and follow-up of self-management programme), around half of our target (worthwhile) difference of 2 points. However, the 95% CI limits effectively excluded any possibility of achieving the target difference: –1.0 points (95% CI, –1.91 to –0.006 points). At the same time point there was also an increase in headache days (mean 1.5 days, 95% CI 0.48 to 2.56 days) in the past 28 days. Findings from the overall analysis of 736 randomised participants were not materially different.
It is possible that the HIT-6 was not the best primary outcome measure to use for this trial. We would now prefer the CHQLQ to better capture the overall impact of chronic headaches. However, no difference was seen on any of its three domains at any time point.
Among our secondary outcomes for pain self-efficacy, there was a treatment benefit at 4 and 12 months but not at 8 months. It is possible this indicates that our intervention affected one of our main intermediate outcomes without feeding through as patient benefit, although these may also be chance observations because of the large number of comparisons made.
It was disappointing that only three-quarters of those randomised to the self-management support intervention attended any sessions. All participants had confirmed availability for sessions prior to randomisation. Poor attendance is common in trials of group interventions for chronic pain. In two similar previous studies we observed non-attendance rates of 11%80 and 17%.25 However, the minimal adherence rate observed in CHESS (69%) compares favourably with that observed on our two previous studies (70%25 and 63%).80 The effect sizes observed in our CACE analysis did not differ materially from the primary analysis, suggesting that poor attendance was not the explanation for our disappointing findings.
Surprisingly, although there was only a modest delay (median 8 days) between being assessed for study entry, when the presence of chronic headache was established, and completion of the baseline questionnaire, just 62% of those randomised reported headaches on at least 15 of the previous 28 days. This might reflect some response shift in questionnaire completions, or short-term variability in headache days.13,81 In practical terms, it is the population we recruited that would be offered this intervention if it had been successful. So this is not of great concern.
Another important aspect of the preparatory work we did for our trial was to develop an approach to classification that allowed us to phenotype our trial entrants. Although many headache classification tools are available, we did not find any fit for our purpose.15 We needed a tool that would both screen out people with headache disorders other than migraine and tension-type headache, and positively phenotype those with migraine. We therefore needed to develop our own tool. These classification interviews were not a substitute for a full diagnostic consultation informed by a prospectively completed headache diary. Nevertheless, at the point in the care pathway where there was a need to identify those who might need further consideration of headache disorders other than migraine or tension-type headache, and to identify those likely to have migraine, they represented the unstructured approach typically used in primary care. Our approach has the potential for implementation in primary care. Beyond this current grant we have secured additional Programme Grants for Applied Research (PGfAR) Programme Development Grant (PDG) funding from NIHR (PGfAR PDG NIHR202614) to adapt this for online use by people with headaches to allow self-classification of migraine.
Developing our intervention programme was another success for the feasibility stage. Finding that we could not use lay co-facilitators was disappointing, but finding this out in the feasibility phase allowed adaption for the main study.
After performing analyses of multiple outcomes at multiple time points and including multiple sensitivity and subgroup analyses, we could not find any evidence of any clinically relevant positive effects from the CHESS intervention. We have conclusively demonstrated that our intervention is ineffective. This is a surprising finding, as the CHESS intervention had a good theoretical underpinning, targeted modifiable psychological variables and was well regarded by the participants (see Appendix 18). Furthermore, the educational and classification components of the intervention should have led to increased use of prophylactic medications and a reduction in medication overuse.
The control intervention was more than just usual care: we gave the results of the classification interview to participants and their GPs. If people in the control group used medication more appropriately as a consequence of this information, then this might have reduced the apparent effect size. That there was no difference in the use of prophylactic medications in either group over time makes this unlikely.
Our base-case cost-effectiveness analysis shows a high probability that the CHESS intervention is cost-effective at a willingness-to-pay threshold of £20,000 per QALY gained. This finding is robust in a sensitivity analysis using the Hernandez-Alava EQ-5D-5L conversion algorithm,61 but not when utilities were calculated from SF-12 data. This may reflect tentative evidence in the external literature that suggests that the EQ-5D generates larger utility gains associated with improvements in headache-related outcomes.82,83 Before starting this work we had concerns that the EQ-5D might not be the best measure of health utility to use in a headache population because its 1-day measurement window would not adequately capture substantial day-to-day, or within-day, fluctuations in health state affecting people with migraine. Data derived from the SF-12 might be more stable. Whatever the explanation for this difference, it does raise the possibility that our base-case analysis may be overestimating the cost-effectiveness of the CHESS intervention. Conversely, the much reduced ICER found when we used a societal perspective suggests that the CHESS intervention might represent better value for money than our base-case analysis if a different perspective is used.
It is not clear why the CHESS intervention appears to generate additional QALYs when it has no meaningful effect on our headache-specific outcomes. It is possible that the EQ-5D-5L, but not the SF-12, is measuring non-specific effects from attending the CHESS intervention not measured using headache-specific outcomes. Or it might be that there is an early benefit, evidenced by the 4-month HIT-6 findings, that is having proportionally larger impact in the area under the curve analysis. Although these health economic findings are important, it is not clear how they can be used to inform treatment choices in the absence of clear clinical benefit.
Not all trials of complex interventions have a process evaluation run independent from the main trial. This work has given us good insights into the experiences of people living with chronic headaches and of their experiences within the trial. Although this process evaluation was thorough, it did not provide us with any insights as to why the intervention was ineffective.
The dose delivered by the intervention was good, in that sessions were delivered as planned. Intervention fidelity was good, and consistent with that observed for other, similar, interventions.14
The premise of our intervention was that those living with migraine had the potential to improve their experience of headache through what they chose to do or not do. Our results add to the evidence that this is not the case. This may come as a relief to those living with migraine, as it reduces the burden on them to control a problem that our process evaluation indicates can feel uncontrollable. It allows migraine to be recognised as an intermittently and unpredictable disabling condition that requires a flexible response from society.
It is unclear why a multifaceted, theory-driven (best-practice), evidence-based intervention, specifically designed for those with chronic headache conditions, that was well delivered and well received failed to make a difference.
Patient and public involvement has helped shape the overall design and development of this programme of work. It has provided important input into the style and content of our intervention, and how we should measure and assess patient outcomes in chronic headache disorders. We have been able to reflect on our experiences of PPI in the research programme. Our thinking has, of course, developed over the lifetime of the CHESS research programme. For the future we think that we can make our processes more efficient and ensure that PPI is rewarding and supportive for all those involved.
Reflections on what was and what was not successful in the programme
Overall, the programme was delivered well, and we achieved our overarching aim of developing and testing a self-management support programme for people living with chronic headaches. We successfully undertook a series of systematic reviews to set the scene for the rest of the work. Our feasibility study was complicated as we were seeking to achieve multiple objectives within the same framework. This did make it difficult to explain to participating general practices and study participants. Recruitment rates for the main trial from general practice were smaller than originally thought. In our original application we thought we would need to work with 30–40 general practices. In the end we needed to work with 164 practices.
Limitations relating to the method or execution of the research
This was an ambitious programme of work that in phase 1 needed integration of interlinked work packages. The original timelines were, in some areas, too ambitious, needing some changes as to how we collected data, notably recruiting for some interviews through our charity partners rather than from feasibility study participants. In addition, work on validating the CHQLQ was not completed prior to starting the main trial, meaning that we set the HIT-6 as our primary outcome, although our view now is that this is not the most suitable outcome for studies of this nature. This has not, however, affected our final conclusion as the results from the two measures are consistent. The COVID-19 pandemic had a limited impact on the trial. Follow-up finished early in the first wave of the pandemic, meaning that we had some reduction in the data available for our secondary outcomes and we needed to make changes to how we collected data from GP surgeries.
Conclusions from the whole programme
Our data effectively exclude the possibility that this short intervention is effective for the treatment of people with chronic migraines, or chronic tension-type headache and episodic migraine. Nevertheless, the health burden of chronic headache disorders, principally chronic migraine, is debilitating. Further advances in this field must be driven by new theoretically and/or biologically informed intervention models.
Recommendations for future research
- New work to better understand the health impact of chronic headache disorders and to identify modifiable risk factors for a poor outcome.
- Development and testing of new non-pharmacological interventions for a tightly phenotyped group with chronic migraine.
- Work is needed to improve classification of headache disorders in primary care to allow better targeting of the available drug treatments of proven effectiveness, and reduce medication overuse.
Implications for practice and any lessons learnt
There is no need, at this time, to implement formal supportive self-management support programmes, of the type tested here, for people living with chronic migraine.