Overall achievements
The work plan changed from that submitted, partly due to changed circumstances such as the COVID-19 pandemic, and partly due to a deepening understanding of the central issues as the study progressed. We restructured WPs 1 and 2 to respond to the immediate needs of the programme at the time. Our new structure, in which systematic reviews were used to develop preliminary conceptual frameworks, which were subsequently extended through empirical research work, was a success. Findings from our systematic review on the impact of recovery narratives were particularly limited, due to the small number of included papers, and hence empirical work was particularly important to create thorough conceptual frameworks. Three new forms of harm were added through empirical work, to the one form of harm identified from systematic reviews. It is possible that research studies can under-state harms, and hence our thorough attention to these was a success, and enabled the design of appropriate harm minimisation strategies.
The capacity provided by a long and well-staffed study allowed several un-planned added-value substudies to be conducted, including secondary analysis of our qualitative data to develop novel findings in relation to institutional injustice, experience of non-SUs, and post-traumatic growth (which led to a new programme of work led by a NEON researcher), exploration of methodologies for assessing diversity in the NEON Collection, and a series of papers exploring the issues around curation of mental health recovery and lived experience narratives.
Our overall management approach, of structuring the NEON study as a series of self-contained substudies, has been a success as it has enabled the early publication of findings. In turn, this has enabled our partners to replicate or extend our published work by conducting and publishing their own studies, for example, in the development and evaluation of a recovery narrative intervention for alcohol misuse,18,71–73 and through the translation of INCRESE to Dutch.74
Patient and public involvement
Our work to include diverse public members in our research was broadly successful. We maintained a panel of 10 public advisers throughout most of the programme, with new members invited in as others chose to depart. LEAP had a meaningful influence over important programmatic decisions, including our decisions to not require anonymisation of narratives, the presentation of our content warnings, and the design of user-facing elements of the NEON Intervention. Given that the original NEON proposal only had one definitive trial for people with experience of psychosis, then we contemplated including only narratives describing psychosis experiences, but LEAP persuaded us that a more diverse collection was better, and we now believe that this was the correct choice. Our CSG developed substantial expertise in thinking about the choice and rationale to include or exclude particular narratives. It was a success to create this group, and to award it decision-making powers. In turn, CSG decided that there were certain narratives, such as those including graphic descriptions of self-harm which may create harm through behavioural emulation, that it did not have competence to decide on, and recommended that the clinically qualified NEON CI should make the decision, which was a success of the processes that we had designed.
Equality, diversity and inclusion
Equality, diversity and inclusion and the Narrative Experiences Online Collection
At several points during the programme, we assessed the diversity of the recovery narratives contained in the NEON Collection by analysing the distribution of INCRESE ratings across all narratives. We then sought to increase narrative diversity by targeted engagement with relevant community organisations, to encourage narrative donation from under-represented groups. This led to an unplanned theoretically informed paper investigating the methodological challenges of assessing diversity in narrative collections and using the NEON Collection as an exemplar.43 This will be of wider use in supporting the development of diverse and inclusive narrative collections.
Equality, diversity and inclusion and the Narrative Experiences Online trials
To maximise diversity of participants in the NEON trials, we used multiple approaches to recruitment, including:
- Paid online advertising on mental health websites and printed magazines.
- Engaging with 775 diverse mental health community groups to distribute promotional materials to diverse communities, specifically including traditionally under-represented groups.
- Engaging with 66 primary healthcare practices to distribute promotional materials to local communities.
- Distributing promotional messaging through paid (to enhance message reach) social media promotions on Facebook [Meta Platforms, Inc.(formerly Facebook) Menlo Park, CA, USA], Twitter [X Corp. (formerly Twitter) San Francisco, CA, USA] and Google (Google Inc., Mountain View, CA, USA).
- Television and radio media appearances by the central study team.
- Working with the CRNs of 11 secondary care mental health trusts.
For the NEON-O Trial, we had an unrepresentative proportion of female-gendered participants (79.3%). The proportion of participants who were White British was close to the proportion reported in the 2021 census for England (census: 81.7%; NEON-O Trial: 77.1%). We found evidence for differential effectiveness by gender in our CORE-10 finding, where women experienced a reduction in psychological distress, but men did not. We found evidence for differential effectiveness by ethnicity in our findings on the presence of meaning in life, where White British people benefited, but people who were not White British did not. The mechanisms creating these interaction effects should be explored through future research, to examine their impact on health service implementation. For the NEON Trial, the proportion of participants who were White British was close to the proportion reported in the 2021 census for England (census: 81.7%; NEON Trial: 75.9%). No interaction effects for ethnicity or gender were found.
Engagement with partners and stakeholders
Existing collections of recovery narratives were critically important partners to the success of our programme, as most of the narratives that were included in the NEON Collection, and hence used in our trials, were sourced from existing collections. We worked carefully to build relationships with existing collections, and to establish appropriate procedures for obtaining and documenting consent to include their narratives. In some cases, collection organisers obtained individual consent from narrators, and in others, collection organisers had pre-existing documentation of consent to share narratives. Engaging with collection organisers required a substantial effort from the study team. Payments to collection organisers enabled success in gaining consent for narratives, particularly where our partner was a smaller organisation, with more limited resources. For example, for some collections, we compensated collection organisers for the time required to contact narrators, inform them about the NEON study, and manage consent records for our use of their narrative. We recruited eleven NHS secondary care trusts to partner with our programme as research sites in our trials. Some trusts also acted as stakeholders, providing access to knowledge on how recovery narrative interventions might be deployed in the NHS, and insights into how clinicians viewed the NEON Intervention.
Institutional capacity building
Narratives describing lived experiences of mental health problems are widely used as a form of intervention in health and community settings,9 and are regularly analysed to understand personal experiences of healthcare problems.75 In our own institutions, interventional use of lived experience narratives was not a common endeavour, and research through narrative inquiry was relatively common. To succeed, we have needed to build capacity to work with lived experience narratives as a form of intervention. This has included developing protocols for managing narrative collections (which have subsequently been re-used in new NIHR-funded studies, e.g. LEND https://fundingawards.nihr.ac.uk/award/NIHR206255) and through developing research staff knowledge on ethical and legal challenges of working with lived experience narratives. Greater knowledge around narratives has in turn generated research success, including through analysis of healthcare narratives provided through patient feedback systems regularly used in England.76
Limitations
Our empirical work gave a thorough account of the long-term impact, as many participants in our NEON narrative interviews were found to be capable of reflecting on several years of their life, and thinking about this impact. It gave a thorough account of short-term impact. There were limitations to this work that could be addressed through future studies, in that our designs may not have catered well for people experiencing challenges with their memory, and may not have been able to capture medium-term detail on processes by which the impact of recovery narratives plays out. In our NEON-O Trial, there was no significant baseline-adjusted difference between intervention and control at 1-week and 12-week follow-up, but there was a significant baseline-adjusted difference at 52-week follow-up, suggesting that the study of how change occurs over a period of around a year could be important. Our empirical studies on the characteristics and impact of narratives, both of which included interviews, had a bias towards participants comfortable with expressing themselves verbally, and might be extended through alternative study mechanisms catering for participants with a preference for other (e.g. non-verbal) forms of communication. This use of interviewing is a consistent bias in health research.77
A limitation of our trials is that they did not capture outcome data over periods longer than 1 year. In the NEON Trial, use of mental health services increased, which was a driver of increased participant cost in intervention compared to control. This is consistent with treatment seeking behaviour, a known mechanism of action for online digital health interventions.78 Especially given current long waiting lists for mental health treatment, then it is possible that a longer follow-up period would find benefits for intervention arm participants, as treatment received once sought starts to help. A substantial challenge to the design of our trials was that mental health recovery narratives are now broadly available to the public. In the NEON Trial and the NEON-O Trial, recovery narratives were accessed by a substantial number of the control group, and the use of NEON Collection narratives predicted the use of narratives outside of the NEON Intervention. This use of recovery narratives by the control group might be conceptualised as a form of contamination, and it may mean that effect sizes are underestimated in both trials. The ICER for both trials may be overestimated if compared to a population-scale deployment, as the NEON Intervention delivery cost incorporates some fixed costs, whose per-participant value would reduce with a larger user base. Our sensitivity analysis using a pattern mixture model demonstrated that our significant finding on effectiveness in increasing QoL was sensitive to small departures from MAR, since it became insignificant if people in the intervention arm with missing data had a reduction of more than 1% in their MANSA score compared with individuals who had observed data.
We are uncertain about our use of payment vouchers for trial participants. In the NEON Trial, £80 was available to participants across four sets of questionnaires, representing a liability of £54,270, which represents a substantial commitment of study resources. In the NEON-O Trial, our amendment introduced payment vouchers for the primary end-point questionnaire only, with a liability of £19,880. The intention of this amendment was to increase completion rates for outcome data collection questionnaires. For the NEON-O Trial, completion rates at the primary end point were higher than at 12-week follow up (565 compared to 442 MANSA scores), which is consistent with voucher payments increasing completion rates. However, for the NEON Trial, 66 (17.8%) of intervention arm participants accessed no narratives, which is consistent with a process evaluation finding that some participants registered to the trial purely to obtain vouchers, thereby diluting any possible effect of the NEON Intervention. An alternative use of the committed study resources for voucher payments would have been additional recruitment effort, for example, through boosting the reach of social media posts, which were an effective route to recruitment.
Recommendations for research
- The NEON Intervention should be evaluated through a RCT with people experiencing psychosis and using mental health services.
- The NEON Intervention should be refined to suit the needs of carers and then evaluated through a RCT.
- The NEON-O Trial should be repeated with narrower mental health populations (e.g. mood disorders, eating disorders) to refine knowledge on effectiveness and cost-effectiveness. This may include refining the narrative collection used with these populations.
- If the NEON Intervention is implemented on a larger scale for people with non-psychosis mental health problems, then studies should be conducted to monitor benefits, continuously assess safety, and documentation implementation processes.
- Future studies should consider alternative forms for presenting recovery narratives, including through multilanguage or multiculture support, and addressing digital exclusion by providing access through widely available technologies, such as smartphones and text messaging.
- Longitudinal designs are needed to document the short-term, medium-term and long-term impacts of recovery narratives.
Implications for healthcare practice
Our NEON-O Trial provided a definitive finding that the NEON Intervention led to a very small QALY gain in a particular and non-representative population. Our economic analyses suggested that it may be cost-effective for people with mental health problems but no experience of psychosis, with cost-effectiveness evaluated from the perspective of the NHS in England, but results lacked statistical significance. These promising findings need to be further tested in the future research programme we have specified.
Our systematic review on the uses and misuses of recovery narratives has provided evidence that recovery narratives can be misused, including through co-option by mental health services.9 For example, we have learnt of examples of the misuse of recovery narratives within the NHS in England, such as mental health workers being asked to collect recovery narratives from SUs, and to edit these narratives to provide a more positive view on the role and actions of mental health services. Mental health service providers should create an expectation that recovery narratives are not misused within their services, perhaps through integrating our published good practice guidelines.47
Impact and learning
Digital health interventions can be relatively cheap to deliver, especially as user numbers grow (and hence the overall cost of providing access is amortised across a larger number of people). The greatest impact from our work may come from provision of the NEON Intervention as a national public health resource. This will require work to re-engineer the NEON Intervention to cater for much larger numbers of users. The strongest impact and learning would come from international replications of our NEON trials, perhaps with an examination of how to select narratives that were most appropriate for different national or cultural settings.