Parts of this chapter are reproduced or adapted from Caldwell et al.1 © 2019 The Author(s). Published by Elsevier Ltd. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. The text below includes minor additions and formatting changes to the original text.
Description of the problem
Common mental disorders are a key cause of morbidity in children and young people (CYP). Globally, depressive disorders are the third largest cause of adolescent disability-adjusted life-years (DALYs) lost, and anxiety disorders are the fifth cause of DALYs lost for adolescent girls.2 In the UK, the most common mental disorders among CYP are anxiety, depressive and conduct disorders. NHS Digital figures from 2017 suggest that 7.2% of 5- to 19-year-olds have an anxiety disorder, 2.1% a depressive disorder and 4.6% a conduct disorder.3 In this report, we will refer to anxiety, depressive and conduct disorders as ‘common mental disorders’ (CMDs), as these are the disorders with the highest prevalence among CYP in the UK. Although the prevalence of CMDs tends to increase with age, it is noted that rates of anxiety and depressive disorders have increased among CYP in the UK over the last 20 years, in contrast to the stability of diagnoses for conduct and hyperactivity disorders.3–5 However, as many CYP or their guardians do not seek help,6,7 these figures may represent an underestimate.8,9
Children and young people with a mental health disorder are more likely to engage in risky behaviours, such as smoking and substance use; are more likely to self-harm; and are more likely to be excluded from school.10–16 Although the causes are multifactorial, with genetic and environmental factors contributing to susceptibility, the distribution of CMDs is socially and economically patterned.17 For example, young people with a common mental disorder are nearly twice as likely as those without a disorder to be living with a lone parent, more than twice as likely to have unemployed parents and more likely to have parents with low incomes and fewer qualifications and living in social housing.11 Evidence from the UK Millennium Cohort18 suggests that children from low-income families are four times more likely to have a mental health problem than those from higher-income families. Longitudinal evidence suggests a linear relationship between the frequency of disorder episodes and the likelihood of adverse social outcomes. In a cohort of CYP aged 16–21 years from New Zealand,19 the odds of later welfare dependence were 1.34 [95% confidence interval (CI) 1.09 to 1.64] times higher among those reporting 1–4 episodes of depression than among those reporting no episodes of major depression. The odds among those reporting ≥ 10 episodes were 2.42 (95% CI 1.31 to 4.45) times higher than among those reporting none.
Although there is ongoing debate about the drivers of increased prevalence,20 there is robust evidence to suggest that lifetime trajectories of CMDs are established by mid-adolescence,21 with half of all disorders recognisable by age 14 years and three-quarters by age 25 years.22 The Royal College of Psychiatrists has stated that greater personal, social and economic benefits can be generated by intervening early in the life course than by intervening at any other time.22 However, Child and Adolescent Mental Health Services (CAMHS) worldwide are under-resourced.23 In the UK, the Local Government Association estimates that > 338,000 children were referred to CAMHS in 2017, but fewer than one-third had received treatment within the year.24 Even in the advent of optimal access and treatment, one economic modelling study has suggested that < 30% of the burden of CMDs could be alleviated by treatment alone.25
Description of the intervention
Against this background, there has been a growing focus on the primary prevention of CMDs among CYP. Primary prevention aims to prevent the onset of disease before clinically relevant symptoms are detectable and, therefore, targets a seemingly ‘healthy’ population. According to the National Academy of Medicine (NAM) (formerly known as the Institute of Medicine), primary prevention encompasses the prevention of disorder-specific symptoms, reduction of preclinical symptoms and prevention (or delay) of disorder onset.26 The NAM definition of primary prevention also refers to universal, selective and indicated prevention26,27 and is distinguished from mental health promotion (Figure 1). Universal prevention addresses whole populations regardless of their risk status or susceptibility to a CMD. Selective prevention targets subgroups with higher than average risk of developing a mental disorder; risk can be defined as biological, psychological or social factors. Indicated prevention focuses on individuals with detectable, but subclinical, symptoms of a CMD. Increasingly, the boundary between indicated prevention and ‘early intervention’ is being blurred by clinicians.28 The NAM framework views mental health promotion as a focus on encouraging mental health and the enhancement of well-being, rather than the prevention of illness.
FIGURE 1 The mental health intervention spectrum for mental disorders. Reproduced with permission from Reducing Risks for Mental Disorders: Frontiers for Preventive Intervention Research.26 Copyright © National Academy of Sciences. All rights reserved. (Figure not reproduced here; see the original article.)
In the UK, schools are at the forefront of the prevention agenda. For example, the green paper Transforming Children and Young People’s Mental Health Provision29 calls for mental health leads to be embedded in schools and a greater role for schools in cross-sectoral support teams. The 2019 green paper Advancing our Health: Prevention in the 2020s30 takes this further, with subsequent policy announcements giving schools statutory responsibility for children’s mental health and well-being. Across the UK, school-based education is compulsory between the ages of 5 and 16 years,31 with further statutory provision for 16- to 18-year-olds in England. In 2019, 8.82 million pupils were enrolled in England,32 698,000 in Scotland,33 234,550 in Wales34 and 330,000 in Northern Ireland.35
Multiple systematic reviews examining school-based preventative interventions for CMDs have been published in recent years, and taken together the results suggest a small but positive effect of psychological and educational interventions. For example, for the prevention of anxiety and depression, Werner-Seidler et al.36 evaluated both universal and targeted (selective and indicated) interventions in school settings and found a small beneficial effect on both depression (Hedges’ g 0.23, 95% CI 0.19 to 0.28) and anxiety (Hedges’ g 0.20, 95% CI 0.14 to 0.25). Johnstone et al.37 focused on universal interventions in school settings and observed a positive effect on symptoms of depression (Hedges’ g 0.17, 95% CI 0.06 to 0.28), but not on anxiety (Hedges’ g 0.09, 95% CI –0.07 to 0.26). Stockings et al.38 included multiple settings in their review, and included both universal and targeted populations. They concluded that universal (Cohen’s d –0.11, 95% CI –0.16 to –0.05) and targeted interventions (Cohen’s d –0.33, 95% CI –0.46 to –0.20) to prevent depression are effective in the short term. They observed that universal prevention had a positive effect on anxiety (Cohen’s d –0.16, 95% CI –0.27 to –0.06), but that indicated prevention did not (Cohen’s d –0.01, 95% CI –0.27 to 0.26). Rasing et al.39 focused on targeted interventions only, in any setting, and concluded that depression symptoms were reduced [standardised mean difference (SMD) –0.25, 95% CI –0.38 to –0.12], but not anxiety (SMD –0.19, 95% CI –0.36 to 0.03).
Much research into the prevention of conduct disorder has focused on indicated parenting programmes to prevent antisocial/disruptive behaviour in young children. Meta-analyses of indicated parenting programmes suggest that they have a positive effect. For example, Piquero et al.40 report a medium effect size for preventing antisocial behaviour (Hedges’ g 0.37; p < 0.001). Meta-analyses of school-based universal interventions have focused on reducing broader ‘externalising’ or general behaviour problems, rather than on the prevention of conduct disorder. For example, Lipsey and Wilson41 found that both universal school-based interventions (Hedges’ g 0.21; p < 0.05; Q76 212; p < 0.05) and indicated interventions (Hedges’ g 0.29; p < 0.05; Q108 300; p < 0.05) had a small beneficial effect in terms of preventing outcomes of disruptive behaviour.
Rationale for the current study
It can be argued that no two preventative interventions are exactly alike, as they are made up of combinations of components, each delivered with differing degrees of fidelity and intensity, to slightly different populations and settings. However, in a standard meta-analysis, intervention complexity and variation are overlooked when studies are combined to form a single comparator for analysis (e.g. ‘CMD intervention’ compared with control). This ‘lumping’, or conflating, of potentially disparate interventions can induce statistical heterogeneity. Estimates of statistical heterogeneity (variability across intervention effects) in meta-analyses of preventative CMD interventions can be substantial. Although heterogeneity may be inevitable in public health meta-analyses,42 it should nevertheless be minimised because of the consequences for policy recommendations and decision-making.43 For example, in a random-effects meta-analysis, the precision (certainty) with which the average intervention effect is estimated decreases as heterogeneity increases, that is CIs are wider.
The need to ‘lump’ interventions, and control conditions, can be avoided by using a network meta-analysis (NMA).44 A NMA combines direct and indirect estimates of intervention effect to allow the simultaneous comparison of multiple interventions in a single evidence synthesis. Crucially, a NMA retains the distinct identity of each intervention analysed.45 It also enables the ranking of interventions according to the probability that each is the best, or worst, for a given outcome. The effect of intervention components (individually or in combination) can be modelled in a meta-analysis using metaregression methods.46 Work since 201447–49 has highlighted the importance and feasibility of NMAs in public health, and how they can be used to explore and minimise heterogeneity in evidence syntheses. A component-level NMA is ideally suited to synthesising preventative CMD interventions, as the complexity of interventions can be incorporated, while providing the coherent and quantitative assessment of effectiveness necessary for decision-making.
Aims and objectives
The overall aim of this project was to identify the most effective and cost-effective intervention component(s), or combination of components, for the universal and targeted prevention of common mental health problems among CYP. For the purposes of this project, CMDs were defined as anxiety, depressive and conduct disorders. This focus was clarified in a protocol update (see the following section and Table 1 for details).
This aim was addressed by the following objectives:
- consult with CYP and their parents/guardians to inform the outcomes of interest for the systematic review
- conduct a systematic review of educational setting-based (1) universal and (2) targeted interventions for the primary prevention of anxiety, depression and conduct disorder that have been evaluated in randomised controlled trials (RCTs)
- develop a classification scheme, or taxonomy, of components used in preventative mental health interventions
- conduct intervention-level and component-level NMAs to identify effective interventions and components of interventions
- conduct an economic evaluation to determine the most cost-effective component, or combinations of components, of targeted and universal interventions by condition and setting.
Changes to protocol, clarifications and additional analyses
The protocol was updated in October 2018, to reflect decisions made at the searching and screening stages of the review. These are listed in Searching and screening, with full details and accompanying rationale reported in Table 1 Further changes and clarifications were made at the analysis stage and are listed in Analysis for transparency.
| Deviation or clarification | Proposal or original protocol | Date | Review stage and change | Rationale |
|---|---|---|---|---|
| Project began: October 2016 | ||||
| Deviation | Proposal and protocol | November 2016 | Searching: reduced number of databases searched | The proposal stated that 12 databases would be searched. In consultation with an information specialist, we derived a more efficient approach involving three stages:Databases – we followed the Cochrane MECIR guidance50,51 for searching, which states that CENTRAL, MEDLINE and EMBASE™ (Elsevier, Amsterdam, the Netherlands) should always be searched as a mandatory first step. In addition, the MECIR guidance states that it is highly desirable that specialist databases are searched. In the updated protocol, we ordered the databases in terms of how likely they were to yield relevant papers, based on databases used in existing reviews.52 As the condition being reviewed was CMDs we chose to search PsycInfo® (American Psychological Association, Washington, DC, USA) in the first instanceSystematic reviews – the second stage of our search strategy involved searching for existing systematic reviews. In addition to the reviews identified via the database searches, we also searched Epistemonikos, which is a database of systematic reviews. The reference lists from reviews were downloaded and screened for potentially relevant studies. These were added to our list of titles and abstracts to screen for inclusion in the reviewScoping – finally, we conducted an informal scoping search of ERIC to check if any additional records (over and above those identified by the above strategy) could be identified. The scoping search did not identify additional studies. We therefore determined that this was an appropriate place to stop the search. In response to peer-review comments on the draft NIHR report, the ERIC scoping search was formalised in June 2020. It was combined with a search of the BEI, and is reported in Appendix 1 |
| Deviation | Proposal and protocol | December 2016 | Screening: change to inclusion criteria | In the original proposal, we stated that the relevant age range would be 5–25 years. To increase relevance to school settings in the UK, the lower age limit was changed to age 4 years. Studies were included if the majority of children were aged ≥ 5 years, or if the mean age was approximately 5 years with a ‘small’ standard deviation. Studies in which the majority of children were < 4 years of age were excluded |
| Deviation | Proposal and protocol | December 2016 | Screening: change to inclusion criteria | In the original proposal, we stated that the relevant age range would be 5–25 years. The original upper age limit was selected to allow sufficient time for multiple follow-ups in tertiary settings, and was not intended to reflect age at baseline (entry to trial). This approach was difficult to operationalise during pilot data extraction, as studies had a wide age range at baseline, spanning the upper age limit (e.g. ages 18–28 years at baseline). Therefore, this was modified to include studies in which the majority of participants were aged ≤ 19 years at baseline |
| Clarification | Proposal and protocol | December 2016 | Screening: clarification of inclusion criteria | ‘Community’ was defined in the protocol inclusion criteria as ‘school affiliated’ and the examples ‘after-school and holiday clubs, church groups, youth clubs and student unions’ given as an illustration. During screening, ‘school affiliated’ was operationalised as ‘attached or linked to a specific school setting’. Studies that used schools as the source of recruitment but that were conducted ‘off-site’ at home or in other community settings were not eligible for inclusion. Multisetting studies that were primarily based in schools were included. This was to ensure that the school was not simply be the point of recruitment for an intervention that was then (entirely) carried out elsewhere |
| Project paused: April 2017 to February 2018 | ||||
| Clarification | Protocol | April 2018 | Data extraction: clarification to conditions included | The original proposal listed CMDs as obsessive–compulsive disorder, phobia, post-traumatic stress, panic disorder, anxiety, depression and conduct disorder. We stated in the proposal that we anticipated focusing on the most common: anxiety, depression and conduct disorder. However, the original protocol did not reflect this anticipated focus clearly enough, and a clarification was needed. After staff absence, to ensure efficiency and expedite the review, a modification was made to the protocol to ensure that the explicit focus was on anxiety, depression and conduct disorder. At the stage this decision was made, data extraction for depression and conduct disorder studies had not started, but was under way for anxiety |
| Clarification | Protocol | January 2019 | Analysis: clarification of analysis plans | Educational settings were divided into UK-specific primary, secondary and tertiary groupings for the purposes of analysis, and studies were grouped on the basis of the mean age (or range) and mapped to a primary, secondary or tertiary setting for analysis. This was not made explicit in the original protocol, which implied that the intervention should be delivered in one of these settings. This would not have been practical from an international perspective, owing to differences in educational systems (e.g. middle and junior-high schools). Studies were grouped on the basis of the mean age (or range) and mapped to a UK-equivalent primary, secondary or tertiary setting for analysis |
| Deviation | Protocol | January 2019 | Analysis: change to analysis plan | We planned to analyse ‘inequality’ as a main outcome. However, few studies reported inequality as a primary outcome; instead, we carried out post hoc subgroup analyses by SES, sex and ethnicity. These characteristics were selected post hoc, on the basis of the most commonly reported study characteristics |
| Deviation | Protocol | January 2019 | Analysis: change to analysis plan | In the protocol, we stated that we would conduct metaregression by intervention intensity, defined as total session time (number of sessions × duration in minutes). However, we determined that this would not be meaningful in a NMA with differing classes of intervention. It would have been possible to conduct the metaregression in a subgroup analysis of psychological therapies only |
| Deviation | Protocol | May 2019 | Analysis: change to outcome measure | In response to reviewer comments on Caldwell et al.,1 we added a post hoc composite ‘internalising’ outcome for inclusion in the NMA. We defined internalising outcomes as combined, or total, scores from depression and anxiety symptom scales. For example, the ‘internalising’ subscale of the SDQ or the total score from the DASS |
| Deviation | Protocol | June 2019 | Analysis: change to analysis plan | Parental reporting of child symptoms was a secondary outcome and, as such, it was not anticipated that we would conduct a NMA. However, based on external evidence of a discrepancy between CYP and parent reports, and that some included studies reported only a parent outcome, we conducted a post hoc analysis of parent-reported outcomes |
| Clarification | Protocol | October 2019 | Analysis: change to analysis plan | In the protocol, we stated that a cost-effectiveness analysis would be conducted if there was sufficient evidence to build and populate a model. If this were not the case, then a cost–consequence analysis would be conducted. We did not identify sufficient evidence to build and populate a model; therefore, we did not conduct a cost-effectiveness analysis. We did, however, conduct a cost–consequence analysis. This, therefore, does not constitute a change from protocol, but we report it here for transparency |
Searching and screening
- The number of databases searched was reduced from the original proposal. Instead, we followed Cochrane Methodological Expectations of Cochrane Intervention Reviews (MECIR) conduct guidelines50,51 on the selection of primary databases and applied approaches for optimising search strategies.52–54
- The protocol stated that the relevant age range for inclusion was 5–25 years. This was difficult to operationalise in practice, and changes were made to the age limits so that the report covers the age range 4–18 years.
- We clarified the intended intervention setting as ‘educational-setting based’. In the original proposal, we stated that the review would be conducted for ‘school and community based . . . prevention interventions’, and defined ‘community’ as ‘school affiliated’ settings. The clarification here pertains to the definition of ‘school affiliated’, which was operationalised as ‘formally attached or linked to a specific school setting’.
- A further clarification relates to the definition of CMDs. In this review, CMDs were defined in reference to their prevalence. The updated protocol clarified that the clinical conditions of interest were anxiety, depressive and conduct disorders, as these are the most common across the included age groups.
Analysis
- The educational setting for each study was categorised as UK-specific primary, secondary and tertiary groupings for the purposes of analysis only. This was not made explicit in the original protocol, which implied that the intervention should be delivered in one of these settings.
- We planned to analyse ‘inequality’ as a main outcome. However, owing to a lack of data, this was not possible; instead, we considered subgroup analyses by socioeconomic status (SES), sex and ethnicity. These characteristics were selected post hoc, based on participant characteristic data that had been extracted.
- We planned to conduct a metaregression of intervention intensity, in which intensity was defined as total session time (number of sessions × duration in minutes). However, we determined that this would not be meaningful in a NMA with differing classes of intervention, and so the analysis was not conducted.
- In response to reviewer comments on Caldwell et al.,1 we added a post hoc analysis for a composite internalising symptoms outcome (which combined depression and anxiety symptom scores).
- A post hoc NMA of parent-reported child symptoms was conducted.
- In the protocol, we stated that a cost-effectiveness analysis would be done if there was sufficient evidence to build and populate a model. As an alternative, we planned a cost–consequence analysis. We found that there was insufficient evidence for a cost-effectiveness analysis, and so a cost–consequence analysis is reported.