Main trial results and economic evaluation

Main trial results

Final sample of schools

We recruited one extra school than was needed to allow for school dropout. The final total was, therefore, 25 schools (12 intervention schools and 13 control schools). Not enough schools could be recruited at each geographical site in each FSM level, partly because of the distribution of schools by FSM status in each area. We therefore merged strata for recruitment and randomisation, where necessary, to maintain balance across the study overall. In England, the lower two strata were merged and in Wales, the top two strata were merged.

The final distribution of recruited schools by geographical area and FSM strata is shown in Table 5.

Participant response rates

All 25 schools remained in the trial until the end and participated in all three data collections. Overall, teacher and student response rates at each time point are shown in Table 6.

Participant flow through the trial is shown in the Consolidated Standards of Reporting Trials (CONSORT) flow diagram (Figure 2). As can be seen in Figure 2, teacher response rates were slightly lower in intervention schools at both follow-ups than in control schools. At each time point, a proportion of teachers and students were not eligible for inclusion and, therefore, were not given a questionnaire to complete (e.g. those who were on maternity leave, had left the school, had parental consent withdrawn or were on long-term absence, such that they would not be in school during the intervention period). Reasons for non-completion at each time point were absence from all data collection sessions and returning a blank questionnaire (which those not consenting to participate were asked to do).

FIGURE 2 The CONSORT flow diagram. Reproduced with permission from Kidger et al.60 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original figure. (Figure not reproduced here; see the original article.)

The total number of teachers included in the primary analysis was 1722. This was well above the mean of 50 teachers per school assumed within the power calculation. The total number of teachers in any secondary individual-level analysis ranged from 1533 to 1739. The total number of students included in the analysis with well-being as the outcome was 2700. The total number of students included in any individual-level analysis ranged from 2700 to 2703.

Baseline characteristics

Baseline teacher demographic data (Table 7) were similar between study arms, with the majority being white female participants. This reflects the wider teaching workforce. Figures for 2018 in England showed that 64.3% of full-time state-funded secondary school teachers were female and 82.6% were white.85 Figures for 2019 in Wales showed that 75.5% of all teachers were female and 90.5% were white.86 Participating teachers had much poorer well-being than the general working population, whose mean score is 51.4.62 Similarly, participant median scores on the PHQ-8 suggest a higher level of depressive symptoms than population prevalences reported in other studies.65,87 Around one-third of participants self-reported experiencing a previous mental health problem, and the vast majority had not received previous mental health training. Most participants reported good relationships between teachers and students and between staff at their school, and just under half had provided support to a colleague more than once a month in the previous academic year.

Baseline student characteristics (Table 8) were well balanced between arms, although the mean total difficulties score was slightly higher among control school students. Students were predominantly white, reflecting the population ethnicity in which the study was conducted. The mean well-being score was lower than the population mean WEMWBS score of 51.7, reported for 11- to 19-year-olds in England in 2017.1 This difference may reflect poorer well-being among our particular geographic sample, but it may also be partly due to the narrower age range in the current study. The baseline SDQ scores also indicate higher total psychological difficulties than UK norms for 11- to 15-year-olds.88 Again, this may indicate poorer mental health in this particular sample or it may reflect wider population changes since these norms were established in 2004. The majority of students at baseline reported good relationships between teachers and students in their school, and just under half had never gone to a teacher for help in the previous academic year.

Schools were also well balanced by most baseline characteristics across arms (Table 9). Intervention schools had a considerably higher mean number of teachers leaving during the year leading up the baseline measures for reasons other than retirement, and a higher percentage of schools in this group had equal to or above the national average in student attainment.

Primary analysis

Table 10 shows the teacher WEMWBS scores by arm at each follow-up time point and the results for the primary analysis. There was no evidence of a difference between arms in mean teacher mental well-being (WEMWBS score) over the course of follow-up (adjusted mean difference –0.90, 95% CI –2.07 to 0.27; p-value 0.130).

Secondary analyses

Primary outcome

The primary analysis was repeated with a time by allocation interaction term included in the analysis model. There was no evidence that treatment effect varied over time (p-value for interaction term 0.654).

Secondary outcomes

Table 11 shows the teacher outcomes at each follow-up by arm.

There was no evidence of a difference between intervention and control groups in teacher depression (PHQ-8) or presenteeism over the course of follow-up when these outcomes were treated as continuous, binary or categorical outcomes. There was weak evidence of a difference between groups in self-reported absenteeism over the course of follow-up, whereby the number of days of absence was 4% higher for teachers in the intervention group than for those in the control group (ratio of geometric means 1.04, 95% CI 1.00 to 1.09; p-value 0.042). Results for the fully adjusted models are shown in Tables 12–14. Appendix 5 shows the unadjusted and partially adjusted model results, which follow a similar pattern.

Table 15 shows the student outcomes at baseline and T2 by arm. As shown in Table 16, there was no evidence of a difference in student mental health outcomes (WEMWBS or SDQ) between intervention and control groups at T2 (see Appendix 6 for unadjusted and partially adjusted models for these outcomes, which show the same null effect).

There was also no evidence of a difference between intervention and control groups in any of the school-level outcomes (i.e. teacher absenteeism, teacher resignation/redundancy, teacher retirement, student attainment and student attendance) (Table 17), although there was a weak tendency towards greater teacher absence in the intervention schools. There was also a weak tendency towards more teachers retiring in the intervention schools, but also towards participants having taught for longer in intervention schools (p = 0.027 for years spent teaching at T2, with a higher number of years in intervention schools). Therefore, teachers in the intervention group may have been slightly older and nearer retirement age. In addition, data were missing from 17 schools for teacher retirement and left for other reasons, and data were missing from 12 schools for school-level teacher absence, which means that caution is required when interpreting these results. For student attainment, data were available for all 25 schools, but, in this analysis, some observations had to be omitted because of issues of perfect prediction and collinearity. Appendix 7 shows the results for school-level outcomes from the unadjusted models, which showed similar effects.

Subgroup and sensitivity analyses

Outliers

The primary analysis model was repeated after the exclusion of any data points that were identified as potential outliers with respect to the primary outcome (i.e. a WEMWBS score > 3 SD from the mean). Ten potential outliers were removed. The interpretation of the results of this analysis does not differ from that of the primary analysis model (i.e. there is no evidence of a difference in teacher WEMWBS score between the intervention and control groups over the course of follow-up) (Table 18).

Missing data

Multiple imputation was carried out under a MAR assumption; potential MNAR was explored by incorporating a scaling parameter that was additionally allowed to differ between arms (i.e. allowing the missing data mechanism to differ between arms). This was carried out by imputing under the MAR assumption and then multiplying imputed values of the outcome by a scaling parameter (e.g. a scaling parameter of 0.95 in the control arm and 0.90 in the intervention arm implies that missing WEMWBS scores in the control arm are assumed to be 5% lower than estimated under a MAR assumption, whereas scores in the intervention arm are 10% lower).

Tables 19 and 20 show the results of the analyses for teacher WEMWBS and PHQ-8.

Under a MAR assumption, there was no difference in conclusions drawn from the analysis of the imputed data compared with the primary analysis (i.e. no evidence of a difference in outcome between arms), which was as expected given the primary analysis model is robust to an assumption of MAR. Using scaling parameters to allow for potential MNAR, there was no evidence of a difference between arms when the same scaling parameter was used in each arm. There was evidence of a difference between arms if the scaling parameters differed between arms and the rescaling parameter was ‘worse’ in the intervention arm, whereby WEMWBS score was lower in the intervention group than in the control group over the course of follow-up.

Tables 21 and 22 show the results for the student WEMWBS and SDQ outcomes.

There was no difference in the conclusions drawn from any of the missing data sensitivity analyses compared with the analysis of the student data before imputation (i.e. there was no evidence of a difference between arms in student WEMWBS or SDQ).

Complier-average causal effect analysis

As shown in Table 23, there was no evidence of a treatment effect in those who attended the MHFA training, although there was a weak effect in the direction favouring participants in the control schools. There was also no evidence of a treatment effect in higher implementing schools, although there was a weak tendency towards higher well-being among lower implementing schools.

Exploratory analyses

Effect modification by demographic variables

There was no evidence of modification of the effect of the intervention on teacher WEMWBS or PHQ-8 by baseline well-being, depression, geographical area, school-level FSM or teacher sex (Table 24).

Similarly, there was no evidence of modification of the effect of the intervention on student SDQ or WEMWBS by baseline well-being, psychological difficulty, geographical area, FSM, or sex. There was, however, evidence of potential treatment effect heterogeneity on student WEMWBS (but not SDQ) by ethnicity, whereby there was a greater reduction in WEMWBS score in black students or mixed-ethnicity students who received the intervention than in white students. These results are shown in Table 25.

Intervention effect on self-report measures of school context

There was no evidence of a difference between intervention and control groups with respect to stress and satisfaction with work, school’s perceived attitude towards staff and student well-being, perceived quality of relationships between staff or between staff and students, or the amount of support given (Table 26).

Effect modification by self-report measures of school context

The variables stress at work, satisfaction at work, staff relationships, teacher–student relationships, perception of how much the school cares about teacher well-being and perception about how much the school cares about student well-being were included as treatment by baseline variable interaction terms to investigate potential treatment effect modification. The results are shown in Table 27. There was evidence of potential effect modification by perceived school attitude to student well-being. However, the interaction term was negative (–4.45, 95% CI –8.38 to –0.53). This suggests that the effect of the intervention was more negative among teachers who were more likely to agree that their school cares about students at baseline.

The same variables were included in the primary analysis model as time-varying covariates (Table 28). In certain cases, this did cause a reduction in the estimate of the regression coefficient of the intervention. However, as there was no evidence of an effect of the intervention in the primary analysis, it is difficult to draw any clear conclusions about potential mediation, particularly as the CIs still included zero.

Cost analysis

The average cost of the intervention was £9103 per intervention school (Table 29). The cost was slightly lower in English schools (£8263) than in Welsh schools (£9943), primarily because of the upfront cost of training the HSC in Wales. Staff salaries and costs for supply teachers (£5566) accounted for the majority (61%) of the total cost of the intervention. As reported above (see Tables 10, 12–14 and 16–17), there was no evidence that this additional cost was justified by improvement in teacher well-being, depressive symptom scores, presenteeism or absenteeism, or by any of the student-reported outcomes.