Process evaluation

Introduction

The overarching aim of the process evaluation for the cluster RCT of HeLP was to contextualise the trial effectiveness and cost-effectiveness results with respect to how the intervention may or may not have worked.

Aims

There were two main aims of the process evaluation:

1. to assess the uptake and fidelity of the intervention
2. to assess whether or not the intervention worked in the way it was expected to in terms of the intervention logic model.

Research questions

To address the aims of the process evaluation, we devised the following research questions.

Aim 1: to assess uptake and fidelity of the HeLP intervention:

1. How much of HeLP did children and families receive?
2. Was the programme delivered as designed?

Aim 2: to assess whether or not the intervention worked in the way it was expected to in terms of the intervention logic model (Figure 8):

1. Did schools, children and parents engage with HeLP?
2. How were the attempts to change behaviours experienced by the children?
3. Were there reported changes to children’s eating and physical activity behaviours?
4. Do statistical models combining potential cognitive and behavioural changes mediate observed between group differences in outcomes?

FIGURE 8 The logic model for HeLP. (Figure not reproduced here; see the original article.)

Logic model

The logic model for HeLP (see Figure 8) attempts to visually represent the theoretical underpinnings of the intervention, the content of the intervention, the process by which the intervention was assumed to work, the context in which the intervention was delivered and the outcomes it was hoped would be achieved. From the outset we wanted to develop a programme that had an impact on both the school and the family environment, as well as affecting individual processes related to health behaviours.122 The underpinning assumptions were to create supportive environments for healthy lifestyles choices and positive relationships at the level of the school, child and family. In line with the World Health Organization’s Health Promoting Schools framework, we aimed to develop activities that had an impact on the school environment and the curriculum, and that linked to families.21 The delivery methods used were highly interactive and encouraged identification with and ownership of the healthy lifestyle messages so that the children were motivated to take them home to their parents and effect change within the family.

The HeLP intervention used the IMB model123 as a guide to selecting cognitive and behavioural antecedents targeted by the intervention, antecedents that could lead to behaviour change. Intervention activities were then ordered to enable, support and sustain behaviour change in accordance with the health action process model.26

The process evaluation explored whether or not the intervention worked in the way it was expected to by carrying out a thematic analysis,124 using a Framework approach,125 of the qualitative data collected from focus groups with children and interviews with teachers and parents, as well as a mediation analysis126 using structural equation modelling (SEM)127 with the MLQ data from both the intervention and the control group.

General methods

The process evaluation of the HeLP study distinguishes between methods and analyses relating to (1) the intervention arm only and (2) the overall trial (including data from intervention and control arms). The analytical approach for (1) is hypothesis-raising only, synthesising data from field notes, observations, interviews, focus groups and questionnaires and for (2) it is hypothesis testing using a meditation analysis. For ease of reading, this chapter is presented in two sections. The first presents all process data collected from the intervention arm of the trial and the second presents the mediation analysis, which includes data from both the intervention and the control arm. Table 36 provides a summary of the data sources for each process evaluation research question and the process evaluation dimension it assessed.

Section 1: process data collected from the intervention arm of the trial

Methods

Sampling and recruitment

This involved collecting data from Year 5 teachers, children and parents of participating children.

All Year 5 teachers from each of the 16 intervention schools participated in a semistructured interview. Children were selected to participate in a focus group by the HeLP co-ordinator based on the child’s level of engagement (see Observations). The HeLP co-ordinator selected groups to ensure that there were equal numbers of boys and girls when possible. Each school had at least two focus groups of between six and eight children (one with children who were categorised as being engaged and one with children who were less engaged).

All parents of participating children were sent a questionnaire about the programme directly to their home address, along with a stamped addressed envelope. Parents were invited to participate in a semistructured interview, and they indicated their willingness to do so by completing a section at the end of the questionnaire.

Data collection

All data were anonymised, and any comments or observations relating to specific individuals or schools in a way that could allow them to be recognised were removed.

Parent questionnaire

A parent questionnaire (see Appendix 10) was distributed to all parents in December 2013/14 after the end of the intervention. The aim of the questions was to obtain data on any discussions that parents had had with their child about healthy lifestyles; their knowledge about the programme’s aims; their awareness of their child’s goals and how easy or difficult they felt it was for the child to achieve them; and details of any changes made to behaviours at a child level and/or a family level and whether or not these had been maintained.

Interviews

All Year 5 teachers in each intervention school participated in a 45- to 60-minute semistructured interview, on school premises, about their attitudes towards, and experiences of, participating in the HeLP intervention and any impact that they believed it to have had on them, the children and the parents, as well as on the school as a whole (see Appendix 11). All teacher interviews were carried out by the trial manager in June/July 2013/14 following phase 3 of the intervention and before schools broke up for the summer holidays.

Those parents indicating that they were happy to be interviewed were contacted via e-mail and/or telephone by the HeLP co-ordinator to arrange the interview. The interviews were conducted either by telephone or face to face at either the school or the parent’s home, whichever was more convenient for the parent. The HeLP co-ordinators carried out all parent interviews for their respective schools. The aim of the interviews was to obtain detailed data on parents’ views of HeLP, discussions that took place at home about healthy lifestyles, the barriers to and facilitators of the achievement of personalised goals, and any child and/or family changes made as a result of the programme and whether or not these were being maintained (see Appendix 12 for the parent interview schedule). These interviews were carried out between January and March 2014/15, during the spring term, after the intervention had finished.

Focus groups

A selection of children from each school participated in a focus group that consisted of between six and eight children. The HeLP co-ordinator for the school carried out two or three focus groups per school, depending on school size. One focus group was carried out with children who were considered to be engaged and another was carried out with children considered to be less engaged children. Details of how these categories were given are presented in the following section (see Observations). The aim of the focus groups was to obtain detailed data on what children had learnt; whether or not they talked about healthy lifestyles at home with their family and peer group and what they discussed; what they thought about the programme and how they felt it differed from the usual curriculum; how they felt about choosing and working with a character like them; what it was like to try to set and achieve their goals; what strategies they used to help achieve their goals; and whether or not there had been any changes at a family level (see Appendix 13 for the focus group schedule).

Focus groups took place during lesson time when the children were in Year 6 (between February and April in 2014 for cohort 1 and in 2015 for cohort 2). They were led by the HeLP co-ordinator for that school and facilitated by an additional HeLP co-ordinator, who took notes and supported the management of the group. To help children remember the details of the programme, visual cues were provided along with a short summary of the activities they participated in for each phase.

Observations

Observations of intervention components were carried out to obtain data on fidelity to form (i.e. were all of the HeLP components delivered in the correct order?) and fidelity to function (i.e. were key components delivered in the ‘spirit’ of HeLP?). To assess fidelity to form, a yes/no checklist was completed (by the HeLP co-ordinator) for all HeLP components to indicate whether or not a component had been delivered (see Appendix 14 for an example checklist).

‘Spirit’ was defined as enthusiastic delivery, open body language, responsiveness to child/school needs, and clear and friendly communication. The key components observed to assess fidelity to function were the parent assembly (phase 1), the healthy lifestyles week (phase 2), the parent assembly (phase 3) and the class-delivered assembly (phase 4).

To assess fidelity to function, a score between 1 and 10 was given for (1) delivery, (2) child responsiveness, (3) parent responsiveness and (4) teacher responsiveness for each of the four key components observed. At the beginning of data collection, the trial manager and the principal investigator independently scored fidelity to function for the parent assembly (phase 1) across three schools. No discrepancy in scoring was observed. Thereafter the majority of observations were carried out by the trial manager. The HeLP co-ordinator assessed the majority of the healthy lifestyle week components (phase 2) after they had carried out initial assessments alongside the trial manager. Once again, no discrepancies in scoring were observed (see Appendix 15 for an example checklist).

Each HeLP co-ordinator also collected informal observational data, in the form of field notes (see below), for child and school engagement. Assessment of a child’s engagement was based on the interaction of the HeLP co-ordinator with the child during the one-to-one goal-setting interview and from more general observations during the course of the intervention. The HeLP co-ordinator gave each child an engagement score between 0 and 3. The criteria for scoring were:

• 0 = uninterested/unaware goals needed to be set
• 1 = reluctant/needs a lot of prompting
• 2 = enthusiastic and happy to chat about goals and how they will achieve them
• 3 = very enthusiastic; has discussed goals at home and has clear strategies for achieving them.

School-level engagement was assessed using three scores based on the HeLP co-ordinator interaction with and observations of the head teacher, the Year 5 teacher(s) and the school support staff.

A score between 0 and 3 was given to each staff member:

• 0 = unengaged/unco-operative
• 1 = supportive
• 2 = enthusiastic and supportive
• 3 = very enthusiastic and used HeLP in other aspects of teaching/school activities.

Field notes

Each HeLP co-ordinator kept recorded notes in a diary of their informal interactions with and observations of staff and children during the intervention, which fed into their assessment of staff and child engagement. In addition, the HeLP co-ordinators recorded any unintended consequences of the programme.

Registers of attendance

The HeLP co-ordinator kept registers of attendance for all intervention components and for parental attendance at parental engagement events (i.e. parents invited to the school to observe activities). The Year 5 teachers were asked to keep a register for each personal, social and health education lesson that they delivered during the healthy lifestyles week, which was given to the HeLP co-ordinator.

Parental signature

In phase 3 of the intervention, children were asked to set goals at home with their parents on a goal-setting pro forma before they had their one-to-one discussion with the HeLP co-ordinator. Parents were asked to sign the form once discussions had taken place. All goal-setting pro formas were collected by the HeLP co-ordinator following the one-to-one discussion, and a copy of the goals was sent back to each parent/carer.

Analysis

All qualitative and quantitative process data were analysed blind to trial outcome and, initially, analysed separately. The different data sources were then combined to address each research question. Details of the analyses and subsequent synthesis are described in the sections below.

Data from registers, parent questionnaire and goal-setting sheets

Child and parental attendance at events, parental signature on the goal-setting sheet and both quantitative and more open qualitative responses from the parent questionnaire were entered into a Microsoft Access^® (2014 version, Microsoft Corporation, Redmond, WA, USA) database. The data were then exported into Microsoft Excel^® (2014 version, Microsoft Corporation, Redmond, WA, USA) and imported into NVivo (version 11, QSR International, Warrington, UK) or Stata.

The parental engagement score was measured using two sources of data: attendance at one or more parent events and/or signature on the goal-setting sheet. A score between 0 and 2 was given to each parent:

• 0 = did not attend any activity/did not sign the goal-setting sheet
• 1 = attended one or more events or signed the goal-setting sheet (but not both)
• 2 = attended one or more events and signed the sheet.

Parents were then dichotomised into two groups (≥ 1 = engaged parent and < 1 = less engaged parent).

Qualitative data (interviews and focus groups)

Interviews and focus groups were digitally recorded and verbatim transcripts were prepared from the sound files. The transcripts were checked for accuracy against the sound files and corrections were made if required. Any comments that could identify people or schools were anonymised before the transcripts were imported into NVivo. A deductive approach was used to code the data. Four transcripts from the cohort 1 focus groups were selected (two engaged and two less engaged) from each HeLP co-ordinator’s group of schools; these were read by the cohort 1 HeLP co-ordinators and the trial manager, and an initial code framework was agreed. Minor edits were made to the topic guide for the cohort 2 focus groups. A similar process of coding was used with the cohort 2 focus groups, and the initial codes were discussed, refined or amended and a new coding framework produced by HeLP co-ordinators and trial manager. The HeLP co-ordinators and one independent researcher then used this coding framework to code the remaining focus groups. The parent and teacher interviews were coded using a similar approach, with the coding framework being edited at each stage. The trial manager coded 20% of the transcripts, with the principal investigator providing verification for half of the 20% checked by the trial manager. The codes were then categorised (second cycle coding) to identify emerging themes and subthemes. Data from all sources (parent and teacher interviews and focus groups) were collated for each theme/subtheme and transferred into tables. The resulting tables were then analysed for agreement, partial agreement, silence or dissonance from the different data sources.

Observational data (checklists and field notes)

Delivery to function

The four scores (enthusiastic delivery, engaged/participating children, engaged parents and engaged school staff) per observation in each school were averaged (mean score) to create a single score out of 10 for each of the four components observed (the parent assembly in phase 1, the healthy lifestyles week in phase 2, the parent assembly in phase 3 and the class-delivered assembly in phase 4). These four scores were then averaged again to produce a single delivery mean score per school. A score of ≥ 8 was prespecified to indicate that the intervention had been delivered as designed (i.e. in the ‘spirit’ of HeLP).

Delivery to form

If all components for each phase were delivered (represented as a tick on the checklist), then it was recorded that 100% of HeLP components had been delivered in that school. If a minor activity (e.g. the practise of a chant or a missed scene from a drama workshop) was missing from a particular component, 2% was deducted for this from the overall total (see Table 38).

Child-level engagement

Individual child scores were dichotomised to create two groups (≤ 1 = less engaged children and > 1 = engaged children).

School-level engagement

Individual scores for the head teacher, the Year 5 teacher and the support staff were aggregated to give a score out of 9 for each school. Schools were then dichotomised into two groups (0–3 = less engaged school and 4–9 = engaged school).

The handwritten field notes were typed up by each HeLP co-ordinator and entered into Microsoft Excel and then imported into NVivo once any comments that could identify people or schools had been anonymised.

Results

The results from this section of the process evaluation are presented under each aim and their associated research questions.

Aim 1: to assess the uptake and fidelity of the HeLP intervention

• Research question 1: how much of HeLP did children and families receive?

Across cohorts 1 and 2, 676 children were randomised to receive the intervention. Table 37 shows the percentage of children participating in each phase of HeLP and the percentage of children receiving the four drama sessions in phase 2 and the one-to-one goal-setting in phase 3 (considered to be the key components of the intervention essential for behaviour change to occur) delivered in the manner in which HeLP had been designed.

Across the programme there were six invitations for parents and carers to come into the school and take part in the programme. These included the parent assembly and observation of the two activity workshops in phase 1, observation of work in progress in the final two drama sessions of the healthy lifestyles week in phase 2 and the forum theatre assembly in phase 3 (see Tables 2 and 3).

Just over half of children (52%, 353/676) had family attending at least one parent event.

• Research question 2: was the programme delivered as designed?

Table 38 shows that we can be confident that all intervention schools received a complete or near-complete programme that was delivered as designed (i.e. all schools scored > 8 for fidelity to function).

Aim 2: to assess whether or not the intervention worked in the way it was expected to in terms of the intervention logic model (see Figure 8)

All Year 5 teachers from the 16 schools that received the intervention were interviewed (n = 28) and all parents of participating children in the intervention schools were sent a questionnaire in the post following the intervention. Just over one-quarter of parents (26%, 176/676) returned the questionnaire, of which 80 (45%) indicated that they were happy to be interviewed. As there is some evidence that health promotion programmes can (unintentionally) widen health inequalities, we wanted to see whether or not engagement with HeLP and the trial process was the same across all socioencomic groups. Analysis by individual level IMD rank (i.e. based on the child’s postcode) showed that there was an equal spread across the socioeconomic spectrum for questionnaire returns. Fifty-two parents (of which two were fathers) were interviewed, with slightly more parents from the higher two than the lower two IMD quartiles participating (60% and 40%). Forty-five parents (87%) interviewed were categorised as engaged, 81% of whom also had an engaged child. Thirteen per cent of parents interviewed were less engaged, of whom 2% (one parent) had a less engaged child.

A total of 35 focus groups (18 with engaged children and 17 with less engaged children) were conducted across both cohorts.

• Research question 3: did schools, children and parents engage with HeLP?

Child, parent and school engagement scores, as well as the qualitative data from the focus groups and interviews relating to enjoyment and engagement of the programme, are presented here. Evidence of possible mechanisms leading to engagement/enjoyment (e.g. knowledge and understanding, discussions at home about healthy lifestyles, delivery methods and identification with the characters) is also presented here. Each quotation presented is referenced with the source (school number, P = parent, T = teacher, LEC = less engaged child, EC = engaged child). We also present relevant data from the parent questionnaire (see Appendix 10).

Child engagement

Ninety-six per cent (652/676) of children set goals with the HeLP co-ordinator in phase 3 and, of these children, 63% (411/652) had parental support, as indicated by a parent’s signature on the goal-setting sheet and/or written comments regarding how the parent would support the child in achieving their goals.

Twenty-four children had missing engagement scores (13 children had moved out of the area, eight children had changed schools before the one-to-one goal-setting discussion and three children were absent on multiple visits by the HeLP co-ordinator) and had not set goals.

Based on the child engagement scoring system, 92% (602/652) of children were deemed to be engaged with HeLP. Similar percentages of boys and girls were considered engaged (91% and 94%, respectively); however, those children in schools that had more than one Year 5 class had a greater percentage of engaged children than those with only one Year 5 class (97% and 82%; respectively). Table 39 shows that HeLP was able to engage children across the socioeconomic spectrum, although there were slightly more children from the most deprived quartiles in the less engaged category.

There was very clear evidence from all sources (teachers, parents and children) in the interviews and focus groups that children really enjoyed and engaged with all aspects of the the programme across all schools. All children spoke postively about HeLP:

I thought it was much more different than I thought it would be, because I didn’t really exactly imagine it as a boring old literacy lesson or anything but I didn’t really think it was going to be that joyful and exciting. It was much more than I expected.

Amazing, fun, healthy, extraordinary and the best!

It was brilliant it was such good fun; the children reacted to it really positively. In fact I have not heard them say anything negative about it at all. They’ve never said ‘oh no we’re doing this again’ they’ve always been really open and engaged . . .

She was coming home and really enthusiastic and sort of you know explaining to us about what she’d made in school and wanted us . . . wanting us to try it, to do it and share it with her, so you know I think that certainly helped a lot you know and . . . she wanted us to be a part of it.

The healthy lifestyles week seemed to be the most enjoyable aspect, with the majority of children reporting that they liked the interactive and dynamic nature of the sessions and working with the actors and characters, with whom they identified:

We had lots and lots of fun and everyone was getting excited for the next day and the next and the next, and for that week of activities, I was really excited about actually going to school, usually I’m just ‘oh I want to stay in bed’ but that week I was really excited about going to school.

The reason I liked the Healthy Lifestyles Week was because you were actually seeing what, sort of like a made-up version of four different people who have trouble and the ways you can improve it by just following them.

Um, my favourite part . . . was probably in the drama was like when we were, because it was so totally different to what we normally do. In school we obviously do literacy and maths when we have to write stuff down but you could really express your emotions through drama, I really liked that.

Awesome, it was really fun though because we got to act and then they told us what was in stuff.

However, a minority of children were less keen on the acting aspect:

I didn’t like doing the acting but it was OK when we were doing the non-acting.

Well they were really good but when the characters asked us to come up do a bit of acting and stuff it was a bit embarrassing, because I put my hand up and I didn’t know what I was in for.

The teachers reported that the delivery of the healthy lifestyles week and the activity workshops engaged the children and, in some cases, even the ones who were usually shy or disruptive:

I thought that the cleverness of the drama was that they did it from a child’s perspective. They acted out as children, but they were like the cool teenage children which children can relate to. Interestingly enough, there are several children in my class that could switch off so easily but they didn’t, because they immediately were drawn in to this character, this role play character. I thought the idea of having the Duncan and the Active Amy and all the rest of it, initially when you heard it you thought oh this isn’t going to work, and then you saw them go straight in to role, the acting was fantastic . . .

Yes, yeah there was one boy who would, well no two of them in fact that would never ever ever get up and do any drama or anything and they were up taking part in everything. I was so touched with emotion I had to run out and tell mum at the end of the day. So I think that it bought him out ‘cause he, he was one of the ones that did it in the hall as well.

And the acting just lifted it 100 times more and to them it was so important and to them . . . we did the lesson in the morning but they couldn’t wait until the afternoon. They were so linked to those characters and it was such a clever thing.

In the parent questionnaire, almost two-thirds of parents reported that their child had talked a lot about the programme at home (see Appendix 10). Parents who were interviewed reported that their child was enthused and motivated to make changes, and that discussions had taken place at home about what they had been doing in school:

She was always coming back and telling me what had been discussed. One of the main things, she was talking about exercise and that was one of the topics at the time and she was coming back and she said that part of her plan was to spend less time looking at the TV or on the computer and spend more time playing, and she went straight out on the trampoline with her little brother and since then – it’s like, she’s out now.

Yeah, he came home with oh saying these are the things that we should be looking for on the shopping list [Interviewer: OK] he was saying about how, you know, what was good to eat and what’s not good to eat and he was particularly talks about it when we all sit round for tea, ‘cause when we – I make sure that at the end of the day, we are all sat together eating the evening meal together [Interviewer: Yeah] so, I think that’s really important cos I think you can all talk and just . . .

Yeah they’ll stay with him for life ‘cause he talks about it a lot [Interviewer: Excellent] and you know I think that they have continued to talk about it at school and he’s taken it on board.

Children’s knowledge and understanding of the key messages were considered to provide further evidence that children had engaged with the programme; there was also evidence from the focus groups of their understanding of the messages around marketing, moderation and food labelling:

What helped me a massive amount was looking at the ingredients and looking at what’s inside stuff, like if it says like fruit on it, it may not actually be made out of fruit. Like fruit winders and stuff and some stuff and my mum said like if they make the big front of the packet really appetising and want to make you feel like you want to buy them but then the back is like all small and you can hardly read it so they are trying to trick you to get the really unhealthy stuff but make it look really appetising.

I think my favourite part about it was doing the food machines. I especially liked how they used acting to show how the foods were made and what process they go through.

Yeah I just think it’s really raised their awareness, yes the 80 : 20 so yes and he did talk about that with his brother and sister and you know that was the bit that really raise his awareness. You know impacted us . . .

Parent engagement

Based on the parental engagement scoring system, just over three-quarters (77%, 520/676) of parents were deemed to have engaged with HeLP. Table 40 shows a similar pattern of parental engagement and IMD rank to that observed for child engagement, with more parents from the lowest IMD ranking considered to be ‘less engaged’ than the highest IMD ranking, although similar proportions were engaged with the programme.

Of the 92% of ‘engaged’ children, the majority had parents who were also considered to be engaged. Of the 130 parents who were considered less engaged, 20 also had less engaged children, while the other 30 less engaged children had engaged parents.

The majority of parents interviewed spoke positively about the programme, reporting that it was a worthwhile project and supported the messages it was trying to promote at home. However, we do acknowledge that 87% of the parents interviewed were categorised as engaged, and thus were probably less likely to be critical in their interviews. The majority of responses from the parent questionnaire were very favourable towards HeLP, with 99% of parents indicating on the parent questionnaire that they were aware of their child’s goals:

. . . it gives me more reinforcement so when I feel like I’m nagging her about you know you really need to eat a wider range of food and la la la, then she’ll kind of go well yeah ‘cause you’re supposed to have blah blah. And it’s good ‘cause she’s learnt it from somewhere else. And if she’s learnt it at school it must be true! [Laughing] Whereas if you learn it at home you can take it or leave it, so it is good reinforcement.

One of the good things I liked about the programme was that you weren’t giving out lists of things that were quite expensive or difficult for people to buy or achieve um and I’ve seen that before and I find that frustrating and people can’t always afford to get you know certain fruits and vegetables and food products. It’s not always easy for them to do that and a lot of the food suggestions that you made particularly in terms of ‘replace this with this’ were simple and achievable . . .

Similar to what was seen in the children, engagement was also evidenced by parents’ knowledge and understanding of the key messages:

Yes we’d look at the cereal boxes and say ‘right better not have that cereal, that’s got too much percentage fat’ I can’t remember what [Interviewer: Sugar I think is it, yeah] sugar yeah um ‘we’ll have that one instead’. Yeah so it did make everybody think ‘ooh!’. Check on stuff that you think some things you think are healthy are actually, got a lot more sugar content than you expected.

No I think it’s been a really good programme actually and just to raise awareness of you know what you need to do to be healthy and you don’t have to stop all of those things because ultimately people think ‘oh you can’t have this and you can’t have that’ and they need to know that there’s a balance actually and that you can have it you just need to be a little bit aware of what you’re eating and yeah so I think’s it’s been brilliant and I wouldn’t change any of it . . .

However, 18 parents reported some negative feelings towards certain aspects of the programme in their questionnaire responses. One parent, who felt that their child was a fussy eater, had a smaller range of foods to choose from as he wanted to cut down on less healthy options. A couple of parents expressed concern about the cost of fruit and vegetables and a few parents reported that their ‘already healthy’ children took the programme messages to the extreme. One parent felt that it was the responsibility of the parents to educate their children about physical activity and health.

A small number of data from teachers also suggested that a few parents were less engaged with the programme. Two teachers from two different schools reported disappointing parental attendence, and in one school (school 7) a teacher informally reported to the HeLP co-ordinator that a parent had talked to the teaching assistant about not wanting to be lectured at about how to look after her child. Another teacher from a different school (school 9) reported that she had heard from a minority of children that their parents had thrown the project leaflets in the bin.

School engagement

School engagement scores ranged from 9 (maximum score) to 2. Out of the 16 schools, only three were categorised as ‘less engaged’ according to the school engagement scoring system (schools 1, 3 and 14). Characteristics of the less engaged schools included lack of senior leadership within the school generally, absent teacher during of the drama workshops and absent Year 5 teacher owing to illness during phases of the programme.

There was nothing to suggest that variation in school engagement was linked to differences in school context, or school characteristics and policies as assessed by the School Characteristics and Policies Questionnaire (see Appendix 16). Table 41 provides a summary overview of each school by cohort in relation to size, deprivation, location and engagement.

Teacher engagement

Regardless of the HeLP co-ordinator’s perception of teacher engagement (see Table 41), all Year 5 teachers spoke positively about the programme in the interviews and there was strong evidence that teachers were engaged with it. Many believed that the programme’s compatibility with the national curriculum made it feasible to deliver and helped to reinforce healthy lifestyle messages that the curriculum already promoted. All teachers reported that the HeLP co-ordinator assigned to their school provided excellent ongoing support, which really helped in the delivery of the programme:

She’s [HeLP co-ordinator] ever so good, she’s obviously worked with teachers quite a lot I think, she knows how it is. I don’t have to say ‘oh because we’re doing this’ she says ‘oh yeah yeah I realise so you know she’s really really good at fitting the scheme, we’re trying to fit in with you’ and she helped fit in with us so you know it worked really well. That flexibility is really important and you know she’s brilliant and she knows her stuff and she knows . . . I’ve been really impressed she’s been brilliant, really helpful and ever so amenable. We’ve had children with possible problems with their eating and you know she was brilliant talking to the parents . . . nothing was too much trouble it was like ‘oh yeah I’ll sit down and chat with them’ and she was brilliant when she was sort of talking to the parents and I think it is really important that you have that point of contact. For me it was so easy that I could you know send an e-mail or [HeLP co-ordinator] would e-mail me and I could e-mail back and you know that’s really really useful and very good and helps the thing run really well.

Many teachers commented that the way in which the lessons were delivered and the personnel used for delivery really helped the children think more deeply about their health and how they could make small, achievable changes to their eating and actvity behaviours:

I think they really engaged with those lessons. You know I was thinking about some of the messages that were coming across from the actors, when we had the actors come in for that week you know we tell them those messages all the time and they don’t sink in as much as they did then. And I think that coming from a younger perspective, the street dance obviously and [name of sports group] you know they nailed it really. Yeah they loved that and yeah I think you know its one thing having, I’ve got great relationships with the kids we all have but it’s one thing having Mrs [XXXX] tell them this and it’s quite another thing to experience it from younger people, or people they perceive to be younger. The actors were teenagers and they loved them. So yeah I think it had a deeper impact because it came from a different perspective. Than their parents or their teachers really.

• Research question 4: how were the attempts to change behaviours experienced by the children?

Whereas phase 1 and phase 2 of the HeLP intervention focused on introducing the messages and engaging children and parents with how changes could be made using the creative delivery methods discussed in Chapter 2 (see Study design, Intervention), phase 3 focused on the children setting personalised goals around the key messages.

Ninety-six per cent of children set goals with the HeLP co-ordinator in phase 3 and, of these, 63% had parental support, which was indicated by a parent’s signature on the goal-setting sheet and/or written comments regarding how the parent would support the child in achieving their goals. Children were encouraged to set three goals based on snacking, fizzy drink consumption and physical activity, depending on the area in which they needed to make changes. Table 42 shows the number and percentage of goals set for each target behaviour by gender.

As Table 42 shows, the majority of goals set by both genders were focused on increasing their physical activity and snacking. ‘Other’ goals included helping others to be healthy, going to bed earlier, studying, doing homework/household chores, trying new foods, eating more and drinking more water. No marked differences were observed in the type of goal set by gender or engagement.

Children reported mixed feelings about setting goals, with some finding it easy and others finding it more challenging for various reasons:

I felt setting the goals, um one of my goals is really easy to set but the other two were really hard. ‘Cause one of them was do, ask my mum to get healthy stuff and we don’t have that much money so we couldn’t get any . . .

I think because my mum understood it, like [Interviewer: Go on] I don’t really know, umm, my mum kind of knew about it, so if I went to speak to her then it was quite easy because she knew what I was talking about . . .

I found it quite easy because my mum was really supportive.

I didn’t have much, well I did talk to my mum but I didn’t have much time to talk to my mum because my brother and sister needed mum too. My brother needed help with his homework, my sister needed help with her homework, my brother needed to read to my mum and my sister needed to read to my mum, and my mum needed to sleep with my sister before going to bed and reading her a story, so it was a bit busy and I did have a quick little talk to her, but I didn’t have much of a talk so it was a bit difficult and we only did about half of it.

I found it quite hard, I thought it was a bit too open, like the sheets we took home, it was really open so it wasn’t really, ‘cause people could easily just put tick, tick, cross, tick, even if they weren’t actually trying. And because we especially did it over the weekend and the holidays, I found that extremely hard because in the holidays and that, I kind of let myself go and like, have a lot of chocolate, that sort of thing. So if we had done it during the week I think I would have found it a lot easier, so maybe next time.

I thought it was hard ‘cause my mum worked late, so did my dad, so I didn’t have anyone to talk to about it.

In the parent questionnaire, 99% (174/176) of parents reported being aware of their child’s goals, 59% (103/176) reported that they thought that it was ‘quite easy’ for their child to make changes based on their goals, and 20% (35/176) reported that they thought that it was ‘quite difficult’. During interviews, parents reported how their children engaged with and took responsibility for their goals:

Especially the school dinners because she does have a school dinner every day and she would always choose the chocolate cake or the pudding or something and then she, I didn’t say it to her, she made the choice to go for the fruit salad or yoghurt or you know the healthy thing and every day she’d come back, I mean we have always said what have you had for lunch and she’d say whatever, and then she would say but I had fruit salad and she was very proud of herself when she managed to choose the healthy option.

Yeah so she really owned her goals as well. She was really bought in from the start. She was very responsible about it which was good because it gave her something to be responsible for and it was only kind of small changes so um . . . she’s not allowed coke at home but at her dad’s she was allowed coke. So she made the decision to go for fruit juice with fizzy water instead and that’s carried on as well so um and she’s . . . she was the one who kind of raised it there and she was making those choices herself it wasn’t kind of like ‘oh no I’ve got to do this or you must be doing this because it’s on your goal’ it was about ‘yeah I need to do this because that’s what I’ve said I’m going to do’.

In the children’s focus groups, both engaged and less engaged children reported mixed levels of difficulty in achieving their goals; some children reported a number of barriers that made it difficult to make changes and others found it easier to achieve their goals:

I found it difficult because I found out that actually I was quite busy, so I didn’t really get time to pick when I was going to do it and choose what I ate. I kind of picked anything ‘cause I didn’t have the time.

Well I didn’t find it difficult but it was hard to change it, because it was like a routine and trying to get them in, get away from the other stuff and trying to get new stuff.

Sometimes I found it hard, but sometimes it was quite easy because when I got more in to the programme I found it easier because I started doing it more, in the beginning I found it really hard to make any changes.

I found two of them I think easier, but there was one which was really difficult, it was one which I could only have I think it was three sweets a week, and it got really difficult because it was the summer holidays and that’s the time when you normally get the most sweets so it was quite difficult to keep that one. I think I kept it half the time and half the time I didn’t.

As the HeLP logic model (see Figure 8) illustrates, it was assumed that changes in children’s eating and activity behaviours would be facilitated by a number of child and parent behaviours. Specific to the goal-setting phase, it was hoped that the children would use a number of strategies they had learnt during phase 2 to help them achieve their goals (e.g. putting their goals up on the fridge, or putting a note on their video game console to encourage them to limit their time). It was also hoped that parents would support them by increasing access to and availability of healthy snacks and physical activity, and enforcing rules about screen time.

When the children discussed their experiences of making changes in the focus groups, some commented that their parents had supported them in this way:

My parents helped a lot with my goals because my mum made a rota which is so I only get an hour or half an hour on screen time every day.

Yeah I told my dad to stop giving me loads of sweets on Fridays, and then he did do that for me and now I don’t eat as much sweets.

A few teachers also reported that parents were supporting their children:

I spoke to parents as well, parents have even said that actually they’ve gone home and they’re getting out the humus and the carrot sticks rather than the crisps and the chocolate bars. So you know everyone’s kind of seen a change and improvement. Which is great.

Many children commented on using a variety of strategies to help them achieve their goals:

With the magnet that you sent out, because I wasn’t eating healthy so my mum put it at my height on the fridge.

Well with mine, we photocopied our sheets so when I go round to my friend [name]’s house he’s got a copy of my sheet and when he comes round to my house we’ve got a copy of his sheet. We photocopied it as well so my granddad’s got a photocopy of my sheet so every time I go down there for lifeguard training I have my tea there and I have a look on the sheet and see what I can have ‘cause we made another one and what I can eat if you get what I mean.

I found it like, we wrote a little note or I think one of my friends did it or we like look at the fridge with our magnet, the thing what you gave us, and then we, sometimes when I open the fridge I’m like, then I remember all the stuff I’ve been taught, and that’s how I remember and how I achieve my goals.

We know that changes at a family level can have a positive impact on children making changes to their eating and activity behaviours,128 and there was strong evidence for this from parents and children alike. The results of the parent questionnaire indicated that 74% of families had made changes (see Appendix 17), with 78% of parents providing further details. These included providing less of the unhealthy foods in the household: ‘we don’t buy sweets/crisps to be kept in the house’ (parent of an engaged girl, school 6); ‘fizzy drinks replaced with squash and sparkling water’ (parent of an engaged girl, school 14). Parents also reported spending more family time doing physical activity: ‘we spend more time at weekends going out. Sometimes cycling or walking or going on scooters’ (parent of an engaged girl, school 9); ‘extra family walks, general family health and fitness’ (parent of an engaged girl, school 11); ‘try not to have unhealthy snacks in the house, make conscious effort to get more exercise’ (parent of an engaged boy, school 1).

This was also supported by the data from the parent interviews:

I’d always felt really guilty about them eating loads of biscuits, it’s just they are a really easy go-to food in the tin and kind of this the healthy eating lifestyle if nothing else it just made me really sit up and think about making changes as a family and so as a family you’ve got to change that and if we change nothing else you do need to change that.

Yeah definitely yeah because like you know if she would get a snack and she has a carrot, and then thinks oh yeah you will all have carrots, so in that respect you know she has pulled the whole family to a healthier snack rather than just herself. [Interviewer: OK] Yes and you know obviously when she does go outside she’ll take at least one of her sisters or brother with her, so yes I suppose in that respect I . . . no so much as discussing it with them or talk to them about it just by her actions really . . .

It’s mainly the 80 : 20 [Interviewer: Yep] and we still talk about it now um we, we always have healthy food because that’s our 80 [Interviewer: Yeah] um and er we don’t have, we, I now don’t have any snacks in the house at all um where I would have done before. I think it’s helped also that the children aren’t having packed lunches anymore as well, that helps not having snacks in the house [Interviewer: Yeah] and if they want to go and get a snack then that’s their 20 and they can walk to the shop and get it.

Yes we definitely decreased the fruit juice that we have. I think all of us. I think I knew but it’s just having it reinforced ‘cause you hear all sorts of things in the news.

In the focus groups, children (both the engaged and the less engaged group) also reported family changes:

Well after I started being more healthy, the whole family kind of started, so now there is no chocolate in the way so you won’t be tempted and we buy more healthy things.

I’ve noticed that normally before the Healthy Lifestyles Programme we used to have our cupboards stocked with different unhealthy foods like chocolate bars and cakes but now 75% of them have been replaced now.

Um yeah, we started doing more as a whole family together and going out more, like going on family walks on the weekend, instead of just having one person go out by themselves, and literally just going out all together.

• Research question 5: were there reported changes to children’s eating and physical activity behaviours?

The focus group and interviews suggested that children had made changes to their diet and/or activity/sedentary behaviours:

The one area that I did really notice that he picked up on is looking at ingredients of food. Looking at the percentage of fat in something and he’ll come home from school and rather than just asking for a biscuit he’ll go and get a huge carrot to have with it.

So um she does tend to eat more fruit now and sort of she’s interested in trying new things and I think that’s probably one of the best things that’s come out of it [Interviewer: Ahum] we’ve always had a battle with [name] eating and that but since she’s done the programme she’s been willing to try.

Teachers also reported being aware of changes the children had made:

It was perfect timing, to have that farm trip after the 80 : 20 programme to see if they had adjusted their lunchboxes to be a little bit more, there was more water and less fizzy drinks, less sweets, more fruit. Because I went around looking at the packed lunches as we were sitting on these bales of hay, and it was a kind of relaxed fashion, and the children were just openly talking about it straight away. And I thought wow the impact has been good.

The evidence from the focus groups and interviews of reported changes was supported by data from the parent questionnaire. Seventy-seven per cent of parents noticed changes in their child’s choice of snacks and 60% noticed changes in their choice of drinks. Fifty-five per cent of parents noticed changes to screen time, with 67% reporting changes to physical activity. In response to the question asking whether or not any observed changes had been maintained, 78% reported that they had (see Appendix 17). When given the opportunity to provide details of these changes, 89% of parents responded and provided a brief description of the change they had observed in their child. The majority of responses focused on changes in snack and drink choice; for instance, ‘X will now opt for a piece of fruit instead of a chocolate bar/biscuit etc.’ (Parent of an engaged girl, school 5) or ‘not drinking coke anymore’ (parent of an engaged girl, school 6). Parents also reported a raised awareness in their child around healthy choices: ‘X is more aware of healthy choices and is much easier to divert to healthy alternatives’ (parent of an engaged boy, school 10).

Parents identified increases in physical activity behaviour and their children taking up more clubs: ‘she increased time on the trampoline/outdoors regardless of the weather’ (parent of an engaged girl, school 11); ‘understands more about the need to be physically active, i.e. joined the swimming club’ (parent of an engaged girl, school 8).

Parents also reported changes in screen time, although to a lesser degree: ‘screen time has reduced and sports has increased’ (parent of an engaged boy, school 11); ‘there was always a limit on screen play but we did not have to monitor it as much as he was more aware’ (parent of an engaged boy, school 15).

Adverse events

There was one adverse event reported during the 5-year trial. A HeLP co-ordinator received an e-mail from a mother of a child from intervention school 16, following the healthy lifestyles week, regarding her daughter’s unhealthy exercising and eating behaviours. The trial manager spoke to the mother and followed up with an e-mail detailing the actions taken as a result of her concerns. The mother was happy for her child to continue with the programme, and the HeLP co-ordinator subsequently set goals with the child that did not focus on changing physical activity or dietary behaviours (which were already meeting the 80 : 20 balance). The HeLP co-ordinator informed both the school head teacher and the class teacher of this event and all were very supportive of the programme and the way the messages were delivered.

The trial manager interviewed this mother after the intervention was completed, and the latter said the following about this event:

Towards the end of the programme what was very good was that the team spoke to her after we chatted, and made that clear to her and suggested that she come up with goals that would help the family and that was a really good thing to do as it enabled her to use her imagination and to take what she’d learnt and put it into practice but without inflicting on herself which she didn’t need to do . . . Overall I think the message that they were left with was pretty good. I was happy that it kind of reinforced what we’ve always tried to teach them at home as well.

Summary

The HeLP intervention was delivered with a very high degree of fidelity to content and in the spirit of HeLP; moreover, 95% of children received what was considered to be the necessary ‘dose’ for behaviour change. Prespecified assessment criteria of engagement and interview and focus group data from teachers, parents and children showed that HeLP appeared to engage schools, children and their families across the socioeconomic spectrum.

There was no evidence to support the possibility of differential take up or engagement with the programme by school, gender or child-level socioeconomic status. This may be due to the engagement scores we developed not being sufficiently nuanced to capture actual engagement, and data from parent questionnaires, interviews and focus groups all suggested high levels of engagement as well as reported changes in the targeted behaviours. However, there was evidence, from the focus groups in particular, that children experienced most difficulty in setting and achieving their goals if they did not have sufficient support from home. All teachers were very positive about the programme and had no negative comments about the delivery or the impact on their workload.

Despite the reported changes, the results of the trial showed no differences in physical activity or sedentary behaviours between the children in the intervention schools and those in the control schools, although a difference was observed in favour of the intervention fewer types of energy-dense snacks and negative food markers. There are very few data to explain why the reported changes did not translate into measurable differences, especially with regard to physical activity. This was the most common goal set by the children, although fewer examples of how this was achieved were reported in the interviews and focus groups. The most likely explanation is that the changes that were discussed in the focus groups, interviews and questionnaires were not of sufficient intensity or duration for their effect to be captured at the 18-month follow-up.

A limitation of the process evaluation was the low response rate (25%) from parents to the questionnaire (which also formed the pool for parent interviews), which limits the strength of the data in relation to reported behaviour change, as well as the ability of the data to represent all participating parents. Eighty-seven per cent of parents interviewed were considered to be ‘engaged’ with HeLP, based on prespecified criteria, compared with 77% of the whole sample, although their socioeconomic status was similar to that of all parents.

Section 2: mediation analyses

Background

The mediation analysis tested whether or not changes in targeted cognitions and behaviour patterns could statistically explain any observed between group differences in trial outcomes. Our analysis plan stipulated that we would assess potential mediators of any observed group differences in the main primary and/or secondary outcomes and, therefore, the mediation analyses were carried out once the results of the trial had been revealed.

Significant intervention–control group differences were observed on two self-report dietary outcome measures, namely the average number of different types of energy-dense snacks consumed during the week (0.4 mean difference in favour of the intervention) and the average number of different types of unhealthy foods consumed during the week (negative food markers), which was a mean difference of 0.6 in favour of the intervention group. Mediation analyses were conducted to explore whether or not changes children’s self-reported knowledge, cognitions and behaviours could statistically account for these self-reported dietary differences.

• Research question 6: do statistical models combining potential cognitive and behavioural changes mediate observed between-group differences in the number of weekday energy-dense snacks consumed at 18 months and/or the number of unhealthy foods (negative food markers) at 18 months?

The My Lifestyle Questionnaire

We developed the MLQ to capture possible mediators of diet and physical activity behaviours that we hypothesised would need to change to affect weight status (see Appendix 3). The MLQ development was based on the HeLP intervention mapping process, and the 50-item questionnaire comprised a number of sections designed to assess knowledge, individual cognitions and motivations, perception of parental behaviours, and the use of behavioural strategies and change techniques, as well as specific behaviours targeted by the programme (see Appendix 18 for full details). Some MLQ items were theoretically derived and based on previous measures of theoretically defined constructs (such as attitude, injunctive and descriptive norms, self-efficacy and intention). Other items (including questions assessing knowledge and behaviours related to discussing healthy eating and physical activity at home with the family, helping to choose healthy options when shopping with the family and participating in cooking) were designed to assess changes in the short- and medium-term outcomes that HeLP was expected to generate.

Methods

Sampling and recruitment

Data were collected from all participating children in both intervention and control schools (see Chapter 2) completing the MLQ at baseline and at 12 months post baseline (i.e. immediately after the end of the intervention).

Data collection

The HeLP co-ordinators collected the MLQ data during a dedicated lesson and read the questions from the front of the class, with the children completing the questionnaire at the same time and in silence. Clarifications were given for specific questions, and a poster was produced to demonstrate percentages so that the children were able to understand a question relating to proportions (question 4 in the knowledge section; see Appendix 3). Children who required additional support completed the questionnaire in a smaller group outside the classroom with an additional researcher.

Analysis

Factor analyses to create composite My Lifestyle Questionnaire variables

An exploratory factor analysis (EFA) was conducted on half of the baseline MLQ data set, followed by a confirmatory factor analysis (CFA) (in the remaining data set) to generate composite variables that loaded together and could be taken into the subsequent mediation analyses. We adopted a parsimonious approach to the construction of these variables, summarising across theoretically derived and HeLP-specific MLQ items (see Appendix 18).

Table 43 shows the five statistically generated composite variables from the factor analyses, the MLQ constructs that they came from, the questions that fed into the constructs and the corresponding score ranges. The variable referred to as ‘knowledge’ was originally designed as the sum of five questions, with each question having several examples for the children to answer. We have called the four composite variables derived from the factor analysis ‘confidence and motivation’, ‘peer norms’, ‘family approval/behaviours and child attitudes’ and ‘behaviours and strategies’.

Structural equation modelling

Structural equation modelling was used for the mediation analyses. SEM allows the assessment of the relationships among a set of variables using multi-item scales, multiple variables and multiple outcomes.129,130 The variables included in the mediational analyses are baseline and 12-month BMI SDS, gender, school IMD score, class size and baseline and 18-month energy-dense snacks and negative food markers, as well as the five composite variables from the MLQ (see Table 43). The independent variable is the allocation group: intervention versus control.

There are two dependent variables: the number of weekday energy-dense snacks consumed per day at 18 months and the number of weekday unhealthy foods consumed per day (negative food markers) at 18 months. These were the secondary outcome variables from the main trial that had statistically significant between-group differences. We took a parsimonious approach to the modelling exercise and hence did not select the two other statistically significant secondary outcomes (the average number of energy-dense snacks and negative food markers foods consumed across the whole week at 18 months). Furthermore, we did not select healthy snacks at the weekend as the adjusted between-group difference from the linear random-effects regression model was not statistically significant (mean difference 0.23, 95% CI –0.04 to 0.50) [see Chapter 3, Primary analyses of secondary outcomes at 24 months (anthropometric measures only)].

Eleven variables were controlled at baseline. These included baseline scores for the outcome variables and the potential composite mediating variables, as well as four demographic measures – gender, BMI SDS, school size and school-level deprivation – that could have influenced (enhanced or diminished) scores on the dependent outcomes.

Variables in the path model to be tested using SEM are shown in Figure 9.

FIGURE 9 Pathway analysis conceptual model. B&S, behaviours and strategies; C&M, confidence and motivation; EDS, energy-dense snacks; FAB&CA, family approval/behaviours and child attitudes; NFM, negative food markers; PN, peer norms; SES, socioeconomic status. (Figure not reproduced here; see the original article.)

Preparation of the data included examination of, and possible replacement of, any missing values. For the ‘knowledge’ items, if a child did not answer an item it was assumed that they did not know the answer and were scored zero for that item. For the composite variables that contained Likert scales (see Table 43), the following rules were used to replace missing values: if up to 75–80% of items were available for a particular variable, this was averaged, and the rounded average was used to replace the 20–25% of missing values, thereby retaining the ordinal nature of the individual items.

The model shown in Figure 9 was tested twice: once for the outcome variable weekday energy-dense snacks at 18 months and once for weekday negative food markers at 18 months. According to our proposed model, the intervention may have a direct effect on the outcome variable at 18 months but also indirect effects through changes in the five composite mediating variables at 12 months.

Structural equation modelling127 was used to test this model, specifically whether the mediating variables function as full or partial mediators between the intervention and the outcome variables. Evidence for full mediation would be that a previously significant effect of the intervention on the outcome variable is no longer present when the composite mediating variables have been added to the model. On the other hand, if a significant association still remains between the intervention and the outcome variables after adding the composite mediating variables, this would indicate partial mediation, at best.

Software and type of factor analysis and goodness-of-fit indices used

Structural equation modelling was conducted with the software package AMOS version 23 (IBM Corporation, Armonk, NY, USA) using maximum likelihood estimation. As chi-squared is known to be inflated with sample size,131 it was not used as an indicator of goodness of fit (and misfit means that there is major deviation between what the model specifies and what exists in the actual data). Instead, a set of indicators was used to provide a fair assessment of goodness of fit. These indicators are known to be sensitive to different aspects of model misfits. The following criteria were used:

• root-mean-square error of approximation (RMSEA) of < 0.06132
• comparative fit index (CFI) of > 0.95132
• standardised root-mean-square residual (SRMR) of < 0.08132
• relative/normed chi-squared [χ²/degrees of freedom (df)] of < 5 (although this criterion is commonly used and well known, it is equally sensitive to sample size133).

Results

Of the 1324 children, 105 had missing data that did not meet the rules for replacing missing values. A conservative approach was taken by discarding these participants, which reduced the sample size from 1324 to 1219.

A descriptive summary of data for the five composite variables at baseline and at 12 months is provided in Tables 44 and 45. None of the variables demonstrated elevated levels of kurtosis and skewness. Cronbach’s alpha was good for all of the composite variables except peer norms, which approached the consensual cut-off value of 0.70 for acceptable internal consistency.

Tables 46 and 47 show the correlation coefficients for the five mediating variables at baseline and at 12 months, respectively. Knowledge had very low correlations, and in one case no correlation, with the other variables. The correlations among the confidence and motivation, peer norms, family approval/behaviours and child attitudes and behaviours and strategies variables were small to medium.

Preliminary analyses indicated that the conditions for mediation analysis126 had been met: (1) the intervention had a significant effect on each of the five 12-month composite variables, namely knowledge [t(1217) = 15.27; p < 0.01], confidence and motivation [t(1217) = 4.00; p < 0.01], peer norms [t(1217) = 4.01; p < 0.01], family approval/behaviours and child attitudes [t(1217) = 3.40; p < 0.01] and behaviours and strategies [t(1217) = 5.66; p < 0.01]; and (2) the composite variable scores at 12 months were significantly (p < 0.01) correlated with the two outcome variables: weekday energy-dense snacks and negative food markers (Table 48). As the outcome variable negative food markers at 18 months as well as the negative food markers score at baseline revealed slightly elevated kurtosis (1.86 and 2.18, respectively), the scores were logarithmically transformed before the SEM was conducted.

Our longitudinal path analysis contained data from three time points: (1) at baseline, including the demographic variables gender, baseline BMI SDS, school size and school-level deprivation, with direct paths to the outcome variable as well as the baseline scores of the outcome and demographic variables; (2) the 12-month mediating variables; and (3) the 18-month outcome data. The baseline score of the outcome variable (energy-dense snack or negative food marker) as well as the child’s allocated group (independent variable) also had direct paths to the outcome variable. The model shows that baseline composite variables predicted scores on these same variables at 12 months, which in turn predicted the outcome variable at 18 months. The intervention versus control variable also predicted the scores on the five composite variables at 12 months. Several of the composite variables were correlated at baseline (guided by modification indices of > 10). The error variances of the mediating variables at 12 months were correlated in an identical pattern to those pairs of mediating variables at baseline.

Table 49 shows the goodness-of-fit indices for the two longitudinal path analysis models (with energy-dense snacks and negative food markers as outcome variables). Overall, the model fits were acceptable for both outcome variables, and although the CFI results were slightly below the 0.95 criterion for an excellent fit, the other indices met the criteria.

Path analysis for weekday energy-dense snacks (18 months)

Figure 10 presents a diagram of the path analysis for energy-dense snacks. Not shown are the correlations of the variables at baseline, error covariances and non-significant regression weights (dashed lines). The following composite variables were significantly (p < 0.05) correlated at baseline: knowledge with family approval/behaviours and child attitudes and confidence and motivation; family approval/behaviours and child attitudes with confidence and motivation, peer norms and behaviours and strategies; confidence and motivation with peer norms and behaviours and strategies; and peer norms with behaviours and strategies. All paths shown in the diagram were significant (p < 0.05), except for the associations between gender and energy-dense snacks (18 months), between school-level deprivation and energy-dense snacks (18 months), between peer norms and energy-dense snacks (18 months) and between behaviours and strategies and energy-dense snacks (18 months). The direct effect of the intervention on energy-dense snacks (18 months) was significant (p = 0.041), indicating partial mediation by knowledge, family approval/behaviours and child attitudes and confidence and motivation. Of these three mediating variables, the effect of family approval/behaviours and child attitudes was the strongest, with a standardised regression weight of –0.18, followed by confidence and motivation (β = –0.09) and knowledge (β = –0.07).

FIGURE 10 Partial mediation model with significant standardised regression weights of pathways for weekday energy-dense snacks at 18 months. B&S, behaviours and strategies; C&M, confidence and motivation; EDS, energy-dense snacks; FAB&CA, family approval/behaviours and child attitudes; NFM, negative food markers; PN, peer norms; SES, socioeconomic status. *p < 0.05, **p < 0.01, ***p < 0.001. (Figure not reproduced here; see the original article.)

Path analysis for weekday negative food markers (18 months)

Figure 11 shows the full mediation model with significant standardised regression weights of pathways for weekday negative food markers at 18 months. Not shown are the correlations of the variables at baseline, error covariances and non-significant regression weights (dashed lines). The results for negative food markers were generally similar, with the same variables correlated at baseline and at 12 months. Similar to the results for energy-dense snacks, there were no significant associations between gender and negative food markers (18 months), between school-level deprivation and negative food markers (18 months) and between peer norms and negative food markers (18 months). However, the association between behaviours and strategies and negative food markers (18 months) was significant (p < 0.05), and there was no significant association between school size and negative food markers (18 months). Additionally, the direct effect of the intervention on negative food markers (18 months) was not significant (p = 0.059) indicating full mediation by knowledge, family approval/behaviours and child attitudes, confidence and motivation and behaviours and strategies. Of these four variables, the effect of family approval/behaviours and child attitudes was the strongest, with a standardised regression weight of –0.18, followed by confidence and motivation (β = –0.10), behaviours and strategies (β = 0.08) and knowledge (β = –0.06).

FIGURE 11 Full mediation model with significant standardised regression weights of pathways for weekday negative food markers at 18 months. B&S, behaviours and strategies; C&M, confidence and motivation; EDS, energy-dense snacks; FAB&CA, Family approval/behaviours and child attitudes; NFM, negative food markers; PN, peer norms; SES, socioeconomic status.*p < 0.05, **p < 0.01, ***p < 0.001. (Figure not reproduced here; see the original article.)

Overall, the effects of the composite mediating variables on the outcome variables were fairly small. The largest association was between family approval/behaviours and child attitudes at 12 months and the outcome variables. For both paths (between family approval/behaviours and child attitudes and energy-dense snacks and between family approval/behaviours and child attitudes and negative food markers), β was –0.18, which means that an increase of 1 SD of the family approval/behaviours and child attitudes variable was associated with a reduction of 0.18 in energy-dense snacks and negative food markers. Additionally, the results of the behaviours and strategies variable need to be interpreted with caution. As outlined in Appendix 18, this variable was the least psychometrically robust of MLQ variables. The path between behaviours and strategies at 12 months and energy-dense snacks was not significant, but the path between behaviours and strategies at 12 months and negative food markers resulted in a significant change in the opposite direction to those of the other mediating variables. This counterintuitive result is most likely a result of collinearity; as shown in Table 48, behaviours and strategies was negatively correlated with negative food markers. However, in the context of the other MLQ variables, this association became positive (β was 0.08; see Figure 11). Such seemingly paradoxical cases have been described in the literature134 and are most likely due to collinearity with other predictor variables or the operation of suppressor variables. Table 47 shows that behaviours and strategies and confidence and motivation were correlated to an extent (r = 0.69) that could question the statistical independence of the variables and further multiple-linear regression analyses confirmed that the coefficient of behaviours and strategies does change direction when confidence and motivation is added as a second predictor. Further exploration of these composite variables could clarify which items are responsible for these effects.

Summary

We developed and evaluated a self-report tool, the MLQ, which aimed to capture changes in knowledge, cognitions and behaviours that could explain changes in weight gain. The MLQ contains items that are relevant, acceptable and feasible for 9- to 10-year-olds to complete in a timely manner and it has undergone psychometric testing, although future additional evaluation of the MLQ could include further tests for reliability and construct validity. Five composite variables emerged from the evaluation of the MLQ and these were used as the mediating variables in the two longitudinal path analyses. The first analysis was for the number of weekday unhealthy foods consumed per day (weekday negative food markers at 18 months) and showed statistical evidence for full mediation as the previous significant effect of the intervention on this outcome variable was no longer present (p = 0.059) after all five mediators had been entered into the model. This result suggests that the composite variables arising from the MLQ could provide a possible explanation of how the intervention generated the observed differences in dietary behaviour. However, further exploration could clarify this explanation. The analysis for the number of weekday energy dense snacks consumed per day at 18 months revealed partial mediation (p = 0.041); one possible explanation for this is that other mediating variables exist that have not been measured in this study. Overall, it is noted that the full and partial mediation effects were both close to the cut-off point for statistical significance (p = 0.05).

The findings arising from our use of composite variables require cautious interpretation; we are able only to conclude that children have changed in a combination of ways for two of the secondary outcomes in the trial. Future work could include more theoretically based modelling work (which would be possible given the theoretical underpinning of many of the items in the MLQ) as well as cluster analyses135 to investigate which specific mediator and which moderator variables might predict healthy outcomes in the whole HeLP cohort.

Conclusions from the process evaluation

Data from the process evaluation show that HeLP was delivered as designed in all 16 intervention schools, with high uptake and engagement from schools, children and their families across the socioeconomic spectrum. Qualitative data suggest that HeLP worked in accordance with the underpinning IMB model and the logic model by changing children’s self-reported levels of knowledge, cognitions and behaviours as intended; however, these changes were insufficient to affect physical activity behaviours at 18 months or BMI SDS at either 18 or 24 months post baseline. The mediation analyses show that the intervention effects on the consumption of weekday energy-dense snacks were mediated by knowledge and two composite variables, namely family approval/behaviours and child attitudes and confidence and motivation, whereas the intervention effect on weekday consumption of unhealthy foods (negative food markers) was mediated by the same variables as well as the composite variable behaviours and strategies.