Parts of this chapter have been reproduced from Tibbitts et al.56 This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
Process evaluations are essential to understanding whether or not interventions have been delivered in the way they were designed, and to identify the factors affecting delivery fidelity. We conducted a process evaluation of Action 3:30 using quantitative and qualitative approaches and the findings were framed within the RE-AIM framework.12 The RE-AIM framework is a conceptual framework comprising five elements that relate to health behaviour interventions (reach, effectiveness, adoption, implementation and maintenance). The goal of the RE-AIM famework is to encourage programme planners, evaluators, readers of journal articles, funders and policy-makers to pay more attention to essential programme elements, including external validity, that can improve the sustainable adoption and implementation of effective, generalisable, evidence-based interventions.12 This chapter is structured using each component of the RE-AIM framework as a section to focus the process evaluation findings on these important elements. In addition to quantitative measures, this chapter includes perspectives from pupils, TAs and school staff involved in Action 3:30, as well as external stakeholders, to present a balanced appraisal of the intervention and factors that may have affected its reach, delivery, efficacy and potential sustainability. Our stakeholder groups were pupils who received the intervention (focus groups were conducted with six boys and six girls from each of the four schools that delivered the intervention), the nine TAs who delivered the intervention in the four schools, seven key contacts from schools involved in the project, the lead trainer, and eight external stakeholders from a range of organisations involved with education, health promotion and school physical activity provision. Four key contacts were school staff with a responsibility for PE/after-school club co-ordination, one was a deputy head teacher, and one was a school business manager. No-one refused to participate. External stakeholders included representatives from Sport England, Public Health England, Youth Sport Trust, school sport partnerships and local authorities, and school sports co-ordinators from schools not involved in Action 3:30.
The duration of pupil focus groups ranged from 19 to 49 minutes. The lead trainer interview lasted 42 minutes. The two classroom teacher interviews lasted 4 and 6 minutes. The duration of TA interviews ranged from 45 to 60 minutes, key contact interviews ranged from 8 to 40 minutes and external stakeholder interviews ranged from 11 to 46 minutes. Quotations have been anonymised and are presented with school ID alongside informant group (e.g. School 25, boy; School 33, TA) or, in the case of external stakeholders, their informant group followed by their ID (e.g. external stakeholder 2).
Reach
Using the definitions provided by the RE-AIM framework,12 and adapting the wording to apply it to the context of this process evaluation, reach is defined as:
The number, proportion and representativeness of individuals (TAs and pupils) who are willing to participate in Action 3:30 when offered the opportunity, as well as factors that may affect this.
School recruitment
Of the 27 schools approached at initial recruitment, 12 consented to take part in the study (44.44%) (see Table 5). School key contacts cited the main reasons for signing up as to add to their after-school provision and to provide more physical activity opportunities to their pupils, especially for those less active. These reasons also aligned with school policies and/or priorities to get less-active children more active:
We actually have a role within school where there’s one person who is trying or making sure that they’re seeing those children that aren’t getting involved in activities and trying to get them to go to clubs. So, Action 3:30 was brilliant because it was kind of doing half that job for us because it’s getting those children who wouldn’t necessarily go to a sports club to go to them and ticking those off to make sure that all children are active within the school.
The key contact from a school with a higher pupil premium (serving a more deprived area) also said that the study was appealing to their school because intervention delivery costs would be covered and the club would be free, a factor they knew was important for parental engagement:
I think the fact that it was free was what appealed to a lot of parents.
Pupil recruitment
Reasons for enrolling
Across the 12 schools recruited, 459 eligible pupils consented (41%) to take part in Action 3:30 and 228 (48%) of those were girls. Boys and girls signed up because they thought that the club would be fun or different, because it would improve their health or fitness, and because it was free:
I wanted to join more clubs. It sounded more active and so I joined it.
I thought it was fun and I thought I would give it a go.
For girls, either meeting new people or being with friends was a reason for enrolling, suggesting that friendships may be an important factor in girls’ participation in new after-school clubs:
I signed up because my friends signed up because they thought it was going to be fun.
Reasons for not enrolling
The main perceived reason why other children did not enrol was that they were not interested in Action 3:30 because they did not like sport and activity, they would rather be doing sedentary activities or they may take part in other clubs:
One of my friends said that they didn’t like it because they’re doing enough sport and they wouldn’t have the time in the week.
These reasons align with those most frequently selected by parents on the ‘reason for non-consent’ form (i.e. ‘My child does enough already’ and ‘My child is not interested in Action 3:30’).
Both key contacts and TAs said that parental engagement could be a barrier to participation because parents may be unable to collect their children from after-school clubs or may be unsupportive in other ways. It was suggested that doing more to involve parents at initial recruitment may be beneficial:
I think the only restriction that perhaps a few of them had with attending the club was parental support, to be honest, because I think there are some children whose parents just couldn’t be bothered to let them do it.
Some girls suggested that others may have felt nervous or shy because of not being with their friends, again suggesting that friendships are an important factor to consider when tailoring recruitment methods to appeal to girls:
A couple of my friends said that they didn’t want to do it because they were a bit nervous because they didn’t know who was going to join.
This was also suggested as a potential barrier by one of the TAs, who linked this to children’s perception of their ability or competence compared with that of others:
I think some of the problems they have, is that they’re in the same group with other people very able. And so perhaps that’s not knowing who else is going to be doing it, I suppose, is probably off-putting for some of the ones that don’t feel confident.
Recruitment method
The recruitment method was designed to appeal to children who were less active by promoting that the club would be inclusive, focus on enjoyment and include a range of physical activities rather than sport. This was reflected in some pupils’ decision to sign up:
The way that they explained it, it sounded more like it would include everybody, not like leave everybody out.
Some pupils mentioned that they signed up because they thought that the club would be sporty, which may suggest that it also appealed to the more active children or those who were interested in being active:
I like sports and I thought it would be sporty.
The recruitment method allowed all eligible pupils to sign up to the study and involved randomly selecting pupils to take part. However, two of the intervention school key contacts expressed that they would have preferred to target less active children only, rather than randomly selecting:
We thought it was going to be targeting those children that aren’t currently active in extracurricular clubs. It wasn’t until we got a little bit further underway that we understood that it was going to be a complete random thing of whoever would go.
Table 9 (objective 2c) indicates that there was no difference in baseline physical activity levels between children who did and children who did not sign up to the study, suggesting that Action 3:30 was successful at appealing to pupils from across the physical activity spectrum, including those who were less active. This was corroborated by TAs:
I think there was a mixture, I think there was a lot of them that do clubs already and are very sporty already. But I think there was also a surprising number of ones that don’t, actually, so obviously you do appeal to lots of them.
Teaching assistant recruitment
Interviews with TAs and key contacts revealed that all nine of the TAs who delivered Action 3:30 volunteered to do so, and there did not seem to be any barriers to TAs taking part in those schools. In two of the intervention schools the project was advertised to all TAs in the school, and in the other two the TAs were asked by key contacts whether or not they would be interested in the opportunity. Two of the six allocated intervention schools were unable to deliver the intervention, one of which cited that it was unable to release staff for the training, which suggests that they did have willing staff. The other school was unresponsive, so it is not clear whether or not recruitment of TAs in that school was an issue that affected the school’s ability to deliver the intervention.
The principal driver of TAs’ signing up was professional development. Other reasons given were that it would be a different experience working with children whom they did not usually work with and that they had a personal interest in engaging children in physical activity:
That’s the kind of thing that they sold it on, that you’d be going to a training course to run a club as an experiment and that there’d be a qualification at the end of it.
Teaching assistants at two schools mentioned that they had wanted to deliver the club together and would not have signed up without the other TA. This could be advantageous or prohibitive if TAs would take part only with certain other staff:
Which was always our argument, we won’t do it with anyone else, we’ll do it together or we won’t do it basically.
Pupil attendance
Of those who started the club, 81 pupils (73.5%) attended > 50% of sessions, the highest being 88.2% in school 25 (see Table 18). Including those who signed up to Action 3:30 at baseline but did not attend any club sessions, this proportion was 66.3%. In either case, this exceeded the 50% progression criterion. The club clearly appealed to and engaged girls as well as boys, as 70% of girls attended at least 50% of sessions compared with 74.4% of boys. School staff perceived attendance to be excellent:
We had great attendance. Our kids were really on board with coming.
The attendance of them staying is probably one of the best in the school.
Furthermore, a key contact highlighted that several of the regular attendees at their club were children who did not normally take part in after-school clubs:
Really happy that they’ve got children who wouldn’t otherwise have gone involved but also that they, they’re staying ‘cause you don’t always see that either.
Reasons for attending
Pupils expressed that they were always keen to attend Action 3:30 and were disappointed if they we unable to attend:
Every Tuesday I’d be like thank goodness it’s Action 3:30 today but on Thursdays, I’d be a little bit depressed because I wouldn’t be able to go.
When asked for reasons that motivated them to attend, pupils said that they liked the leaders, said that the club was fun, non-competitive and something to do with friends and alluded to the club’s autonomy-supportive nature (i.e. ideas being listened to, close relationships with friends, task adaptation):
Because of the amount of fun we had and the connection between everyone and we would have great ideas and how our friendship would get better.
I liked when they improved all the games and they added more stuff.
Barriers to attending
Scheduling conflicts with other clubs
Except for individual reasons, such as family commitments and sickness, the main barrier to pupil attendance was scheduling conflicts with other clubs. When pupils were asked why they could not attend all Action 3:30 sessions, many stated that they did other after-school activities either in or out of school and because of this some children attended only one session per week:
I could only meet on the Wednesday because on Mondays I have swimming.
Teaching assistants and key contacts agreed that scheduling conflicts were likely to be the main reason for non-attendance. Schools found that attendance varied termly depending on the number of other clubs that had been offered at the start of each term:
You’ve got those that used to come on Tuesday but wouldn’t necessarily come on Thursday because youth club was running.
Some of them might have come at the start and then last term have gone to a different club but now they are coming back again.
Parental support
Teaching assistants and key contacts mentioned that parental support was also a potential barrier to attendance. It was suggested that parents may not want or be able to come back to school to pick up their children, especially if they had other children who finished school earlier. In addition, because the Action 3:30 club ran twice per week and was free, parents may have felt less committed to ensuring that their child attended both sessions:
I think one of the factors was that some parents didn’t want to come at 3.15 p.m. to pick up a younger sibling and then come back again at 4.15 p.m.
One of the other issues we see is when it’s a free club, where there’s completely no barrier to entry, you don’t always get the commitment from people.
Summary
The quantitative and qualitative data collected about recruitment and attendance indicate that, once schools had signed up to the study, Action 3:30 was successful in reaching and engaging a variety of children, including girls and those who normally did not engage with after-school activity opportunities. Engaging TAs in the project was not a problem in four intervention schools but presented a barrier in a school that subsequently withdrew, suggesting that engaging TAs earlier in the process would be recommended. Appealing to parents and offering an alternative to existing provision were highlighted as key drivers of school interest. Barriers to attendance, particularly scheduling conflicts, may have limited the reach of Action 3:30 in intervention schools. The potential impact of reduced attendance at Action 3:30 clubs on intervention effects is addressed in Chapter 7.
Effectiveness
In the context of this process evaluation, effectiveness is defined as:
The impact of Action 3:30 on the children, club leaders and schools involved, as well as factors which affected, or may affect this.
Effectiveness of the training
As part of the intervention, TAs attended a 5-day training course before the start of the club. The course was intended to teach TAs how to deliver fun, inclusive and active physical activity sessions to a group of children. When the TAs were asked what they had taken away from the training, one replied:
Well, how to motivate children; not only in sport, in general. And it’s something that we can apply with any other session that we’re running with children; giving them that sense of belonging, giving them that sense of autonomy.
The training involved both practical and theory-based elements, which seemed an effective way of engaging all of the TAs and ensuring that different learning styles were catered for:
I liked the mix of doing obviously the paperwork side, the theory stuff.
The training was also grounded in SDT to promote autonomy, competence and relatedness need satisfaction in the pupils. The training aimed to encourage TAs to adopt an autonomy-supportive teaching style and a positive motivational climate in which children felt a sense of belonging to the club, choice in the activities they do and increased confidence to be active. TAs felt that this was a crucial part of the training and that the content was understandable. TAs perceived their teaching style to be more autonomy-supportive after the training (6.48 ± 0.47) than before (5.83 ± 1.05) (see Figure 3 – scores out of a possible 7). This is also reflected in the TA interviews, suggesting that the theoretical underpinning was successfully taught:
Well, how to motivate children; not only in sport, in general. And it’s something that we can apply with any other session that we’re running with children; giving them that sense of belonging, giving them that sense of autonomy and all those things. That’s what I found interesting.
Effectiveness of the intervention
As shown in Table 19, there was no empirical evidence of difference in daily MVPA between the intervention arm and the control arm at follow-up (adjusted difference in means –0.5 minutes, 95% CI –4.57 to 3.57 minutes). However, key contacts in intervention schools believed that, perhaps because of the design of the activities, the club had a positive impact on pupils’ engagement in physical activity, especially for those who did not take part in other, conventional clubs:
In terms of those children that aren’t accessing anything, I think the games in the multiskills-based thing is good because quite often those children have already decided that they don’t want to take part in football or tag rugby or anything like that and maybe need some ideas in games just to be able to get active. So, in terms of getting children more active, I think it’s been a positive thing.
Action 3:30 was brilliant because . . . it’s getting those children who wouldn’t necessarily go to a sports club to go to them and ticking those off to make sure that all children are active within the school.
In addition to aiming to increase MVPA levels, the intervention was intended to demonstrate effectiveness in increasing pupils’ enjoyment of and autonomous motivation towards physical activity.
Enjoyment
Pupils enjoyed Action 3:30 very much. When asked to rate Action 3:30 out of 10, pupils were very enthusiastic about the club, with all rating it highly:
One hundred out of ten.
I’d give it a ten.
The pupils stated that they liked that Action 3:30 was fun, had plenty of variety, included lots of active team games and was a mix of two year groups. Pupils from all focus groups mentioned that they enjoyed the child-led focus of the club, which they described as having choice and being able to make up new rules and games. This shows evidence of intervention fidelity, as what pupils describe suggests that TAs followed the leader’s manual and took on board the theory taught to them during training:
I had enough choice. When we were playing games, we were able to make choices to improve the games, an example, when we played dodge ball, we changed the rules a bit to make it better, it was more fun.
Teaching assistants also felt that the child-led activities and active team games were the most enjoyed by pupils, and key contacts perceived pupil feedback to be very positive:
Most of them enjoyed the ones where they got to add in more rules or change it and we literally said, ‘It’s up to you.’ They loved that.
I think they’ve really enjoyed it . . . no one’s ever said anything negative and the children that have attended have been really positive about it, especially when I spoke to them . . . it’s nice to see that some of those children that didn’t engage before are now engaging and being very positive about it.
Interim process evaluation measures during the intervention reflect the qualitative data presented above, as children also rated sessions as highly enjoyable at the time. Figure 4 shows that the mean enjoyment rating exceeded 4 for each process evaluation visit (the maximum possible score being 5).
Barriers to enjoyment
Pupils expressed that they enjoyed Action 3:30 less when there was disruptive behaviour (which meant that less time could be spent on activities), when there was a lack of space to play a game properly and when playing individual games that tended to be less active. TAs from two schools expressed that they thought that the pupils did not particularly like the skill-based activities because they were too similar to PE; however, TAs from another school felt that their pupils enjoyed these activities because they could see an improvement in their own performance. One TA also felt that introducing too many activities in a single session decreased pupils’ enjoyment because there was too much to learn.
During the focus groups, girls were specifically asked whether or not they felt that the club was suitable and enjoyable for them and whether or not they liked participating with the boys. Girls sometimes felt that boys were slightly dominating, disruptive and unfair, and they had mixed views about whether they would prefer a mixed or a girls-only club. However, being with the boys did not seem to be a major barrier to girls’ enjoyment and engagement. TAs in all schools reported that girls were engaged in sessions throughout the intervention:
It was very equal, but I personally think that the girls were very, very engaged. You have children like [names of girls], they really got into it and they felt they could lead games and tell people and organise.
Underpinning of self-determination theory
There was no evidence to suggest that the intervention had the potential to change quantitative measures of autonomous motivation for physical activity, or psychological need satisfaction for physical activity at follow-up (see Table 23). Somewhat conversely, the qualitative evidence suggested that the intervention may have resulted in autonomy, competence and relatedness need satisfaction of the pupils.
Autonomy
The focus groups with children suggested that they did feel autonomous during the Action 3:30 club, probably as a result of the autonomy-supportive teaching style that the TAs adopted after training. In addition, Figure 3 presents data from the TA-self efficacy questionnaire and shows that TAs perceived their teaching style to be autonomy-supportive throughout the intervention (as shown by a mean score of at least 6 out of 7 post training and at each process evaluation visit). This was also reflected in the child autonomy-support questionnaire, as mean scores ranged from 5.91 to 6.04 (maximum score of 7.0) across the three process evaluation visits (see Table 17). Interviews with TAs also revealed that they offered their pupils plenty of choice and created an autonomy-supportive atmosphere:
We did a lot of breaking it down, getting them to reflect and then changing it, adapting it and they loved going back to the game.
During focus groups, when children were asked what they liked about Action 3:30, many of them alluded to the fact that they had autonomy. Pupils felt that they had a say in the activities and games they played and were able to adapt them to make them better because TAs listened and took their ideas on board:
It was different because in PE we have to do it, whereas in Action 3:30 we get to say what we want to do.
They encouraged us to ask questions and tried to understand the way we wanted to do it.
Both free choice (e.g. ‘what would you like to do?’) and option choices (e.g. ‘a or b’) were offered to the pupils to promote autonomy. Sometimes pupils were able to freely adapt rules or make up new games and other times they voted to choose between games, a mix they seemed to enjoy:
. . . they always ask you, ‘What could we improve in this game?’ or ‘What new things could we add in this game?’
. . . everybody sat down, and everybody put their hand up for which games you wanted.
Teaching assistants highlighted that giving autonomy to every pupil was sometimes difficult because, for example, not all pupils wanted to play the game that the majority voted for or not all pupils were able to express their ideas in every session:
. . . but it is difficult because you know we’ve got, say even if we’ve just got 19 children . . . we’ve got 19 answers and I choose one, so that’s great for that one autonomy but the 18 . . . so you know it’s tricky to hit all of them using that sometimes.
Competence
The interviews with TAs and the focus groups with pupils suggested that both perceived that pupil competence had increased as a result of Action 3:30. TAs felt that their pupils had become more confident at taking part in physical activity and that they had improved their physical and social skills:
It was extremely rewarding to see them change in so many different ways with their physical activity, with their skills, with the way that they became sports people and encouraging.
When pupils were asked about the difficulty of the activities, those in one school explained that TAs spilt the group based on ability and competence and asked the pupils which group they would rather play in, so that they felt more confident about joining in. Pupils seemed to like this structure because they felt supported and able to join in at their own pace:
. . . you had one side, that was the side where you don’t feel very confident, but on the other side that was like the one where you felt really confident and knew you could do it. I think on the side where you didn’t feel confident, [name of TA] was helping them out, and then on the side that I was on [name of other TA] was saying, ‘Yeah, you’re doing this right, don’t worry’.
Pupils in the other schools also felt that TAs adapted activities to suit their capabilities, their confidence had improved and Action 3:30 encouraged them to participate in more activities than before:
I like doing sports and all that and games, it’s just that I was too frightened before I started that everyone would laugh at me. Then as I got better and as the sessions went by I felt like I could become more confident in it. Then that’s how I started doing the activities more.
Among those pupils who took part in the focus groups, it could be suggested that Action 3:30 influenced their competence in other physical activity settings, such as PE:
When I was doing PE before we were doing Action 3:30, I was a little like scared if we were doing something like rounders . . . but when I done Action 3:30 I got more confident.
The process evaluation findings show that pupils felt more active and confident to participate in physical activity, which may suggest that the intervention was effective at increasing competence for some pupils.
Relatedness
Teaching assistants expressed that they thought Action 3:30 had a good club atmosphere and that pupils showed good teamwork and sportspersonship, supported each other and were able express themselves freely. TAs also felt that they developed a good rapport with the pupils:
They’re well aware of each other as well. So, they knew the children that weren’t necessarily getting the ball and maybe couldn’t run in quite the same way, so they compensated their behaviour in a way.
I think they do enjoy coming and they enjoy seeing us there and they like love to call out in the dinner hall.
A friendly and inclusive atmosphere was created within clubs at each school, and both TAs and pupils made the club their own. In one school, children decided to answer the register with ‘Action 3:30’:
They felt that it was their group and they had that sense of belonging and all those things. I think that’s when they really enjoyed it and got the most out of it. They took it on board. So, when we were doing register they’d shout out instead of saying, ‘Yes,’ or, ‘Here,’ they’d go, ‘Action 3:30’.
Pupils expressed that they were motivated to attend Action 3:30 because they developed new friendships, liked working as a team and felt encouraged and cared for by the Action 3:30 leaders:
Because of the amount of fun we had and the connection between everyone and we would have great ideas and how our friendship would get better.
The qualitative data and observation visits suggest that relatedness was supported and experienced within Action 3:30, and a sense of belonging was alluded to by pupils in the focus groups.
Summary
There was extensive evidence that pupils enjoyed Action 3:30. The after-school programme successfully engaged both girls and boys, suggesting that the format of the sessions and the delivery style appealed to girls. Need satisfaction for pupil autonomy, relatedness and competence was met, according to data from child focus groups. However, there was no quantitative evidence of promise that the intervention could change autonomy, relatedness and competence need satisfaction. These findings and implications for measurement are covered greater detail in Chapter 7.
Adoption
Within the context of this process evaluation, adoption is defined as:
The number, proportion and representativeness of schools (settings) or TAs (agents) who are willing to initiate Action 3:30, including factors that may affect this.
Adoption in the current project
Over 40% of schools approached were willing to take part. Each of the 12 schools that committed to take part in this feasibility trial did so in the hope that it would receive the intervention. The main barrier given from schools that were initially contacted to take part in the study but did not consent (data not presented here) was that they did not have the capacity for additional after-school provision. Interviews with external stakeholders corroborated this, as they thought that primary schools often already had plenty of after-school clubs and that having the capacity to provide more would be particularly challenging for smaller schools:
I think at the moment primary schools are at a place where they’re building up their after-school offer on the whole. Some of them have felt they’ve probably reached capacity in some respects.
Interviews with key contacts in non-intervention schools confirmed that their schools did already have plenty of after-school provision, but this was not a barrier to adopting the intervention:
We have got a large school site so, we do have several things that happen after school every day, but we have space to accommodate more.
Within the schools that delivered the Action 3:30 intervention, each of the TAs volunteered to do so, suggesting that offering TAs the opportunity to deliver after-school programmes such as Action 3:30 is feasible and can result in adoption. In one of the two intervention schools that were unable to deliver the intervention, the reason cited for this was difficulty releasing staff to attend training to be club leaders, and not a lack of willingness from staff:
We had a couple in place but then again, they have full-time jobs and so they couldn’t go to the training. We then found someone else who couldn’t go on two of the training days and then they wouldn’t be fully trained.
Contrary to this, a key contact in a control school expressed that releasing staff would not have been an issue for them had they been chosen to receive the intervention:
No because it’s CPD [continuing professional development] for them and if it’s something that they’re interested in thinking about building confidence, supporting PE and stuff and I’m not certain all our TAs have.
The difference between schools that were able to deliver the intervention and those that were not seems to have been related not to staff interest levels but rather to individual school factors, such as the level of staffing and school priorities. This suggests that more needs to be done at the school recruitment stage to understand school priorities and staff capacity considerations before the intervention is adopted at the school level.
Factors that could affect adoption
Interviews with key contacts and TAs in intervention schools highlighted certain key factors that they felt would affect whether or not Action 3:30 would be adopted by their school or by others, having had insight into the programme. The primary factor key contacts cited was whether or not delivery could be funded:
I think the only way that would happen would be if we started charging the children, because the school doesn’t have any funds to fund teaching assistants to do clubs.
This view was echoed by one external stakeholder, a primary school PE lead, who identified that the cost compared with that of other provision was a crucial factor for schools when considering adopting any new programme:
[If you are] . . . going to be introducing something new, it would need the costing to be in line with what we already have in place.
When asked how the programme could be enhanced to maximise adoption in schools, external stakeholders also suggested increasing the flexibility of delivery to allow schools to target different subgroups of pupils in line with their school objectives, or to allow different delivery formats (e.g. at a different time of day) that would fit better with existing school provision. Targeting certain children was also mentioned by both key contacts and TAs, and this is described further in Maintenance:
Might be inhibitive to some children, so you just might want to consider how the club can be flexibly delivered at different parts of the school day.
Several additional factors were highlighted in external stakeholder interviews as influential in schools’ decisions about whether or not to adopt new programmes. These broadly fell into three themes: needs-related, evidence-related and practical considerations.
Needs related
External stakeholders stressed the importance of addressing a gap in provision, which may mean adding an option that creates balance within the programmes they offer:
It won’t just be about sport, it will be about delivering a balanced programme . . . Because each school might have a slightly different perspective on what they’re wanting.
School targets were also highlighted as key drivers of school needs. The notion of whole-school outcomes influencing adoption was supported in external stakeholder interviews, for example whether or not the programme could generate improvements in well-being for all students and not just for one class or year group:
If we start with the programme itself and its relevance the key thing about sustainability is that it always has to deliver what the school needs . . . the key to sustainability for most schools is that it has to deliver whole-school outcomes.
Aligned with school needs and gaps in provision was a suggestion to provide schools with guidance on how the programme could (1) benefit them and (2) be used by them. By addressing a need, and by identifying how a programme could cater for it, schools may see the value of a programme more readily:
The content is well set in terms of allowing those kids to come into that non-threatening environment and I think if schools were given a bit of guidance on who this might be good for it’d be very simple.
Evidence related
Repeated mention was made of the value of evidence in relation to the potential for adoption of new programmes. External stakeholders emphasised that credibility gained from evidence of impact is a strong driver of commissioning at a regional level via public health promotion teams within local councils:
So, with the strategy, we are looking for the evidence base and that’s where programmes like yours come in. We tend to only want to commission things that have a good evidence base if possible.
One external stakeholder, who was also a PE lead in a primary school, suggested that schools are more interested in evidence of reliability and consistency:
. . . we’d maybe look for that reliability and consistency. Because we have had companies that sent in different members of staff, you know with the children that’s not ideal.
Two stakeholders also mentioned credibility in a different context: credibility of affiliation to a respected institution, or ‘brand credibility’:
. . . the one thing that they want to be able to do is to put a logo on their headed paper and to be able to demonstrate to their investors and their governors’ networks that they are working with a credible brand.
A third angle on credibility was highlighted by several stakeholders, whereby existing coaching companies that work in schools already have accrued credibility of a different kind, as they have earned the trust of schools by working with them for some time. They suggested that alliances with these external companies might be more advantageous than offering schools a programme that replaces them:
. . . actually the company that we built a relationship with now, he was from [local partnership], so I had some background from him . . . and had confidence that he knew how our partnership worked.
I think a conversation with a commercial provider . . . I mean if you think about it, University of Bristol carries with it a certain kudos, however, the coaching companies are in every school . . . So, they’re already in there, they’ve already got some credibility so by buying in your product they would enhance what they’re delivering, and it would also stop you having to think so much about how you can expand and grow.
Practical considerations
The most frequently cited potential barrier to adoption suggested by external stakeholders was the burden of sending staff to 5 days of training during school hours:
I guess to take them out of school for the five days is one element and then it’s picking up the after-school cost is a totally separate issue. So, whether or not they can physically afford to do that in addition to the coaching that they might already be paying for, it’s a challenge, I would imagine.
External stakeholders and key contacts drew links between the burden of the training programme and the school budgetary concerns, relevant in an economic climate where funding for TAs has been cut in many local authorities:
. . . releasing staff was difficult for training because the two staff that we decided on were TAs in resource base where they were one-to-one so that was a cost to the school because we had to make sure that they were replaced.
I think the availability and resources for TAs have been stripped back and cut hugely in the last 2, 3, 4 years and so actually those people in schools are now not really there or if they are they are stretched to the very limit of their capacities.
However, this was not deemed a barrier in all intervention schools:
The release for the training was fine. Obviously, when you lose your teaching assistant it’s not great, but you cope with it, it wasn’t a huge amount of time.
Although the training format was burdensome for some schools, intervention school key contacts and TAs felt that 5 full days of training during school hours was appropriate in terms of both where the burden was placed and creating the right environment for learning:
I think having a day you get more time to discuss things. So, you’ve got more time to go over something if someone’s unsure of it. Whereas if it was only a couple of hours, you more likely to rush through it maybe.
Funding was a recurring theme throughout all interviews. Although there has been a recent doubling of the PE premium funding given to primary schools, which could be used to pay for staff training and after-school provision,63 there is still very little clarity about what schools are choosing to spend this money on. It is possible that some schools are not aware of what the guidance says they can and should spend it on. Key contacts in two intervention schools stated that they would use PE premium money to run Action 3:30 as an independent club; however, others did not. The external stakeholders we interviewed for this study all believed that this money could be used to implement programmes such as Action 3:30 and that it would be sensible to highlight this when it came to promoting the programme more widely:
. . . if you’re looking at who is going to buy it I think they would use sports premium to do so, so you need to link what it is with sports premium, with the Ofsted [Office for Standards in Education, Children's Services and Skills] framework, with the childhood obesity plan and then you’re half-way there.
Recommendations for how to promote adoption
We asked external stakeholders for suggestions of the best ways to promote the adoption of Action 3:30 or an after-school programme like it. The most prominent recommendations related to targeting decision-makers in schools with useful information, most notably by highlighting how adopting the programme could help schools to meet their targets:
I suppose again it goes back to what are their objectives because most people are looking for how can they improve what they do and how can they meet their objectives better.
Mentioned in several interviews with school staff and external stakeholders was the value of staff training and continuing professional development (CPD) opportunities. External stakeholders suggested that this might be important for both staff and pupil well-being, especially in the context of promoting adoption of a programme that contains such staff training within it. This was often mentioned in conjunction with helping schools to meet their targets:
I can think of schools that would let their staff out to do that if they could see the benefit, I think a lot of this will be around how you sell it to them in the start to link it in with the benefits that are going to come to the children by doing it I would suggest.
There was also a recommendation to demonstrate the evidence base and readiness to implement the programme at scale:
I think often these things are commissioned at scale, so you get groups of schools, partnerships of schools, who would commission something like this together. So, I think it might be quite important to consider that and show that it’s ready to be purchased by a number of schools at the same time potentially.
Several of the external stakeholders worked within a regional or national health promotion context, and intimated that a strategic route to accessing the decision-makers and sharing the pertinent information with them would be regional county sports partnership and local school sports partnership organisations, as these have a trusted position advising schools in their area. In addition, two external stakeholders suggested addressing calls for evidence.
Summary
More than half of the schools recruited to the study were above the local authority median for percentage free school meals (an indicator of familial socioeconomic position), suggesting that they represented catchments with a diverse range of socioeconomic backgrounds and were, therefore, representative of their region. All schools involved in the project began with the intention to adopt Action 3:30; however, after randomisation, two schools were unable to proceed because of the impracticality of releasing staff for training. In the four schools that delivered the clubs, adoption by TAs was excellent, indicating that TAs are viable delivery agents. Other barriers to adoption at the school level included congested after-school programmes and the cost of implementation. The unique training programme for TAs was viewed as CPD, a positive factor that aligns with many school priorities. External stakeholders suggested that the best way to promote a programme such as Action 3:30 is to demonstrate that the cost of delivery is at least comparable with that of existing provision; provide evidence of the potential impact (towards school goals or targets, on pupils and staff); provide guidance on how the programme could be used to generate that impact; and use existing regional sports partnership networks to provide decision-makers with this information.
Implementation
In the context of this process evaluation, implementation is defined as:
Absolute fidelity as well as factors affecting the fidelity of the delivery of Action 3:30 when compared with intended delivery, both at the individual level (TA adherence to protocols and leader’s manual) and the structural level (school factors, resources, environment).
Intended delivery of the intervention
Two TAs from each of the intervention schools were required to attend 25 hours of training. The training was designed to equip TAs to deliver Action 3:30 after-school clubs in line with the Action 3:30 core philosophy and session plan manual. The Action 3:30 after-school clubs were scheduled to run twice per week for 15 weeks. Sessions were designed to last 60 minutes and were intended to be delivered in the order they were numbered in the leader’s manual. The joint foci of the club were enjoyment and being active. Sessions were designed to promote maximal participation, skill development, co-operation, problem-solving, physical activity and choice. Within the delivery of the club, TAs were trained to promote and foster autonomous versus controlled motivation drawn from SDT and satisfy the three psychological needs, namely autonomy, relatedness and competence. The following sections use evidence from quantitative and qualitative sources to explore, in detail, whether or not these fundamental intervention components were delivered as intended, and discuss the challenges to implementation and potential improvements.
Intervention fidelity
Dose adherence
Adherence was high in the four intervention schools that delivered the programme, with all schools delivering 30 sessions of 60 minutes each. One of these schools had an extended break for the whole of December because of conflicting school scheduling. As a result, its club ran later in the school calendar than the others and had less continuity.
The prescribed frequency and duration of the intervention was acceptable according to TAs in intervention schools:
From a leader’s perspective for us it was fine. We knew it was going to run for that long and we were fine with it. It didn’t bother us at all.
Content adherence
To capture adherence, TAs were asked to complete a log book of delivery (see Appendix 3). Of all the sessions included in the leader’s manual for TAs to deliver, TAs reported that 25% (n = 30 between the four schools) were delivered fully, most sessions were delivered partially (n = 83 between the four schools, 69.2%) and 5.8% of session plans were not delivered at all (four in one school, three in another). The reason given for non-delivery of sessions was a temporary space limitation at the school, which meant that the TAs chose activities from previous sessions that could be better delivered in line with the constraints of space on that day. This suggests that TAs felt suitably equipped to adapt sessions based on local needs, which was a key focus of the training programme.
To more robustly evaluate fidelity to the intervention content, trained researchers attended and observed three club sessions at each intervention school. All of the sessions that were observed were scored as either fully (n = 8, 67%) or partially (n = 4, 33%) delivered (see Appendix 5), the reason for partial delivery being a lack of time to fit in all of the activities.
Teaching assistants in one school felt that the format of the sessions was too similar to that of PE, and that children would rather move on to new activities than return to the same game after developing game-specific skills:
They didn’t really enjoy stopping to do a little skill development, like one-to-one sort of games, like partnered games . . . Maybe because it felt too much like a PE lesson.
Potentially as a result, TAs reported using the session plans as a guide rather than a script as their teaching skills developed and they became more confident in delivery:
We used it, initially, as the bible, so we went through it and it worked for us for that because there was just way too much content per session so then we took it as a starting point, didn’t we?
Factors affecting implementation fidelity
Training
The TA training was implemented as intended. Attendance at training was 100% for eight TAs and 80% for one TA. The timing of the sessions meant that there were no conflicts of schedules, and all TAs were able to stay for the full duration of each of the training days.
All of the training sessions were delivered fully, according to the lead trainer’s log book. This was corroborated by the accounts of project staff who supported the delivery of the training days. Although TAs had a wide variety of experience levels, their engagement during the training was excellent according to the lead trainer who, in a post-training interview with project researchers, stated that they were one of the most engaged groups he had taught. He attributed this to well-explained course aims and a clear translation of Action 3:30 into relevant practice for the TAs: further evidence of training fidelity. When asked whether or not any content was missing from the training to equip club leaders with the knowledge and skills to deliver the intervention, he suggested that it was comprehensive. No barriers to implementation of the training were stated by either the TAs or the lead trainer:
I personally think you would struggle to add anything that’s going to add value to it.
Resources
As described in Chapter 2, The Action 3:30 intervention, in addition to the leader’s manual (containing the 30 session plans), TAs were supplied with a training guide and £200 was provided to each intervention school to buy any equipment they needed to deliver Action 3:30. Based on recommendations from the original Action 3:30 feasibility trial,18 an online video archive was also created. It is unclear whether or not or how TAs used the training guide after the training. One TA suggested that it was useful but did not expand on how. One TA expressed that they would have liked to have used the training guide more during the training, as that would have helped to embed the training concepts. TAs focused on the session plans as their main resource. Session plans, although sometimes content-heavy for the time allotted, were valued by the TAs because they were pre-prepared:
. . . the fact they were even there, made it more enjoyable . . . you can rock up and you know, know there’s something you’re going to be able to do and it’ll be fine and they’re going to enjoy it, rather than it being the pressure on to rock up and think, ‘What are we going to do today then?’
Because the session plans were written, a limitation of them was that some of the activities were difficult to visualise:
I think some of the games were like, ‘I don’t get that,’ and I had to really break it down. I used to get a piece of paper and was like okay, spots, and drew spots, and then it was like hoops, right hoops and so we were almost teaching ourselves the game.
The online archive containing demonstration videos of 22 of the session activities was used by TAs from three out of the four schools that delivered the club. However, tracking statistics reveal that videos were accessed more frequently towards the beginning of the delivery period. Videos corresponding with sessions that appeared early in the delivery manual (sessions 1–6) were accessed, on average, 23 times each by a range of TAs. Access numbers fell as the intervention continued, and the videos for sessions 13–30 were accessed only four times each on average, and by only two people. TAs stated that they found the videos most useful if the activity in the session plan was unfamiliar to them, but pointed out that many of the games learned in the training programme were those covered in the videos, which was not as helpful. Others expressed that because the videos showed only six to eight children performing an activity, they lacked relevance for conducting the activity among larger groups. However, TAs in one school perceived that they usually had insufficient time to watch the videos before sessions anyway.
The £200 provided to intervention schools to buy sports equipment needed to deliver the club was crucial to delivery, according to TAs:
That was extremely helpful, and it was needed and we wouldn’t have been able to do some of the sessions without it.
School support
School-level support within the intervention schools that delivered Action 3:30 varied. The TAs and key contacts perceived the school’s main supporting role as one of logistics. However, differing levels of engagement from the school administration staff contributed to very different experiences of delivery for TAs. In two schools, TAs felt that their school prioritised the club and facilitated delivery and attendance:
We are lucky because there was one occasion where they wanted the hall space and we just went and said, ‘Actually, do you remember we’ve got this?’ And they went, ‘OK, we’ll go there’. So, nobody turned around and said, ‘You can’t have the hall today’. They knew on a Tuesday and Wednesday that was our hall space.
Teaching assistants from the other two schools described administrations with less interest and poorer communication of key information, which contributed to them feeling less supported:
No one really took much notice, to be honest with you. In actual fact, one session we had to go from the Key Stage 2 hall to Key Stage 1 because they organised with some life team or something to be in school and they never put any thought into that actually we were going to be using the hall and those sorts of things.
However, the TAs at one school went on to state that the school had shown support in other ways, and suggested that they would receive support if they asked for it. The key contact at the same school expressed that the autonomy demonstrated by the TAs running the club had been noted and was valued:
They sort of just run with it. They went on their training. I checked in with them. They haven’t ever come to me and asked me anything, which has been fab because I’m really busy with my PE as well.
School policies and the physical environment (school context) could have affected the level of school support provided to Action 3:30 clubs in the intervention schools. The results of the school context assessments in both intervention and control schools are presented in Chapter 4.
Quality of delivery
Space and season
The most frequently cited factors that affected the delivery of Action 3:30 sessions were weather and space. Space, in particular, was cited as a main barrier to delivery fidelity, to the extent that some children reported safety concerns about playing in a limited space containing obstacles:
I think our biggest barrier for any club is always our space, but you just deliver whatever works for you.
I hurt myself like loads of times on chairs.
Weather was closely linked to the space issue, as running Action 3:30 during winter months often meant that the club had to be conducted indoors, where there was less space. Conversely, if the inside space was not available, then the club would need to be run outdoors in adverse weather, which could also have affected the quality of the session. Taking into consideration the season in which Action 3:30 is delivered may address some of these issues in schools that run the club:
. . . and it’s been through the winter as well so it’s not even like you could – we did go outside as much as we could. You just get on with it, don’t you? You match your games to suit.
Delivery style
Children’s feedback characterised the delivery style of the club leaders as clear, fair, encouraging and supportive:
They’re good teachers, they’re really supportive and they always want to hear anything you say, they’ll listen.
Children also highlighted that TAs adapted games in the sessions to make them more fun, as well as enabling the children to dictate the adaptations, giving them a sense of autonomy:
I liked the games that they made. I liked that they tried to make different rules and they tried to make it different in order to make it more fun.
Social factors
Other factors that may have affected the quality of delivery were identified in interviews and focus groups. TAs in two schools found that friendship issues, especially between girls and between siblings, seeped into the club and affected the dynamic of the session.
Knowing the children before the club started was also raised as an important factor that could affect club dynamics and, therefore, delivery:
We’re privileged because we know our children really well but in other schools they might not necessarily know their children quite so well and that could have quite an impact on how the club runs but it is just something you can’t foresee.
Teaching assistants made little mention of disruptive behaviour adversely affecting delivery, except in one instance:
I think that the problem with that is it was the behaviour was actually what we expected from that particular – there were a couple of children that we knew would be that way, so we knew it would happen. I just think at that particular point that one particular day it was just really, really bad.
Children expressed that disruptive behaviour did sometimes adversely affect their enjoyment, but that TAs dealt with it well overall. Children reported the use of a warning system, which was suggested in the training guide as a way to manage disruptive behaviour. This again provides evidence of good intervention fidelity:
Very well. Sometimes, they made them go next to them or they said, ‘You’re not playing for 5 minutes’ which is good because some people were shouting, so they told them to quiet down. They had three warnings.
The key contact in one school mentioned that a group of children with mixed levels of ability adversely affected the TAs’ ability to deliver a high-quality club:
One of the main things is probably the difference is the ability of the group that’s difficult to manage . . .
However, mixed-ability groups were not seen as a barrier by the TAs themselves as they used peer-to-peer learning, and this view was supported by the key contact from another school:
If they’re all struggling, then you wouldn’t have been able to manage it, would you, you need some that are a bit more able cause actually they can show other children . . .
Sometimes there can be quite a mental block for children if they find it difficult or it’s very competitive, so it’s quite nice to have those children that are already keen to spur those other children on.
Potential improvements
Teaching assistants, key contacts and children raised certain recommendations of ways in which the implementation of the programme could have been further optimised. The main improvements suggested were to the session plans, for example adding diagrams to make them easier to understand and visualise:
Sometimes when you read it it’s hard to set it out, to visualise it. So, if you had a little picture of how it needs to look.
Additionally, TAs and children alike would have liked to spend more time outside, a factor linked to enjoyment, space and safety, all of which are essential to the fidelity of delivery of the planned Action 3:30 sessions. These factors are addressed in more detail in Maintenance.
Summary
Overall, TA training was implemented as intended, with few suggested improvements. Intervention adherence was consistent across schools and acceptable. Among the resources provided, the leader’s manual and money for sports equipment were used extensively by all TAs to facilitate consistent delivery of the intervention. The training guide that accompanied the training and the video archive were used to a lesser extent, although all TAs intimated that these were reassuring resources to have available. Limitations of the resources included confusing descriptions of some activities in the session plans, and a lack of videos for activities that were not covered in the training. The training was deemed comprehensive and supported high adherence to content. Adherence was primarily affected by resource utility and environmental factors. The overall fidelity of implementation of Action 3:30 core principles was high, despite different TA experiences.
Maintenance
Within the context of this process evaluation, maintenance is defined as:
Factors perceived to affect potential maintenance of Action 3:30.
This section will present evidence of maintenance in Action 3:30 intervention schools, highlight the factors which may affect maintenance and propose potential improvements that may be useful to consider for future development and implementation of physical activity after-school programmes.
Evidence of maintenance
One intervention school decided to continue to run Action 3:30 post study. Both the key contact and the TAs expressed that they would be running Action 3:30 twice per week for the rest of the current term and the following academic year, and that they would be making minimal changes to the way that the club was run:
We might change the age range but no, other than that, I think we will just carry on.
Key contacts from the other three intervention schools also expressed an interest in continuing the club, and the other TAs voiced a desire to run the club again if given the chance.
The TAs believed that the intervention resources provided ample activities and ideas to reuse, adapt and continue with. They also felt that the autonomy-supportive teaching style, adopted because of the training, was an important factor in the maintenance of the club. All TAs said that they would continue to include child-led activities if they were to continue, and one school was particularly interested in tailoring the club to the needs of the new intake of pupils and ensuring that there was a child-led focus:
. . . it’s really great to be child-led I think . . . So, I think that whatever works for the children, they probably will have different children so it’s probably good to try out those sessions first of all, see what the children like and then run with what they like.
Both the TAs and the key contact in school 27 discussed the possibility of delivering the club to younger pupils on one of the club days; they felt that the content was easily adaptable and felt that Key Stage 1 pupils have little opportunity to get involved with extracurricular activities:
. . . we’ll run it slightly differently as we won’t have two sessions for a week. We’ll have one Key Stage 1 and one for Key Stage 2 . . . we would like to get Key Stage 1 more active because we haven’t got any provision at the moment for that.
This was also echoed by other TAs, who thought that Action 3:30 would be well suited to a wide range of year groups and that the resources and delivery style were flexible enough to suit younger children who may not have the same opportunities to participate in active clubs:
I think there were some children that I personally work with who would benefit a lot from it seeing how much other children have got out of it. And I think it’s suitable for all of primary school.
. . . if you ran across Year 2 and Year 3 they’re probably the age groups which don’t really have that much opportunity to participate.
External stakeholders were also of the opinion that flexibility of delivery was an important factor in the sustainability of physical activity programmes and that being able to target different age groups would be beneficial for schools:
Perhaps schools can use it as a resource to then target more throughout the year. So, they might have different groups accessing it at different times in the year.
Interviews revealed that one school had decided to continue a very similar version of the Action 3:30 club, and that TAs and key contacts agreed that the resources and training provided enough of a foundation to continue Action 3:30 without the support of the study team. TAs, key contacts and external stakeholders all agreed that Action 3:30 was a flexible programme that would be suitable for a range of year groups. Together, this suggests that Action 3:30 may be a sustainable programme that could be maintained within schools for the foreseeable future. However, several barriers were raised that could prevent schools from maintaining the delivery of the club.
Factors affecting maintenance
Funding
The main cited barrier to maintaining Action 3:30 post study in the intervention schools was funding. Delivery was funded by the Action 3:30 research project for this study; however, going forward schools had to be able to identify funding (whether core budget, Sports Premium Fund or other external funds). All intervention school key contacts mentioned that they would have to find funding to cover staff costs, as TAs would be unlikely to be willing to deliver Action 3:30 on a voluntary basis, a view reinforced by TAs, as only one said that they would deliver voluntarily. The viability of paying TAs for delivery twice per week was raised and one school suggested that parents would need to be charged for delivery. Three schools mentioned the government’s PE Premium as a way of funding the club.20 The key contact in school 27 said that they would use this funding to continue Action 3:30. The key contact in school 33 advocated that this funding could be used if they were to continue, but that delivery would be difficult to sustain once the funding runs out in 2020:
I think the only way that would happen would be if we started charging the children, because the school doesn’t have any funds to fund teaching assistants to do clubs.
. . . at the moment we have this Sport Premium funding from the government and so we’ve got quite a lot of money to be able to spend. It’s something that after 2020 we wouldn’t be able to fund.
One external stakeholder, who was also a primary school PE lead, reduced the sustainability potential of after-school clubs to two components, which were cost and pupil enjoyment:
I guess from my school it basically comes down to cost and pupil enjoyment.
When probed further, this stakeholder highlighted that an important factor for schools and parents alike was running costs being comparable with those of other activities on offer:
Well if they’re charging the parents, then it needs to be a competitive rate in line with all the other things that we’ve got on offer currently.
Equipment
As part of the study, intervention schools were provided with £200 to buy the required equipment. When asked about this, TAs expressed that the money had been crucial to delivering some of the session plans, despite the equipment list detailing only equipment that is typically found in schools. This suggests that if the programme were to be adopted and maintained in other schools, a lack of money to buy sufficient equipment may be a barrier in some:
That was extremely helpful, and it was needed, and we wouldn’t have been able to do some of the sessions without it.
Club days
The Action 3:30 intervention was delivered twice per week. Some of the TAs, key contacts and external stakeholders suggested that schools may find it more sustainable to deliver the club once per week, as a result of lack of funding, scheduling around other clubs, burden on TAs and commitment for parents. Nevertheless, TAs and the key contact from school 27 stated that they planned to continue to deliver the club on 2 days, which suggests that the intervention can be sustainable in its current format depending on school-specific issues:
That’s the only thing, really, just the once a week instead of twice a week.
I suppose that works with your staffing, doesn’t it? And with the clubs you’ve got running in the school.
If it’s too frequent it’s quite a big commitment so sometimes numbers drop off because of that.
Training
Training is a key aspect of the maintenance of Action 3:30. TAs were provided with the skills, expertise and qualification required to deliver a physical activity club, which they can continue to use and/or develop. The qualitative findings show that school staff valued the need for CPD and perceived Action 3:30 as a good chance to upskill their TAs for the future:
We valued them going out and doing the training.
Many of the external stakeholders also voiced that schools are always interested in developing their existing staff, and so the Action 3:30 training model should be very appealing to schools because it helps them to achieve sustainability:
They would have a resource in place in the school that they would be able to use subsequently so I think using existing staff is a really good model.
The primary schools are being challenged around the sustainability of how they use that funding. So, great focus is given, and encouragement is given around the agenda of upskilling their existing school workforce.
External stakeholders described the TAs as a valuable resource to the school, which is good for sustainability. However, they also raised that if the TAs were to leave or change role, the Action 3:30 programme would be difficult to maintain if continued training was not provided for new staff. It was suggested that an approach could be taken whereby the existing TAs are also trained to be able to train new staff and therefore transfer their knowledge and skills. This would then increase the likelihood of the programme being maintainable:
So often just training two people is not sufficient, you need to maybe train four and keep them involved on a rotating basis in the programme if possible. So, you keep the skills up, but you also keep a number of people involved and whether there’s some sort of train the trainer so those who are already there, if somebody new starts after the funding runs out, they have got a model to take it forward.
Overall, the training model used in Action 3:30 is valuable to schools and is sustainable if the same TAs continue to use the training.
Targeting children
Targeting children to attend the club was raised by different stakeholder groups. As described in Reach, some key contacts initially thought that they would be able to target certain children at the recruitment phase. Key contacts and TAs said that, if the club were to continue, they would like to first advertise the club to certain children who they felt were less active and did not participate in other physical activity after-school clubs, before opening the club to others:
Probably what we’ll do is we’ll target and then we’ll open it up to everyone.
Think there are certain things that we would tweak and change. It would be something that we could potentially run again . . . but we would target specific children from specific groups.
External stakeholders agreed that schools would probably want to target which children they deliver the club to or extend the club’s reach to other pupils and agreed that ensuring the programme is flexible enough to cater to schools’ needs is important for its sustainability:
You can just adjust it dependent on the children you’ve got in front of you. I think targeting and selecting, sympathetically selecting children could be developed a little bit more because we’ve always been a bit uncomfortable with the idea of just saying, you know, these are the kids that aren’t doing anything.
Teaching assistants also mentioned that they would prefer a smaller group if they were to run Action 3:30 again and so would reduce the recruitment target number to between 15 and 20, as some felt that their group was too big for their facilities:
I think they’re thinking we will do it for about 20 rather than 30 when we do it for the school.
Selecting the number and type of children seems an important consideration for the maintenance of Action 3:30 in schools. Using Action 3:30 to target children appears to be driven by school priorities around providing physical activity opportunities for those seen to be doing less or to have fewer opportunities. Maintenance of Action 3:30 is therefore contingent on its utility in meeting school targets.
Club focus
Although not raised as a barrier, the focus of Action 3:30 sessions was something that TAs and key contacts suggested would be tweaked to suit the children in the group to make the club sustainable. TAs and key contacts liked that Action 3:30 was centred around enjoyment, being active and having child-led activities, and suggested that they would like to focus more on these aspects of the programme rather than skill development:
Our focus is enjoyment so . . . we are not there to teach them cricket or to teach them . . . but we are there for them to have fun and learn some more social skills of being active and enjoy being active differently.
I know it needs linked to skills but, but it would have been, you know sometimes I felt it was more about just enjoying it rather than trying to build skills, teach skills, cause that’s what their PE lesson is more for.
External stakeholders also agreed that the programme should be based on the children’s needs, so schools may want to adapt the intervention resources to achieve a slightly different outcome or to better cater for the targeted group of children:
With the children that probably will be targeted for this, which are the less sporty, less active, less confident children . . . there may be ways of developing what’s actually offered to them, so they could increase the amount of activities and sports covered within those sessions or make it a little bit more child-centred in terms of the contents.
Evolving content
Several external stakeholders asserted that ensuring that programme content is regularly refreshed is essential to the sustainability of any successful after-school programme, and thus may be an important factor to consider for the maintenance of Action 3:30:
As soon as you start to take away the elements particularly around investing in people – so your focus on teaching assistants – and continuing to keep the content fresh and exciting for children, you start to see a rapid decline in how clubs are embedded and how programmes are embedded and sustained.
Potential improvements
Potential improvements were discussed during interviews with TAs, key contacts and external stakeholders and during focus groups with pupils. Several themes emerged from the qualitative data, which were categorised into space and season, parent engagement and peer leaders.
Season and space
Both TAs and pupils stated that they would have preferred the intervention to have run through the spring/summer term, so that they could have used the outside facilities more and had more space for the big group games:
I think it would also be really nice to do, instead of in the winter months, do it [in] spring/summer months . . . I wondered maybe even if more children would attend those months.
To improve benchball, you could have a bigger area. We could take benches outside.
Space and season were also mentioned as barriers to implementation. Therefore, for maintenance, considering the season in which the club runs seems prudent.
Parent engagement
Parent engagement has been raised previously as a likely barrier to both recruitment and attendance of Action 3:30. Despite key contacts and TAs stating this as a potential barrier, they did not highlight many ways to increase engagement, except for trying to have face-to-face conversations with parents.
External stakeholders suggested that involving parents in the delivery of the club, potentially as volunteers, may be a way to increase parental engagement. However, this may not be a viable approach for schools and is likely to engage only the more proactive parents.
Although parental engagement may be a potential barrier to the sustainability of Action 3:30, the findings from this process evaluation do not suggest many ways to improve this. Future physical activity interventions aimed at primary school children should aim to better understand how to engage parents.
Creating peer leaders
One external stakeholder suggested involving children who had already participated in Action 3:30 in the continuation of the programme as a potential improvement. The external stakeholder proposed that letting children help with the delivery of the club and giving children ownership is an effective way to achieve sustainability:
The tipping point for us, or the landmark point for us, was when we gave ownership of the delivery of the clubs to children . . . Year 6 pupils who had ownership of the club, the delivery of the club, and were able to tinker with the activities and the way in which they were delivered and provided almost like a buddy relationship with the children that were involved and at that point the clubs really exploded, not just in terms of the amount that were delivered but, you know, even consistently now there’s still over eight and a half thousand clubs running and they are largely due to the fact that they are delivered by children for children.
This was also suggested by the key contact as a strategy that school 27 (which is continuing Action 3:30) may consider adopting:
Maybe we might even make some leaders because we’ve done that before, so some of the children who are doing it now could then go and help.
Summary
Evidence of maintenance was seen in this iteration of Action 3:30, as one school decided to continue running the club and the other intervention schools also expressed an interest in doing so. Interviews with key stakeholders suggested that there are several factors affecting the potential maintenance of Action 3:30 as an after-school programme in other schools. These included delivery costs comparable with those of existing provision, funding for delivery and equipment, continued TA training, the number of weekly sessions, and the versatility of the club to enable targeting while keeping children engaged by evolving the content.
Trial design and evaluation
In addition to the RE-AIM analyses, a number of key trial design issues arose from the qualitative work. These are discussed below under the headings of Recruitment, School burden, Re-enrolment and Potential contamination.
Recruitment
In general, the pupils were positive about the recruitment method. Pupils across the intervention schools stated that the study was clearly explained to them by the project staff and that they had the opportunity to ask questions:
I think it was clear . . . I understood all of it.
Table 8 shows that when comparing the two recruitment methods (briefing only vs. briefing plus taster) there was no difference in the number of pupils who signed up. However, qualitative data show that pupils who received a taster session thought that this was a fun experience, which aided their decision to join Action 3:30. In addition, in one school that received only a briefing, TAs suggested providing a taster session for the pupils, which parents could also attend, as a potential improvement to the recruitment method. Therefore, although the quantitative data show no difference in recruitment rate by recruitment method, taster sessions generated enthusiasm for taking part in Action 3:30 or were suggested as a way to do so. Key contacts agreed that advertising wearing the accelerometers as part of the project was a key interest for the pupils and one of the reasons they decided to sign up.
School burden
Key contacts in both the intervention and the control schools alluded to the increased administrative burden during recruitment and data collection visits. Key contacts mentioned that arranging suitable times for project staff to visit, sending out paperwork and chasing the return of accelerometers and questionnaires were the main jobs that increased their workload. However, most stated that they did not find this particularly burdensome and that the workload was manageable. This was also the case for key contacts and school teachers in opt-out schools, where burden may have been higher because they had to organise data collection with all pupils from two year groups. Three out of four key contacts from opt-out schools said that project staff gave advanced notice, communicated well and provided good resources during the project, which helped to mitigate the burden. School staff acknowledged that the opt-out consent process was an effective way of collecting data from two year groups and lowering the burden on parents. All key contacts agreed that the thank-you gifts provided during the opt-out phase helped to create interest for the study as well as encourage the return of accelerometers. Although it was helpful to collect qualitative data from school staff about the opt-out process, it should be noted that as a result of changes resulting from the introduction of General Data Protection Regulations legislation, an opt-out consent process will no longer be permitted in research with primary school age children in Europe. Therefore, the viability of this method is very limited. However, this is not the case for countries outside Europe and so this evidence could help to inform other similar school-based studies. In addition, TAs expressed that they had received enough support from project staff throughout the project, on training days and during the club, which also helped to mitigate burden.
Re-enrolment
A re-enrolment phase was conducted after approximately the 12th session of the intervention in all four of the intervention schools. Table 15 refers to the number of children who were recruited at the re-enrolment phase in each school. As the table shows, re-enrolment had mixed efficacy at backfilling the available spaces in the club as intended. The re-enrolment process appeared to be similar to the school norm in two schools. In addition, key contacts and TAs from the other two schools understood the value of the process. In three out of the four schools, TAs expressed that the additional pupils positively influenced the existing group of children. Generally, TAs and key contacts did not feel that the process added to their workload too much and were happy to distribute the resources. However, the key contact and TAs from one school suggested that the resources could have been editable to reduce burden. Overall, the data suggest that the re-enrolment phase was not overly burdensome to schools and increased external validity.
Potential contamination
There was no evidence of potential contamination in this study. None of the key contacts and TAs was aware of any other schools in the local area that may have adopted an after-school programme similar to Action 3:30 because of seeing or hearing about their school running Action 3:30.