Summary of findings
The aim of the ICALM study was to inform a future trial of the effectiveness and cost effectiveness of IPC-A by answering: (1) Are trial procedures, including recruitment (of participants and therapists), randomisation, research assessments and follow-up, feasible and acceptable? (2) Are IPC-A and TAU delivered and how and why does intervention delivery vary across differing service contexts? (3) To what extent does contamination of the control arm occur and should it be mitigated against in a future trial? (4) Does the interval estimate of benefit of IPC over TAU in depression scores post treatment include a clinically significant effect?1
The aim was to recruit 60 participants for the study. Thirteen different sites participated, and nine SPQs were received. Thirty-two participant referrals were received, of which 16 participants (26.7% of target) were recruited, which is less than the progression criteria of 80% of target. Due to factors related to COVID-19, appointment attendance, increased risk and complexity following baseline assessment and progression of symptoms, only seven participants received an intervention. Three participants received the IPC-A intervention, while four received TAU. Fifty per cent of participants attended at least three therapy sessions within the 10-week treatment window, which was less than the progression criteria target of 70%. Of those that received an intervention, 100% of follow-up assessments were completed, which was above the progression criteria target of 80% at 10 weeks and 70% at 23 weeks. Twelve participants completed the RCADS depression score at the 10-week follow-up. No clinically significant difference was found between the two interventions, meaning the progression criteria were not met; however, this result is likely to have been impacted by three participants not receiving the IPC-A intervention as intended. Eight interviews and one focus group were completed with stakeholders. Sixteen interviews were completed with participants; eight of these were with participants in the intervention arm and eight in the TAU arm. A further eight interviews were completed with YPWs: one IPC-A supervisor, six IPC-A therapists and one TAU therapist. Four participants consented to their recordings being analysed for the process evaluation. One hundred per cent of reviewed treatment sessions met treatment fidelity criteria, which was above the progression criteria target of 80%, and there was no evidence of contamination effects. The feasibility trial was suspended in the first couple of months, for 6 months, due to the COVID-19 pandemic. An extension was applied, and recruitment closed at the end of the agreed recruitment window. The study was not able to randomise the target number of participants; therefore, the data provided were insufficient to answer key research questions regarding the feasibility and acceptability of the intervention.
However, the feasibility of the RCT was accompanied by a mixed-methods ethnographic process evaluation to generate an understanding of intervention implementation across services and explore the acceptability of the intervention from the perspective of young people and other key stakeholders. Following difficulties with recruitment to the main feasibility trial, the funder advised that the process evaluation could refocus on understanding the barriers of running a clinical trial in the various community services that make up CYP’s mental health landscape. We have provided detailed insight into these barriers, with implications for other researchers conducting future research in mental health and community services.
The findings on recruitment, retention, fidelity and contamination are summarised in Chapter 5, Progression Criteria, in relation to the criteria for progression to a definitive RCT. As mentioned above, we assessed the feasibility and acceptability of trial procedures, how intervention delivery varied across the services, how contamination can be mitigated against and whether there is a clinically significant benefit of IPC over TAU.
Results of the feasibility study
Recruitment of participants
Due to low recruitment, potential suitable cases being stuck on a ‘green’ list and services not always having TAU or IPC-A capacity, new pathways were set up which appeared to fit with the changing environment. The new pathways were provided so that services could work together to identify and treat individuals depending on their randomisation to the appropriate service. A fast-track service was also set up from Site-10_Central referral point so that those under a RAG rating of ‘green’ that were potentially suitable could be referred to the ICALM study. The fast track helped with the fact that services were reporting increased complex referrals that were not suitable for ICALM. Although the changes to referral routes were not successful, the different services showed great flexibility and went above expectations to help the study.
As discussed in the process evaluation, due to challenges faced by services in recruiting suitable young people and delivering IPC-A, it was felt by Early Help managers and practitioners that IPC-A may be more suitable for U18 well-being services and school nurses, as they were overtly set up to treat mental health difficulties. The well-being and school services are also normally the first points of call to refer CYP experiencing mental health difficulties.
To encourage recruitment, teams were willing for ICALM to attend allocation, triage and team meetings. This allowed questions to be answered directly, for ICALM to understand referral difficulties, help identify possible referrals and build working relationships across the network. Teams were mostly happy to test alternative procedures and showed a willingness to be involved in the study.
It was also noted that half of participants came from services within NHS Trusts and not Early Help teams. A possible reason for this is due to experience in research studies or a culture of research and evidence-based practice within traditional health services, leading to more confidence and also knowledge/education around the randomisation process. As was suggested by Early Help managers, NHS Trusts are more likely to be set up to identify and screen for mental health, possibly leading to more referrals.
COVID-19 and the recurrent lockdowns impacted clinical presentations of low mood. An increase in presentation of anxiety cases amongst services was noticed with the start and progression of the pandemic. Despite comorbidities between anxiety and low mood, young people expressed greater preference in having their anxiety rather than low mood treated. Since ICALM’s core criterion was to provide IPC-A to young people presenting with low mood, many young people were screened ineligible and were not further referred from services, influencing the number of referrals received in ICALM. Receiving mild to moderate cases of depression is what the study demanded, but such cases appeared to either no longer exist or did not come into the services or to their attention. Ultimately, the study was unable to recruit the required number of participants despite repeated mitigation measures being put in place.
Recruitment of therapists
A high turnover of staff in the services meant that the majority of staff who had received training to deliver IPC-A either left the services or moved on to other roles. By the end of the study, only half of the teams had active therapists who were attending supervision sessions. This reduction meant that regardless of the number of participants coming through, capacity from existing therapists and sites was limited. The previous single-arm study (pre COVID-19) had a high retention rate, and the calculation of the number of trained staff required was based on this study.
A lack of staff trained to deliver IPC-A added pressure to the remaining IPC-A practitioners. This especially affected services delivering targeted services (i.e. services with mental health support as their core offer), as only a small proportion of their staff members do not have a prior professional mental health qualification (CWP qualification). As the study progressed and teams had a reduced number of IPC-A-trained staff, further training sessions were offered, but after the second training offer, no further staff members expressed an interest nor were any put forward by their managers. Future studies may wish to consider funding staff to be trained as IPC-A therapists and deliver the therapy as a core offer, as happened in the initial single-arm pilot.
A major barrier to this trial’s working was existing staff being unable to deliver the trial intervention for multiple reasons. A future study would be more likely to be successful if it also funded staff time for study treatments, training and supervision. We accept this is not normally done for research in NHS services, as such treatment is often considered ‘core business’. However, we have demonstrated the additional barriers to conducting RCTs in tier 1/2 services, which may justify this extra funding.
Randomisation
Initially, staff participants struggled with the concept of randomisation, voicing that they felt it was unfair to refer a young person for the study, knowing that they may not receive IPC-A. This was particularly noticeable within social care settings. One team manager stated that he would not be putting any young person forward as he thought randomisation was unethical. Changes to training were made to include more information on randomisation and less of a focus on IPC-A; however, randomisation continued to be a barrier throughout the study. Once identified as suitable for IPC-A, clinical staff struggled to accept that their standard offer would be of benefit.
Participants in the study expressed concern mostly around the randomisation of the study and getting the help they needed. Some expressed disappointment when randomised to the TAU arm of the study, while one participant was concerned about being allocated to IPC-A as it was an untested intervention without evidence of effectiveness.
Using a cluster randomised trial (CRT) could potentially offer solutions to some of the above-mentioned barriers. In a CRT, there would be dedicated sites for either TAU or IPC-A, meaning all people within a service would receive the same intervention, potentially avoiding staff’s ethical concerns about service users receiving different treatments within the same service. A CRT would also reduce the chance of contamination due to all those within a cluster being treated similarly.59
Research assessments
The proposed outcome measures appeared to be acceptable to young people. Most measures, researcher-led and self-completed, were completed, and young people did not appear to find the measures to be a burden. One participant found the assessments beneficial as they enabled them to see how their mental health had improved since the start of the study. COVID-19 restrictions led to some complexity in completing measures, which was eventually overcome by incorporating new technological solutions such as video consent procedures.
Acceptability of the intervention
It was reported by a couple of participants that IPC-A focused too much on the positive aspects of their relationships. These young people stated that they would have preferred a greater focus on the more difficult elements and were able to understand these before moving on to positive aspects.
Some parents were not aware of the contents/discussions of the sessions. However, when one parent was included at the request of the young person, the parent had reported this as being helpful to be able to learn how to support the young person at home.
In interviews, IPC-A was acceptable to both young people and parents, with IPC-A being seen as offering positive benefits for young people. As reported in the process evaluation, young people reported being more open to family, socialising with friends and being able to engage in ordinary activities such as shopping. Parents also reported improvements in young people’s well-being after the second or third session.
As reported in the process evaluation, young people reported having a positive experience during sessions and felt that the structure and duration of the sessions were important for providing stability. They felt that the sessions provided a private space which allowed them to open up about the challenges they were facing. They felt that the developing of a positive relationship was an important factor in facilitating discussions, as well as being focused on the positive rather than negative aspects of their lives which helped contribute to the improvement of their symptoms and have a better outlook. Overall, IPC-A as an intervention appeared to be acceptable.
Quantitative analysis
This study was not powered to detect any significant changes in outcomes, and from the data collected there were no clinically significant differences between the two interventions. Half of the participants who were randomised to IPC-A and completed 23-week measures did not receive the intervention; therefore, this would have had an impact on this finding. The mean changes in the outcome measures suggest they are sensitive to change. Therefore, if the intervention was effective, this would be able to detect differences in a future trial.
Health economic analysis
The health economic measures appeared to perform well and were completed. The CHU-9D had a high rate of completion and showed a possible improvement over time. The responses were also spread over a majority of levels. This suggests that there is scope for the CHU-9D to detect differences over time and that there is scope for improvements in health to reflect in the CHU-9D. Similarly, for the CSRI, the majority of fields appeared to have been completed for most cases, showing reduced healthcare use over time. It should be noted, however, that most of the health care was not mental health-related. Due to the CSRI performing well, it would be a potential candidate for use in a future trial of ICALM; however, it would likely benefit from simplification. Owing to the very small number of individuals who had the IPC-A intervention, it was difficult to estimate what the likely cost of the intervention would be. Of particular interest would be turnover of staff. A very high turnover rate, as seen in this study, means that many staff are likely to need to be trained to provide intervention sessions, and much of the benefits of this training may be lost if staff move on to different roles.
Results of the process evaluation
The objectives of the process evaluation were primarily to provide a description of how IPC-A and TAU were delivered and observe how delivery is shaped by the context of differing service models. Few RCTs have been carried out in tier 1/2 mental health services, despite the need to develop evidence-based interventions. The process evaluation findings share some insights into key barriers to conducting research (in particular RCTs) in this setting, while highlighting the current challenges faced by the services in supporting young people experiencing mental health difficulties.
Workforce capacity building strategies and research culture
Workforce capacity challenges were a key recruitment barrier in the ICALM study, as the increased pressure at central referral points led to the prioritisation of urgent cases and consequently long waiting times for ‘green cases’ (e.g. low mood) and low referral rates into services delivering IPC-A. Process evaluation findings also indicate a lack of research culture and research capacity amongst the participating teams, which cannot be isolated from the high demand faced by the services. The findings indicate that some services did not view research as a priority, considering it as something ‘extra’ that could be left out to ease pressure as demand increased, particularly during the COVID-19 pandemic. The IPC-A training sessions and meetings with teams also unveiled a lack of familiarity with research processes, particularly randomisation by the teams, with some teams expressing reluctance to randomise young people.
The low participation in research, fuelled by the need to manage high demand and the lack of processes and frameworks for identifying and implementing evidence-based interventions in this setting, means there is a lack of evidence-based guidance with respect to what should be offered to whom, resulting in local interpretations of best practice. This is especially true for clinical presentations such as low mood, whereby various interventions (e.g. family interventions, behavioural activation, counselling or I-PCA) might equally be used to treat a clinical presentation. This narrow approach of intervention provision which develops naturalistically within this context poses a challenge for local commissioners to establish processes and frameworks for identifying and implementing evidence-based interventions. Such an approach was also to allow individual services to decide whom they treat and how in isolation as demand increased and overwhelmed them. This meant that clarity of who should get what treatment was further disrupted.
There is, therefore, a requirement to develop effective unifying workforce capacity-building strategies, considering areas of greatest need and the professional requirements, training and supervision which will address these needs. To ensure continuity of service, there is also a requirement to develop and implement workforce retention strategies that could incentivise staff to remain in specific posts. As part of wider workforce capacity-building strategy, developing a culture which recognises the value of developing, evaluating and implementing evidence-based interventions will facilitate future research. More practically, teams could be introduced to research processes gradually, through training to increase understanding of specific processes such as randomisation and by initially involving them with lower-level research, for example, single-arm pilot studies, before and after studies, before exposing teams to large RCTs. There is also a need for commissioners and service leads to develop a framework for identifying and delivering evidence-based interventions that could be incorporated within service models, workforce development and supervision. Building such a research culture and incorporating appropriate frameworks may increase success of recruitment and implementation of RCTs in this setting.
Training and review of core service offers
The ICALM study sought to train teams of young people’s workers to deliver IPC-A within tier 1/2 services. While these workers might support CYP with mental health presentations, they do not typically deliver specific mental health interventions. This task-shifting approach, which has had successes in the primary healthcare setting, was met with several barriers in the ICALM study.
Firstly, due to the lack of mental health training, prior experience and confidence in supporting young people with mental health challenges, Early Help teams struggled to identify suitable young people for IPC-A, assessing most referrals with mental health difficulties to be too complex for their service. The lack of system integration in referral pathways, the variations in frameworks and measures used to assess eligibility of cases and the subjectivity of some aspects of the assessment process meant that some cases may have been classed as too complex by one team and eligible by another. Secondly, interviews with Early Help team managers indicated that despite seeing a large proportion of young people with mental health challenges, the teams did not perceive mental health to be part of their core offer. Thirdly, some Early Help teams did not have a standard intervention for low mood, despite the Early Help threshold matrix indicating that CYP displaying signs of deteriorating mental health and episodes (e.g. low mood/mild depression) should be classed as level 2 and are suitable for Early Help or targeted support services. It was anticipated that IPC-A would be a solution to this, in part because of the relational nature of the therapeutic focus. This posed some challenges with respect to setting up the two trial arms in each service, as young people randomised to TAU were being referred to targeted services (also involved in the trial). Consequently, the additional pressure on targeted services to support young people being referred in from multiple routes, including Early Help teams, posed capacity challenges, which resulted in services having to collaborate to either deliver IPC-A or TAU.
Therefore, if Early Help teams are to be involved in supporting young people experiencing mental health challenges, there needs to be an explicit remit to do so. Early Help practitioners also need adequate mental health training/support to help them identify suitable cases at the triage/assessment stage and to enable them to deliver mental health support for CYP experiencing mild to moderate mental health challenges. Equipping Early Help teams to support young people with mental health challenges has a potential to reduce the capacity pressures currently faced by tier 2 mental health services and to prevent young people from being passed around the system, allowing them to build trusting relationships with one service. Arguably, there is a larger piece of work that is needed to map out the core service offers across the tier 1/2 services, including interventions being delivered to support young people experiencing mental health challenges in order to identify gaps in service provision and capacity to support young people with mental health difficulties. This mapping exercise could then guide both commissioners and researchers in identifying and developing new evidence-based interventions that align with core service offers.
The disconnect between policy and practice
The process evaluation findings highlighted a clear disconnect between mental health policy and transformation models and what happens on the ground with practitioners. The current challenges faced by the central referral point, including poor completion of referral forms from front-line services, a lack of knowledge of front-line staff (e.g. GPs and school staff) with regard to mental health treatment pathways and long waiting times, resonate with the 2020 ‘No wrong door’ report, which highlighted how CYP need to navigate through a very complex system and end up either on a waiting list or falling through gaps where there are no services to meet their needs. The experiences captured through the ICALM study indicate a poor implementation of the core recommendation of this report, which was the need for services to wrap around families and provide support wherever young people present, that is school, general practice. It should be noted that this was during the time and aftermath of the COVID-19 pandemic, which significantly increased demand for all services. There is therefore a need to support front-line services, where young people are presenting with mental health difficulties, especially schools and general practices, to ensure that young people receive interventions at the earliest stage or are supported through the mental health system until they receive the support they need. The transformation plan, which is currently underway, is one way of supporting front-line staff in managing under-18s presenting in general practice with mental health difficulties. To ensure successful implementation of the transformation plan, link workers need to be provided with adequate support, training (especially safeguarding) and supervision to work with under 18s. The successful implementation of the transformation plan could reduce backlogs at central referral points and ensure that CYP receive timely help. Link workers could help increase research capacity by providing much-needed access for recruitment of young people into research investigating evidence-based mental health interventions.
Finally, the increased demand for mental health services and the perception of mental health services providing gold-standard interventions suggest that greater effort is required to promote the philosophy of mental health as ‘everyone’s business’. Establishing a common language with regard to mental health difficulties and educating society with regard to their role in supporting positive mental health is therefore crucial in managing some of the current challenges faced by the services. Such education could, for example, focus on the key components of the Thrive model,60 which is a public health approach to promoting good mental and emotional well-being by supporting and educating schools, communities and parents on the key elements of mental health, the importance of self-care, how to build support within families and communities and when to seek help. More investment in prevention work as part of curriculum is needed by including well-being, resilience, managing emotions and developing healthy relationships. The process evaluation also highlights the need to ensure that acceptance criteria and thresholds for treatment are joined up across the system to ensure easy navigation by young people and families and that there is flow to the most suitable service for the right intervention at the right time.
Implementation and theoretical fidelity discussion
| Main theme | Example data extracts |
|---|---|
| Engagement from young people as crucial for successful, collaborative intervention | It did make me fight hard to make sure she was listened to, and when she was offered advice she took it, because I also know there are lots of teenagers out there who are not ready, or don’t think that there’s anybody out there who can help them, and I was just so proud she just sort of took the bull by the horns and said ‘no I don’t like feeling like this, I want to get myself better’ (Parent-IPC-01, Case 1)‘Because it’s, ultimately, them that are going to attend and do the work, and like I think if young people aren’t there because they want to be, and when I say want, it doesn’t mean they look forward to a session, I mean just a very basic, ‘I’m going to turn up for this and give it a go’. And like even if that’s relatively passive, like ‘oh, well, I don’t trust it’ll change but I’ll go’ … but as long as they want to be there” (Therapist-IPC-06)I’ve had a couple of young people say this to me recently, it’s not that I want you to do anything, I just want you to listen to me. I think that fits really nicely with ICALM because actually that’s what it is. It is being able to listen, and I do have that sometimes with young people, they just want to let anything out and they don’t want you doing anything, it’s just they want to get it out. So, they’ve shared it (Therapist-IPC-02)I think he looked forward to seeing the lady that was dealing with it because he knew that he could just talk about what he wanted to without being judge, while being listened to (Parent-IPC-03, Case 2)[Interviewer] What do you think has changed that has made you able, I guess, to go to do all these things that you weren’t able to do?[YP] I think like talking to people 100% (YP-IPC-04, Case 1)I know there were three grown-ups and a 12/13-year-old, but she was asked and was part of the group, rather than feeling that we were telling her what to do, she was actually being able to make decisions as to whether she – which path she wanted to go down, and whether that was the path she would be happy (Parent-IPC-01, Case 1) |
The IPC-A intervention was acceptable to young people, parents and therapists. However, due to the challenges faced in implementing the study, not all stakeholders felt that the intervention was suitable for Early Help services. The process evaluation findings indicated that evidence-based interventions such as IPC-A can be successfully implemented in tier 1/2 settings, although these services are often overwhelmed with ‘new things to do’ meaning that they struggle to have the capacity to implement new interventions systematically. However, in some cases, there appeared to be a tension between delivery of IPC-A according to its structure and engaging with the young person’s needs and desired outcomes, highlighting a tension between implementation fidelity and theoretical fidelity. Observations of the IPC-A intervention recordings suggest that to meet the aims of IPC-A, practitioners should orientate closely to young person’s needs and perspectives, as a focus on correct delivery alone may not always lead to desired outcomes. Such an approach needs practitioners to be flexible and confident to use IPC-A manual/training as a guide rather than a box-ticking exercise, although if delivered regularly as part of routine care, this confidence should develop. To achieve this level of flexibility and confidence requires long-term investment, ongoing supervision and support of the teams. It is worth noting that despite enthusiasm from services to implement new or research interventions, often services are overwhelmed with the additional tasks these involve and do not always have the capacity to be able to support the implementation of them.
The engagement of young people and collaborating with the practitioner are crucial for achieving desired outcomes from IPC-A. Such engagement and collaboration can be facilitated by the practitioner’s use of terms that the young person is comfortable with, an ability to relate to and empathise with the young person’s experiences and adapting activities to suit the young person’s experiences and abilities. Practitioners need to show they are on the young person’s side by acknowledging the young person’s disclosures and acting as an advocate when needed.
Progression criteria
Below we summarise our findings in relation to the criteria set out at the beginning of the study to inform a future trial of the effectiveness and cost effectiveness of the intervention (IPC-A) (Table 21).
| Sub-themes | Example data extracts | Conversational analysis |
|---|---|---|
| The importance of feeling heard | Within this extract, the therapist uses both directive and non-directive questions to elicit information from the young person and to emphasise their understanding of what the young person is feeling while sharing this information. Towards the end of each directive question, the therapist can be seen to increase their pitch, which seems to suggest ‘answer if you can’, rather than assuming the young person will be able to answer the question. This approach seems to reassure to the young person who is comfortable answering the question in detail and actively participating in the collaborative search for triggers to their symptoms. After disclosing what they feel might be triggering their symptoms, the young person becomes quieter and more passive, which is responded to with reassurance and acknowledgement from the therapist. Affirmative responses from the young person in response to this reassurance suggests these attempts at reassurance are successful. A non-directive question (‘I should imagine you’ve had some nice memories in that house, haven’t you?’) is asked by the therapist, which appears to aim to give the young person space to share more if they would like or simply spend time on thinking about these memories, rather than seeking information – again without pressure for the young person to answer in a particular way. Whilst the young person’s response does not expand on these memories, they signal a clear message to the therapist that they have understood the young people ‘Definitely (.shih)’. Having established a shared understanding, this approach arguably provides a space for the therapist to offer their own perspective and proposal that a ‘big change’ will be the key problem area. Encouraging the young person to continue to actively participate within the session, by showing that any disclosures they make will be met with acknowledgement, reassurance, and indicating that they have been heard and understood by their therapist enacts the intended implementation of IPC-A, in which part of the therapist’s role is to be positive, encouraging, active, non-judgemental and on the young person’s side | |
| In this extract, the therapist and young person are trying to understand what may be causing the young person’s symptoms. The young person repeatedly says that they are unsure of what the cause may be. The young person’s tone appears to shift to illustrate this frustration at having to explain themselves again and the sense of feeling unheard is reinforced by their expression, ‘that’s what I’m saying though’ (line 3). As a culmination of this final disclosure by the young person and not receiving the response they need from the therapist, the young person’s responses illustrate a shift to a form of passive participation, offering one-word responses to show they are listening but not actively engaging with what is being said. In response to this passivity, the therapist attempts to ask direct questions to help the young person re-engage, however these questions fail to acknowledge the uncertainty that the young person has already expressed. Here the young person’s passive participation appears to stem from not knowing how to engage with the task, as expressed by the young person, or resistance to the therapists’ position and approach | ||
| Either way, the lack of engagement from the young person appears to be interpreted by the therapist as active resistance, which seems to lead to greater hesitance from the therapist – as shown by their pauses and awkward laughter while trying to emphasize the importance of finding strategies that will help the young person’s symptoms. Towards the end of the extract, the therapist can be seen to try a more collaborative approach, describing making a plan together rather than aiming direct questions for the young person to answer alone. This approach may be undermined by their previous questions which focus on the young person knowing what their triggers are, putting the onus on the young person to know the answer. As a result, this attempt at collaboration is still met with passive resistance, and a long pause before the therapist goes to speak again. This use of direct questions for the young person to answer alone goes against a key component of IPC-A by removing the collaborative element of therapist and young person working together to understand the young person’s triggers. The resulting response highlights the centrality of the young person feeling heard as a mechanism for enabling their active engagement, and the potential for effective and supportive delivery of IPC-A | ||
| By pausing and changing their tone, the therapist appears hesitant in asking the initial question, perhaps in expectation of the young person’s reaction to it. The therapist seems to use this question to test whether the young person’s relationship with their mum is a viable area of exploration in their sessions. The young person’s response gives a clear indication that they have no desire to focus on this area. After a pause, the young person is quick to disclose that they do not have time, followed by laughter which seems to indicate an awareness of how their statement may be perceived as dismissive. This laughter is then shared by the therapist, acknowledging this awareness. Alternatively, the young person’s laughter may be trying to indicate that the therapist has no appreciation of how unrealistic their suggestion was. The young person follows up with a justification of why they cannot, rather than because they do not want to. This is met with a response of disbelief from the therapist, whose questions appear to attempt to find a time when the young person is free, despite the young person expressing that they are not. The young person responds by actively resisting these attempts by the therapist to further describe why they do not have enough time. In these resistant responses, the young person replies very quickly before slowing down to make their point, ‘>I mean, yeah, because like < I’ll have stuff that’s been set that’s due’ (line 8). This speed to respond, suggests the young person is responding reactively, perhaps feeling the need to defend their view, which may explain their active resistance rather than deferring to the therapists’ view. The therapist finally concedes the point by stating that the young person is a ‘busy lady’. Based on their tone and previous discussions with the YP, this seems an attempt at being light-hearted – however, the young person’s response suggests it was not taken in this manner | ||
| Orienting discussions to a young person’s perspective | In this extract the therapist can be seen to reframe the discussion to reflect the young person’s perspective of their relationship with their mum, ‘because they’re both very very close relationships; how could you make that better, how could they help you more?’ (line 7–9). In doing so, the therapist shows that they are listening to the young person and acknowledges their point of view. This is reinforced in their delivery by putting emphasis on improving the relationship with both mum and dad and getting more help where support already exists. When asking how the young person can feel more supported the therapist can be seen to correct themselves and reframe the question they are trying to ask. In this reframing, they appear to take the onus away from the young person and what they can do alone and back onto the interpersonal relationships and how they can feel supported. This reinforces the theoretical aims of the IPC-A intervention by drawing focus back to how interpersonal relationships can be utilised to help to support the young person | |
| The young person can be seen to put emphasis on the aspects of depressive symptoms which do not feel relevant, in this case the persistence of low mood. In response to the young person’s perspective on depressive symptoms, the therapist asks questions aimed at understanding what the young person is feeling, and what may trigger their fluctuations in mood. Throughout the interaction the passivity of the young person’s responses suggests the therapist’s interpretation of what the young person is feeling does not match what the young person is trying to get across. The young person is unable to articulate an alternative suggestion and so unable to actively participate. This potential frustration at being unable to articulate their thoughts may be reflected in the young person’s face and misinterpreted as confusion by the therapist. Unable to resolve this potential misunderstanding or actively engage the young person, the therapist returns to discussing depression despite the young person’s perspective that they do not relate to this term. The young person then appears to defer to the therapist’s approach, acknowledging they understand but without affirming the therapist’s perspective of their symptoms | ||
| In this interaction, the therapist notices the young person’s difficulty in engaging with the task set (start a timeline to track symptoms). This perception is met with openness from the young person who again tries to describe what they are finding difficult – which appears difficult in itself and requires pause. The young person’s pace quickens when they feel uncertain or unable to explain. In response to this uncertainty, the therapist replies quietly and gives the young person the space to explore why they are finding the task difficult. By giving the young person space to talk about their thoughts, the key term ‘stress’ is discovered and leads to collaboration between therapist and young person to renegotiate the terms they use – away from ‘depression’ to ‘stress and pressure’. This renegotiation/reframing of the discussion is a turning point for this session which enables active participation from the young person | ||
| In comparison to previous extracts from this pair, active participation is seen in the back and forth of responses between therapist and young person, with fewer pauses and one-word responses. Using specific scenarios may help the young person to engage with the task by allowing them to relate to what is being asked – utilising their perspective to foster engagement. This again requires flexibility from the therapist to adjust questions to the young person’s perspective | ||
| This development in interactional style between the pair is further illustrated by the final exchange of this extract. The therapist shares their interpretation of the young person’s parents’ dynamic when arguing and the young person’s response affirms this interpretation as illustrated by the increase in pitch from the young person’s response, ‘Sort of -yeah’ (line 24). The increased level of detail shared by the young person appears to improve the accuracy of the therapists’ interpretations. In contextualising questions to the young person’s experiences, the young person can actively engage and share more information, which the therapist can then use to better understand the young person’s perspective, and further encourage their active participation | ||
| Reliability of interpersonal relationships: mini goals and advocacy | Throughout the young person’s explanation of how their mini goal went, and the response received, the therapist reacts and modulates their tone to express their understanding of what the young person is saying – higher pitch for positives, and lower pitch for negatives | |
| The therapist can also be seen to not only listen and respond to the young person, but to also encourage their consideration of others’ perspectives. This consideration is a key theoretical focus of the agreed problem area worked on in Case 2, ‘relationship disputes’. Importantly, the therapist remains non-judgemental while discussing different people’s needs and expectations from relationships but remaining the young person’s advocate, ‘but for you, what I see is actually you need to be heard’ (line 49). In this extract, the therapist’s interpretation of the young person’s needs strikes a particular chord with the young person and leads to a significant disclosure about suicidal thoughts. This response from the young person further emphasises the importance of feeling heard and understood in facilitating young people’s engagement with IPC-A sessions and difficult issues which need to be addressed to enact change for the young people | ||
| This extract highlights the positive impact that having the therapist act as an advocate for the young person can have. The young person’s responses suggest that since having spoken to the therapist, their mother has made changes in the way they respond to the young person. However, the young person also discloses that they have not had a proper discussion with their mother. This suggests that outside of the sessions, the young person may not be fully engaging in or able to enact mini goals. The therapist then focuses the discussion on the young person’s expectations of how they would like their relationship with their mother to be. This may work to focus their sessions to consider what the young person’s desired resolution for this relationship dispute is. This may also allow the therapist to re-orient the discussion to consider what the young person and therapist can work on together, or what mini goals are achievable, to reach this ultimate goal |
| (.) | Just noticeable pause | |
|---|---|---|
| (0.3), (2.6) | Examples of timed pauses | |
| ↑word,↓word | Onset of noticeable pitch rise or fall (can be difficult to use reliably) | |
| A:B: | word [word[word | Square brackets aligned across adjacent lines denote the start of overlapping talk. Some transcribers also use ‘]’ brackets to show where the overlap stops |
| .hh, hh | in-breath (note the preceding full stop) and out-breath respectively | |
| wo(h)rd | (h) is a try at showing that the word has ‘laughter’ bubbling within it | |
| wor- | A dash shows a sharp cut-off | |
| wo:rd | Colons show that the speaker has stretched the preceding sound | |
| (words) | A guess at what might have been said if unclear | |
| () | Unclear talk. Some transcribers like to represent each syllable of unclear talk with a dash | |
| A:B: | word==word | The equals sign shows that there is no discernible pause between two speakers’ turns or, if put between two sounds within a single speaker’s turn, shows that they run together |
| word, WORD | Underlined sounds are louder, capitals louder still | |
| ºwordº | Material between ‘degree signs’ is quiet | |
| >word word< <word word> | Inwards arrows show faster speech, outward slower | |
| → | Analyst’s signal of a significant line | |
| ((sniff)) | Transcriber’s effort at representing something hard, or impossible, to write phonetically |
- Recruitment rate is at least 80% of target.
Recruitment was impacted from the beginning of the study due to COVID-19, with the study being suspended on 18 March 2020. Following the restart of the study in July 2020, meeting the target sample size within the recruitment period required a recruitment rate of five participants per month. To reach 80% of the target, a recruitment rate of four per month would have been sufficient. The recruitment period was extended to 18 months to allow for the additional referral pathways to take effect, which altered the 80% recruitment rate to 2.7. Following the end of the recruitment period, the final recruitment rate was 0.8 per month.
- At least 70% of those randomised to receive the intervention attend at least three therapy sessions within the 10-week treatment window.
Of the six participants randomised to the IPC-A arm who had come to the end of the 10-week treatment window, three (50%) attended three or more treatment sessions.
- Follow-up assessments are completed by at least 80% of participants at 10 weeks and 70% of participants at 23 weeks.
Of the 12 participants who had reached the 10-week follow-up, 12 (100%) were followed up at 10 weeks and 23 weeks. All of these visits were conducted remotely with the participants.
- At least 80% of IPC treatment sessions reviewed meet treatment fidelity criteria.
From the four recordings reviewed, clear systematic use of the principles of IPC-A was identified; therefore 100% of these sessions met the treatment fidelity criteria.
- Contamination of the control arm can be sufficiently limited for individual randomisation to be justified.
Although there were fewer recordings rated due to lower than anticipated recruitment, there was no evidence of contamination effects from IPC-A training for TAU therapists for the recordings that were provided.
- The mean RCADS depression scores of the IPC-A and TAU groups at 10 weeks are indicative of a clinically significant difference in depression (3 points).
Of the 12 participants (6 randomised to IPC-A and 6 randomised to TAU) who completed the RCADS depression score at 10 weeks, there was no clinically significant difference between the two interventions. The estimated treatment effect was 1.14 points with a p-value of 0.699. Three of the participants who were randomised to IPC-A did not receive the intervention as intended, which may have had a bearing on this result.
Patient and public involvement
This section is adapted from Abotsie et al. (2020). Interpersonal counselling for adolescent depression delivered by youth mental health workers without core professional training: a feasibility randomised controlled trial study protocol. Pilot Feasibility Stud. https://doi.org/10.1186/s40814-020-00733-8 and is licensed under CC-BY-4.0.1
This study has been informed by two PPI events attended by 14 young people, most with personal experience of accessing mental health services. The first event was held at a local school, and the second was with members of Suffolk Children and Young People, Action and Transformation (CAT) group. The young people consulted stressed the inadequacy of current mental health provision for young people and supported the idea of extending access to treatment by training existing staff working with young people to deliver IPC-A. They highlighted that knowing workers have appropriate training is important to building trust and that they would prefer to be treated somewhere familiar to them rather than attend a specialist clinic.
A Youth Advisory Group was formed for this study, made up of three to five young people with personal experience of low mood. This group was facilitated by the dedicated PPI lead co-applicant for the trial.
Before the trial commenced, one young person who had received IPC in the first trial also spoke on a Google Hangout (www.youtube.com/watch?v=Q5rTpX8Ko1s&t=2436s) with a colleague in the USA about their experience. This experience was an invaluable part of the decision to do a wider trial.
The advisory group have contributed to the key decisions regarding the conduct of the trial such as helping the study to identify the appropriate wordings and phrases to explain randomisation to young people in the participant information sheets. Further valuable advice was provided when the study team were considering the possibility of moving the face-to-face IPC-A intervention to telephone/video calls during the national social distancing policy for COVID-19. The concerns raised by young people helped the TSC to recommend a costed suspension of the ICALM study during the COVID-19 crisis.
One of the barriers to engagement of PPI was the pandemic. The group that had originally been formed were made up of young people in years 9/10/11, due to the trial having to be extended this meant that that the priorities for the group had changed. For example, some members began studying for their General Certificate of Secondary Education (GCSE) exams or had already moved on to college and therefore moved on to other priorities.
We originally only had one member of the group who had received IPC-A as a therapy due to the low numbers of young people who had received the therapy.
Although the young people within the steering group were able to use their lived experience around mental health support in general, they declined the offer to join the main steering group meetings as they did not feel the information would be relevant to them. They preferred to be presented with a specific set of questions that they could relate to.
Equality, diversity and inclusion
The study population, although targeted towards individuals between 12 and 18 with low mood, did not have a diverse range of participants, with the majority being white British. Although finding individuals with low mood reflects the target population, the results cannot be generalised. One reason for this is that the referrals all come from a central referring point where people self-refer, and the research team is dependent on services for referrals. For a future study, it will be important to consider how to improve inclusion, whether by joining equality, diversity and inclusion groups to discuss inclusion in a RCT or by considering sites with a more diverse population. It will also be important to consider different referral pathways other than self-referral, as this can be known to be biased towards white British individuals.61
In the case of potential participants who may have difficulty with the English language, information sheets and other materials would have been translated into the preferred language of the potential participants where practically possible. In the case of potential participants with sensory impairment or mild learning disability, information sheets and other materials would be converted into the preferred format (e.g. large print, audio-recording, Easy Read) where practically possible. Where interpretation is necessary for informed consent or other aspects of the study, trained and accredited interpreters would have been used wherever possible. In terms of the research team, characteristics other than age and profession were not included in the study. The research team consisted of a range of professionals from Clinical Psychologist, Psychiatrist, Counselling Psychologist, Research Associates, Head of Children and Young people’s involvement, Research Fellow, Professors of Clinical psychology and of Medical Statistics, Research Assistant Psychologists and a Co-production and engagement lead contributing to a mix of different disciplines at different experience levels.
Ethical considerations
Within the study, ethical considerations were thought about at study design, such as IPC-A having not yet been formally trialled and therefore may not be effective in non-specialist services. It was discussed that it was acceptable for TAU to be given in this study due to no evidence suggesting IPC-A would be better. However, it was not anticipated that at some referral sites there would not be a TAU, resulting in zero treatment being offered. It would be important for future studies to ensure that the TAU is clearly defined for each site, as sites not offering TAU would need to be considered if they are ethically correct to use. Ensuring a TAU could help with barriers to recruitment that came from practitioners feeling that consigning young people to no treatment at all was not appropriate.
Another factor not anticipated with referrals in the trial design, particularly Site-10, was there being no feedback loop allowing for cases to return to the hub if considered inappropriate. Although a lack of system integration in referral pathways was highlighted, it is important for any future studies to clearly identify the referral routes for any potential sites and anticipate any obstacles during this process.
Similarly, the trial was initially hampered by the site's understanding of equipoise and the randomisation process. The trial team supported the sites by including additional information sheets for strategic leaders in an amendment to improve their understanding of the study and attending team meetings to explain research terminology and how this relates to ICALM. Providing this information at the beginning of the study may have resolved some of the difficulties within teams, particularly around the concerns over the randomisation outcomes in TAU, with practitioners communicating a preference for a single-arm study design.
Until a change in Trial Manager in October 2021, the previous Trial Managers were blinded to the randomisation of participants and staff that were providing an intervention for the study. Ideally, in a future RCT, the Trial Manager should be unblinded, as this would resolve issues around participants not taking part in the intervention.
Strengths and limitations
While the study did not meet its aim, certain strengths were acknowledged. Firstly, most methodological elements were acceptable, with participants finding the study procedures and measures easy to understand and complete. Through interviews, staff participants, young people and their parents further acknowledged the usefulness of IPC-A as an intervention to treat low mood. This is of significance as it shows that while we found a definitive trial is not feasible, participants valued the intervention.
Further, training young people’s workers to deliver the intervention was viewed as acceptable, despite this task-shifting strategy being met with a number of challenges in its implementation. Non-qualified practitioners were able to deliver IPC-A, suggesting that such staff could be used to deliver this or similar interventions in the future. Last but not least, one of the main strengths of the study was the in-depth process evaluation, which provided great insight into the barriers to running RCTs in non-specialist mental health services providing clear directions for future research.
A major limitation of the research was the number and complexity of referrals coming through services. This resulted in services with limited capacity prioritising urgent cases, and lower-risk cases, whose presentation would likely be suitable for ICALM, being set aside.
There was a significant impact from COVID-19 due to the recurring lockdowns, causing a delay in recruitment. There was also an impact on the clinical presentation of low mood, with an increase in anxiety cases amongst services from the start and progression of the pandemic. Evidence suggests that the frequency and severity in adolescent mental health presentations increased during the pandemic, and particularly increased for anxiety, which was reflected in the referrals received by services.62 Despite comorbidities between anxiety and low mood, young people expressed greater preference in treating anxiety than low mood. Consequently, accessing suitable participants was an overarching limitation of the study.
Additionally, the study focused on tiers 1 and 2 services plus Early Help which is provided by social care. As discussed previously, it was suggested by stakeholders that IPC-A may be better suited to services where young people are worked with individually, rather than with the whole family, as is often the case in tier 2 services, although young people with mental health difficulties present in large numbers at all of these services who are set up to address their needs. Likewise, challenges were identified in implementing a RCT in all of these services due to practitioners’ lack of familiarity with research procedures. This indicates the importance of the study team ensuring services are familiar with research processes and implementation of clinical research, something that could have been avoided with further training.
The large turnover of staff delivering IPC-A and TAU and thus the repeated need for training of new staff was another challenge, which had not been anticipated by the research team. This resulted in services being limited on the number of participants they could see, especially under the IPC-A arm, thus creating another potential barrier to recruitment of participants. A larger recruitment of staff and/or services is suggested to prevent such limitations. Concerns over the randomisation outcomes in TAU were a particular challenge, with practitioners communicating a preference for a single-arm study design.
Further, we had intended to provide evidence of the cost effectiveness and potential benefit of IPC-A when compared to TAU. However, due to the low recruitment number, the study was not powered enough to provide any meaningful results on potential benefits of the intervention or the cost effectiveness to be able to comment.
Conclusion
The findings of this feasibility study and the process evaluation indicate that conducting a randomised clinical trial of IPC-A in non-specialist community services is not feasible in the current climate (staff shortages, high staff turnover, high pressure on services making trials a low priority, low familiarity with trials, unclear boundaries between services). It remains unknown if IPC-A is a useful therapeutic addition to universal and mild to moderate services for CYP which should be recommended. This is despite the best efforts of the trial team, clinical teams and commissioners to find ways of making the study succeed. The results did suggest that the data collection and outcome measures were suitable. The intervention was acceptable in all settings, but factors such as high staff turnover, increasing demand for services and increasing complexity of presentations of mental ill heath made it impossible to conduct the feasibility trial within community settings for mild to moderate mental health services.
The impact of COVID-19 was substantial both on services and delivery of the study, and for a full trial to be successful, significant enduring barriers to implementation need to be overcome. It is impossible to know if the original trial design would have been successful, but the process evaluation highlights core complexities within the CYP’s mental health landscape which need to be taken into account for any future trial to be successful. Using the learnings from this study and mapping out the core services in tiers 1 and 2 and the current pathways between them before embarking on any future trials within CYP’s services is critical.
The process evaluation describes in detail the individual elements at macro, meso and micro levels, which ultimately made it impossible for the feasibility trial to meet its progression criteria. While only conducted within two counties within the UK and across multiple systems (health, social care, education and the voluntary sector), it is highly likely that this process evaluation describes issues pertinent to other areas of the UK and the rest of the world. We highly recommend paying specific attention to these issues before setting out on future trials in this clinical area or making assumptions about services involved in clinical trials in the post pandemic era.