Process evaluation

This chapter is adapted from Katangwe-Chigamba et al.43 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) licence, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The text below includes minor additions and formatting changes to the original text.

Design

An embedded mixed-methods ethnographic process evaluation to generate an understanding of intervention implementation across services and explore the acceptability of the intervention from the perspective of young people and other key stakeholders.

Objectives

The objectives of the process evaluation were to:

1. provide a description of how IPC-A and TAU were delivered
2. assess implementation and theoretical fidelity to the IPC-A model over time
3. observe how delivery is shaped by the context of differing service models
4. identify any harms arising from treatment (including end of treatment)
5. establish the extent and source of any contamination of the control arm.

Recruitment procedure

Consent for young people and parents to participate in the process evaluation (interviews and having sessions video/audiotaped) was sought during the main consent procedures and ascertained at the 23-week assessment. All parents whose child participated were invited to take part and asked to complete a separate consent form. However, it was not a requirement that a young person/parent consents to the process evaluation in order to take part in the study. IPC-A or TAU YMHWs’ participation in the process evaluation (providing recorded audiotapes of therapy sessions and/or participating in an interview) was also optional; staff were given participant information sheets and asked for informed written consent. Consent to complete the SPQ and participate in an interview was also obtained from managers and/or service leads.

Methods

Data collection methods included SPQ, policy documents, observations of training workshops and supervision, video-/audio-recordings of treatment sessions, fidelity ratings of IPC-A and TAU sessions according to the IPC-A rating scale, interviews with young people (and parents/carer), focus groups with YMHWs’ and wider stakeholders. Data collection aimed to facilitate an understanding of how the delivery of IPC-A was organised within the context of differing service models by setting out macro-, meso- and micro-contextual features relevant to implementation.

Data collected to inform the macro- and meso-contextual features shaping implementation

To understand the macro-contextual features shaping implementation and delivery of the ICALM intervention, we conducted a documentary review of key policy documents to identify contextual features likely to shape implementation of ICALM at the meso and micro levels. To understand the meso-contextual features shaping implementation and delivery of ICALM, we collected SPQ from participating sites. The questionnaires aimed to understand the broader service context in which the intervention is delivered, including TAU for young people with mental health needs; policies, protocols and procedures used by staff; numbers of young people with mental health needs and proportion with depression; training and experience of staff in treating depression in young people; and allocation and distribution of staff to support young people with mental health needs. In conjunction with the SPQ, we planned to also conduct focus groups with key stakeholders (including commissioners, education representatives and service managers) at the end of the study to review study findings and discuss implementation barriers and sustainability of implementation.

Data collected to inform the micro-contextual features shaping implementation

To understand how therapists delivered IPC-A at a micro-contextual level, we collected/conducted:

1. Observations of IPC-A training workshops and supervision: Researchers’ field notes from observations of training and supervision of IPC-A therapists were used to provide a detailed description of process and content of staff training and supervision.
2. Video-/audio-recordings of treatment sessions and fidelity ratings: All therapy sessions were video- or audio-recorded, subject to consent. The recorded therapy sessions were rated by one of the supervisors according to the IPC-A Audio Recording Rating Scale28 to monitor implementation fidelity to the IPC-A treatment model and to assess the degree of contamination. This scale includes ratings of techniques for the assessment (e.g. ‘Complete an interpersonal inventory’), how sessions ended, specific focus areas (e.g. ‘Exploration and discussion of differences in expectations for relationship disputes’) and general IPC techniques to be used for all sessions (e.g. ‘Clear focus on depressive symptoms and interpersonal relationships’). TAU therapists were also asked to submit sessions for rating. The aim was to understand and describe TAU and assess contamination (i.e. evidence of TAU therapists delivering any elements of the IPC-A intervention), rather than to rate the quality of the young person’s therapy.
3. Interviews with participants (and parents/carer) from the IPC-A and TAU arms: Young people participating in the RCT (and their parents) were invited to take part in in-depth interviews after their final assessments (23-week assessment). Subject to consent, a separate interview was also undertaken with a parent/carer. During the interviews, participants were asked about their experience and views of the process of accessing help, the content of intervention sessions, contacts they had in addition to study therapy sessions, whether they felt they had benefitted from receiving the intervention, the experience of ending therapy and suggestions for improvement.
4. Interviews with IPC-A YMHWs (see changes below): Following completion of delivery of the IPC-A and TAU arms, we conducted interviews with YMHWs to understand staff perspectives of each study arm. For IPC-A YMHWs, interviews focused on barriers and facilitators to successful delivery, experiences and views of intervention sessions, additional work required to support delivery of IPC and suggestions for improvement. For TAU YMHWs, interviews focused on how TAU is delivered, the additional support young people in TAU have received and their awareness and perspectives of IPC-A.

Data analysis

A linguistic ethnographic methodology,44,45 which included multiple and mixed methods, was employed to investigate how implementation of ICALM was shaped by the wider context of mental health services and to provide a broad description of intervention delivery. Thematic analysis was used for SPQ, interview and focus group data and populate macro-, meso- and micro-contextual features. Macro, meso and micro levels refer to levels of context, providing an organising structure for our investigation.46 Along with observational field notes of training sessions, these represented the ethnographic element of this methodology. Conversation analysis was used on the interaction video/audio data to understand interactional sequences that would provide insight into theoretical fidelity of the intervention. The analysis of qualitative data was iterative, moving between session recordings and interviews to test emerging theories. A constant comparison analytical technique, involving triangulating and looking for connections between data, was employed to ensure rigour as the analyst’s interpretations developed.

Thematic analysis

Data sources: site profile questionnaires, interviews, focus groups, observations

To understand how participating organisations were organised to provide mental health services to young people and the interaction between contextual features and intervention delivery, we analysed SPQ and stakeholder interviews/focus groups. Interview/focus group data were transcribed verbatim and thematically analysed together with the questionnaire data to describe the participating services and to set out macro-, meso- and micro-contextual features relevant to implementation within each provider. By framing the analysis of intervention implementation within a macro-, meso- and micro-contextual framework, we aimed to make the transition from the identification of routines and patterns of use in the specific services participating in the current study to theoretical explanations of how different structural relations and mechanisms of the intervention organise moments of delivery, which then impact on specific outcomes. Such insights were particularly evident in highlighting the challenges of conducting key processes for the trial which we report on in detail.

To provide a description of the process and content of therapy sessions, staff training and supervision, researchers’ field notes from observations of training, supervision of IPC-A therapists and interviews with young people, parents/carers and therapists were transcribed verbatim and thematically analysed with the aid of NVivo software. Aspects of speech delivery, such as loudness and changes in pitch, were indicated using symbols (see Appendix 1, Table 24). For intervention arm participants, we developed a coding scheme to evaluate how the process and content of IPC-A as delivered by the YMHWs functioned from the participants’ perspective. In the control arm, we assessed how participants experienced the TAU provided by their YMHW and any other sources of support used. A constant comparison approach was adopted, working iteratively between data obtained from different interviewees within and between implementation sites.

Conversation analysis

Data source: session audio-recordings

Audio-recordings and transcriptions of therapy sessions were analysed to evaluate implementation and theoretical fidelity. To monitor implementation fidelity in the intervention arm and to assess the degree of contamination in the TAU arm, we rated sessions against the IPC-A treatment model. To evaluate theoretical fidelity, a purposive sample of extracts from recorded IPC-A sessions was transcribed according to Jeffersonian conventions and subject to conversation analysis in order to identify how IPC-A components are communicated by therapists and received by young people, including how the mechanisms of the IPC-A intervention function to affect change within and across individual counselling sessions.

Changes to process evaluation

Process evaluation objectives

Due to recruitment and implementation challenges, the funder requested a shift in focus for the study, with greater emphasis being placed on understanding the main barriers to running such trials (i.e. RCTs) within the mental health setting. Therefore, to gain insight into how (or how not) to design future studies, while still reporting on all objectives, the process evaluation places greater emphasis on objectives 1 and 3 in order to highlight the challenges of running this type of research in this setting. Therefore, in the findings, we provide a description of the services, the challenges the services were experiencing and how this impacted ICALM.

Data collection

Low recruitment rates (both for the study and the process evaluation elements) and challenges in obtaining data from participating services negatively impacted the quantity of data collected for the process evaluation and led to changes in data collection methods, as outlined below:

• Site profile questionnaires: For each site, we originally intended to collect questionnaire data at baseline and at the end of the study. However, due to delays in obtaining data from participating services, we only collected one questionnaire per site at various study time points.
• Focus group with professional stakeholders: We intended to hold one focus group with professional stakeholders at the end of the study to review study findings and discuss implementation barriers and sustainability of implementation. However, due to the change in key focus outlined above, in-depth stakeholder views were sought by primarily conducting interviews, with commissioners and managers and/or service leads from each participating organisation before the end of the study. We therefore did not conduct focus groups with key stakeholders to review study findings at the end of the study.
• Interviews with participants and parents: We originally planned to conduct 20 interviews with young people (10 IPC-A and 10 TAU) and 20 interviews with parents/carers (10 IPC-A and 10 TAU). However, only 16 interviews in total were conducted.
• Young person workers focus groups: We intended to conduct one focus group per arm per service provider with participating young person workers. However, due to some workers moving on from the services and the low number of consenting workers available, we conducted interviews instead. This change allowed us to have an in-depth understanding of implementation and intervention delivery, despite the recruitment limitations.
• Video-/audio-recordings of treatment sessions and observations of supervision: To evaluate implementation fidelity, we originally planned to randomly select therapy sessions (15% in each arm) for rating by one of the supervisors according to the IPC-A Audio Recording Rating Scale. To evaluate theoretical fidelity, we planned to use the ratings to purposively sample extracts of recorded IPC-A sessions to be transcribed according to Jeffersonian conventions and subject to conversation analysis in order to identify how IPC-A components were communicated by therapists and received by young people, including how the mechanisms of the IPC-A intervention function to affect change within and across individual counselling sessions. However, with only a few participants consenting to their session recordings being analysed for the process evaluation, all sessions were assessed for implementation and theoretical fidelity.

Results

• Site profile questionnaires: We collected a total of 9 questionnaires out of the 13 participating sites, including 5 from Suffolk (Site-01_Early Help and NEET Team, Site-02_Early Help and NEET Team, Site-03_Early Help and NEET Team, Site-07_Charity and Site-04_Wellbeing Service) and 4 from Norfolk (Site-05_Charity, Site-06_Charity, Site-08_Community NHS Trust and Site-09_Family Support Team). An additional questionnaire was obtained from Site-10_Central referral point.
• Interviews with professional stakeholders: We conducted eight interviews and one focus group with stakeholders, including: head of service (one), clinical/therapeutic leads (two), clinical managers (two), senior practitioner (one), team managers (four), clinical nurse specialist (one). Stakeholders represented all 13 teams involved in the study as well as Site-10_Central referral point which supported referrals to the Early Help and NEET teams in Suffolk.
• Interviews with participants and parents: We conducted 16 interviews in total, 8 in the intervention arm (4 young people and 4 parents) and 8 in the TAU arm (3 young people and 5 parents).
• Young person worker interviews: We conducted eight interviews in total; one with an IPC-A supervisor, six with IPC-A therapists and one with a TAU therapist.
• Video-/audio-recordings of treatment sessions and observations of supervision: Only four participants (three IPC-A and one TAU) consented to their session recordings being analysed for the process evaluation. Therefore, all recordings were rated and analysed.
• Observations of IPC-A training workshops and supervision: Two training observations were conducted, and 10 supervision session recordings with 2 IPC-A therapists were provided to the process evaluation team.

Results structure

In order to retain anonymity, organisations and participants involved in the study have been given an ID (Table 11). Firstly, we provide a description of the participating organisations (Tables 11 and 12). Secondly, we report macro-, meso- and micro-contextual features of non-specialist mental health services which had an impact on study and intervention implementation. In Tables 10–13, we set out the macro- and meso-contextual features, with illustrative quotes from SPQ and interviews. Thirdly, we present findings on acceptability and implementation/theoretical fidelity of the intervention. Within this analysis of session recordings, evidence of how macro- and meso-contextual features were made salient by therapists at a micro-contextual level was less evident. However, we provide insight into implementation and theoretical fidelity of the intervention using telling cases to illustrate mechanisms and contextual conditions which appeared key to successful delivery of the intervention. Finally, we then use these findings to make recommendations on (1) how to strengthen mental health services to meet the needs of young people; (2) how to strengthen mental health services to enable new interventions such as ICALM to be implemented (including a consideration of the challenges of taking on research); and (3) how to adapt the intervention content or implementation strategies (e.g. training, referral processes).

Description of services involved in the ICALM study

Nine sites across seven organisations, including two County Councils, were involved in the delivery of IPC-A (green) and/or TAU for the study (Figure 5), three in Suffolk and four in Norfolk. Two of the organisations, Norfolk and Suffolk County Council, consisted of several teams, some of which did not (fully) participate in the study (grey). Four of the organisations worked collaboratively to provide either TAU or IPC-A: two in Suffolk (Site-04_Wellbeing Service and Site-07_Charity) and two in Norfolk (Site-05_Charity and Site-06_Charity).

FIGURE 5 Services involved in the ICALM trial. (Figure not reproduced here; see the original article.)

A summarised description of the services involved in the delivery of IPC-A and/or TAU is provided in Table 12. Participating services were tier 2 services, consisting of three charities, three Early Help teams and NEET teams, one wellbeing service and one family support team, each providing a range of support, including mental health interventions for CYPs aged 0–19.

Types of mental health support provided by the services

Tier 2 services (Early Help and targeted services) are designed to offer a range of support, including early and short-term interventions for CYP experiencing mild to moderate mental health problems. Service providers estimated that approximately 40–90% of the cases referred to their services involve some element of mental health difficulty, in particular low mood and anxiety. Other mental health difficulties seen by the services included panic attacks, phobias, suicidal ideation, obsessive–compulsive disorder (OCD), emotional dysregulation, poor emotional literacy and emotionally unstable personality disorder/self-harm (EUPD, also referred to as borderline personality disorder).

While all services included in the study are designed to offer early help, only four services (Site-04_Wellbeing Service, Site-05_Charity, Site-06_Charity, Site-07_Charity) offered targeted services for mild to moderate mental health problems. Targeted services were already providing support other than IPC-A to CYP experiencing mild to moderate mental health difficulties such as anxiety, low mood and mild trauma-type presentations. Types of support target services offered for these presentations included advice, guidance, counselling, treatment/therapeutic interventions [including cognitive behavioural therapy (CBT)-informed interventions] and guided self-help. Interventions offered by the services were typically short-term (six to eight sessions) one-to-one interventions, delivered either virtually (during and post COVID-19) or in various settings according to preference of CYP.

Support offered by Early Help teams and NEET teams in Suffolk focused on supporting families, young people at risk of being NEET and delivery of interventions and support to families in receipt of social care and child protection (CP). Apart from IPC-A, Early Help teams did not offer any specific mental health support but addressed mental health-related issues, for example, peer-related issues, emotional health promotion, motivation, confidence and self-esteem, including low mood occurring within the young person’s life. In Norfolk, support offered by Site-08_Community NHS Trust and Site-09_Family Support Team primarily focused on emotional resilience and confidence building. The teams also depended on targeted services via onward referral for support with specific mental health-related issues, such as low mood.

Mental health is a very small part of what we do but is also very common. Our interventions focussed on safety predominantly but then health and wellbeing is a secondary important focus. We would work alongside any families experiencing any mental health problem, we wouldn’t necessarily take the lead on the mental health part.

There’s no other intervention [apart from IPC-A] that’s specifically focussed on mental health, just our Trauma Informed Practice and our use of Signs of Safety, which is a way to map worries.

Referral pathways into non-specialist services involved in the ICALM study

In Suffolk, young people (and parents) can self-refer either directly to all non-specialist mental health services or through a central referral point (Site-10_Central referral point in Suffolk). Other major gateways into the services include general practice, schools and social care [via common assessment framework (CAF)]. In Norfolk, referrals to the Early Help teams (family support teams) are received from professionals via the Children’s Advice and Duty Service (CADS) or directly from young people and families. Referrals to targeted services (i.e. Site-05_Charity and Site-06_Charity) are received from professionals via a single point of access (SPOA).

Site 10_Central referral point

The majority of CYPs presenting at General Practice with mental health difficulties get referred to the services through Site 10, which is a central referral point for families, professionals and young people (age 0–25) to access mental health services. The referral point consists of a team of 10 staff members, including 7 clinicians, whose role is to screen referrals and triage them by rating them against risk criteria and identifying the right service for them. The referral pathway to and from the referral point is outlined in Figure 6. Each case is RAG (red, amber, green) rated by two qualified clinicians within 24 hours, and the outcome and rationale are recorded.

FIGURE 6 The referral pathway for Site-10_Central referral point. YP, young people. (Figure not reproduced here; see the original article.)

Red cases: High risk of harm to self (and others); severe mental health difficulties, for example, eating disorders, signs of psychosis, actively suicidal (not just thoughts/plans but showing a clear intent), not leaving the house for over a year; and severe OCD. These cases are dealt with within 24 hours by referring them straight to secondary services (tier 3) or emergency departments.

Amber cases: Moderate risk, for example, suicidal ideation – but with lesser intent/ planning, self-harm requiring medical intervention alongside presenting issues and some chronicity/impulsivity, safeguarding concerns, significant phobias, symptoms having a significant impact on daily life or self-care, multiple trauma, significant tics and selective mutism. These cases are seen within 72 hours. These cases may go to secondary services (tier 3) or primary services (tier 2, e.g. Under 18 wellbeing Service).

Green cases: Mild risk, for example, low mood, anxiety due to COVID-19, trauma, bereavement, mild to moderate OCD, attachment difficulties, safeguarding, anger management and emotional dysregulation. These cases are dealt with within 28 days.

The Common Assessment Framework

Schools and social workers can refer CYPs experiencing mental health difficulties directly to early help and targeted services. These referrals usually come through pastoral support, teachers, head teachers and school nurses who identify CYP through self-disclosure or observation, for example, recurring patterns of a problematic behaviour. Referrers complete a CAF, which has a pre-assessment checklist to help identify CYPs with additional needs and decide who would benefit from a common assessment. Schools then put the CAF into a portal which is picked up by the CAF Admin team who then sends them through to the relevant geographical/local teams. The CAF is designed to promote a co-ordinated service provision to meet CYP’s needs. The assessment covers three domains: development of the child or young person; parents and carers; and family and environment.47

Children’s Advice and Duty Service

In Norfolk, CYP, families, professionals and social workers can refer CYP to Early Help and family support teams directly or through CADS. CADS consists of a team of Consultant Social Workers who provide advice and support to families, social workers and professionals with concerns about a child. The Consultant Social Workers also help with signposting by identifying the correct services to support CYP. CADS also decides whether referrals need a social work assessment or family support.

The single point of access teams

The SPOA in Norfolk is a team of administrators who receive referrals from professionals, parents/carers and young people. The team assesses referrals and signposts them to appropriate clinical teams and services to receive support and guidance. The team is not clinical and does not offer treatments.

Assessment processes for eligibility into the services – Suffolk

Services reported using various assessment processes, frameworks and measures to make decisions on whether referred cases were appropriate for their service or needed signposting/escalation. Assessment processes differed between and within services providing Early Help /family support teams and targeted support services in Norfolk and Suffolk. The services reported holding regular triage or transfer meetings to discuss referrals classed to be outside the service remit or needing escalation. The meetings are attended by representatives from Early Help teams, Point 1 services and well-being (tier 2 Norfolk) services and tier 3 services, for example, Norfolk and Suffolk NHS Foundation Trust (NSFT) and social care who would work together to identify the right services for cases.

Assessment processes for services NOT offering targeted mental health services

Suffolk Early Help and NEET Teams (Site-01, Site-02 and Site-03): Early Help teams within the same organisation, for example, Suffolk County Council had a similar assessment process, which involved three stages (Figure 7): (1) an initial triage involving assessment of the CAF by a practice lead and speaking to CYP or referrer (e.g. parents, school) to identify presenting need, history, etc.; (2) a risk assessment using a threshold matrix to identify whether the service threshold is met. The matrix classifies cases into four categories and gives an indication of the type of support that may be required at each level: level 1 (children with universal needs), level 2 (low risk to vulnerable – early help or targeted support services), level 3 (medium risk/high or complex level of additional needs ranging from targeted early help to CIN) and level 4 (high risk/complex or acute level of additional needs requiring specialist or statutory integrated response). While the matrix serves as a decision-making guide, it recommends professional judgement to always be informed by any known evidence and in consideration of views of children and families and the impact that any risk or uncertainty is likely to have on their safety and well-being.48 Therefore, the teams reported making case-by-case judgements of CYP suitability for the service and (3) cases meeting the service threshold are assigned to a FSP or a young person’s worker (YPW), who then meets with young person, family and professionals to map Signs of Safety, strengths/protective factors and worries/difficulties using the Signs of Safety and well-being model. This holistic map with family and professionals guides the team to provide a tailored intervention plan. Where there is a safety concern, the teams consult with consultant social workers who offer advice and appropriate next steps. Early Help then develops a family plan (normally CYP-led except where there are safety concerns) in consultation with other professionals. The plans are only individual (not family-oriented) if young person has capacity to be independent. CAF cases last around 6 months from assessment to closure, some longer than a year if complex.

FIGURE 7 Assessment process for Early Help and NEET Teams (Site-01, Site-02 and Site-03). (Figure not reproduced here; see the original article.)

Community NHS Trust (Site-08): The assessment process for Site-08_Community NHS Trust (Figure 8) also involves an initial triage by speaking to the referrer (if necessary) and undertaking a risk assessment. However, unlike the Early Help teams in Suffolk, Site-08_Community NHS Trust uses the RAG rating document to assess risk: green = low risk (appropriate for CCS, e.g. new low mood or anxiety); amber = moderate risk (need further discussion) and red = significant risk (e.g. severe mental health difficulties, low mood associated with other risk factors such as self-harm or suicide plan; ongoing issues (over several years); bereavement; experience of domestic abuse).

FIGURE 8 Assessment process for Site-08_Community NHS Trust. (Figure not reproduced here; see the original article.)

Red cases are deemed unsuitable for the service and escalated immediately to targeted tier 2 services, for example, Point 1 or Wellbeing Service. Point 1 is a tier 2 commissioned service which provides assessment, support and therapeutic help by skilled professionals for CYPs aged 0–18 years old who are experiencing early signs of mild to moderate mental health issues and emotional difficulties. Following the triage, green or amber cases are further assessed for day functioning, motivation, young people’s aims and goals and risk of self-harm or suicidal ideation, etc. using RCADS to decide if appropriate for Site-08_Community NHS Trust pathway. A score over 80 on the RCADS is deemed unsuitable and is escalated to tier 3 CAMH services. If appropriate, a resilience worker gets in touch with young people and offers six to eight sessions of low-level CBT-informed work (not CBT therapists but have some training).

Family Support Team (Site-09): The process for this family support team involves assessing referrals (mainly those coming from CADS) and using professional judgement to decide whether the case falls within the service remit, that is self-esteem and emotional work or if there is an apparent mental health issue. Where there is a mental health issue or CYP identified as requiring a more specialist service and more intrusive treatment (e.g. self-harm), the teams make decisions in consultation with CAMHS and trauma-based workers during a Point 1 group supervision to decide what intervention is needed. Alternatively, they refer young people to general practice so that they can obtain a referral to CAMHS.

Assessment processes for Services offering targeted mental health services (Site-04_Wellbeing Service, Site-05_Charity, Site-06_Charity and Site-07_Charity).

Initial screening processes, including measures and tools used, also differed between services providing targeted mental health interventions (see Figure 7). For example, Site-06_Charity and Site-04_Wellbeing Service tended to carry out an initial triage process to screen cases against risk before carrying out an in-depth assessment, whereas Site-05_Charity and Site-07_Charity did not (Figure 9). Although the in-depth assessment processes for these services involved similar activities (i.e. setting up meetings with young people and parent/carer to look through history, presenting issues, risk and safety planning and identifying suitable treatment), the services described used different measures and tools to further decide on suitability of the cases. For example, while both Site-04_Wellbeing Service and Site-07_Charity described using the RCADS, Site-06_Charity employed the Five P’s formula as an aid to identify both current and historic factors affecting CYP, and Site-05_Charity completed a detailed assessment in collaboration with the young person and his/her family as part of the first session of the treatment package (a specification of pathway) which they use to allocate young people to either youth-centred counselling or youth-centred mental health work, which are standard offers for the service.

FIGURE 9 Assessment process for teams offering targeted intervention for mental health (Site-04_Wellbeing Service, Site-05_Charity, Site-06_Charity and Site-07_Charity). PHQ, Patient Health Questionnaire. (Figure not reproduced here; see the original article.)

Staff characteristics

A summary of the staff characteristics for the services is presented in Table 13. Most services consisted of a small number of staff members (12–20 staff, including management) employed on a mixture of full- and part-time contracts. Staff expertise differed according to whether the services were offering early help only or targeted mental health services. Generally, Early Help teams tended to primarily consist of staff with no professional mental health backgrounds, whereas targeted service providers consisted of some staff with mental health qualifications.

Services offering early help primarily consisted of FSP and YPW. FSPs had no professional qualification and came from a mixture of backgrounds, including a qualification in a related subject (e.g. Level 3 qualification in Health and Social Care, Nursery Nursing or Child Development) or relevant experiences (e.g. education, social care, police, health, health visitors, parenting support and Early Childhood Family Service or children centres). YPWs were primarily Joint Negotiating Committee-qualified youth workers (with a degree focused on Youth, e.g. Youth and Community Studies/Youth and Community Development/Youth Work) or had a Level 3 qualification in Health and Social Care. Promotion to Senior FSP was primarily based on experience. Site-08_Community NHS Trust largely consisted of a team of workers known as REHPs (Band 4) who did the bulk of the interventions and had a similar background to FSPs, with some coming from a school/pastoral and children centres' backgrounds. Practice leads and practice management staff also had similar backgrounds as FSPs and YPWs, with promotion based on experience.

Services offering targeted services were primarily made up of a team of CWPs, PMHWs, CBT and counsellors. CWPs do not have a core professional mental health qualification but hold a postgraduate CWP practice qualification (with undergraduate degree in a related field). The CWP qualification is a yearlong course supervised by a trained CWP supervisor and involves working with the clinical team to complete pre-specified clinical hours with CYP/parents and various written tasks, for example, portfolio and reflections on practice. The course teaches various therapeutic interventions, including graded exposure and psychoeducation for young people aged 12 + presenting with anxiety; behavioural activation for young people aged 12 + presenting with low mood; and parent-led graded exposure ‘helping children with fears, worries and feelings of distress’ for children under 12 presenting with either anxiety or low mood.

Primary mental health workers come from a mixture of backgrounds (e.g. nurses, school nurses, social workers, drama therapists), have a core mental health qualification and are registered with respective professional bodies, that is Social Work England, Royal College of Nursing, etc. Cognitive–behavioural therapists are trained and qualified CBTs and registered with British association for behavioural and cognitive behavioural psychotherapists.

Ongoing training and support

There were wide variations in the types of ongoing mental health training and support offered to staff, delivering early help only and targeted services.

Most Early Help practitioners received basic short-term optional mental health training as part of their professional development. The staff also have access to an online database of resources and tools which includes modules on mental health topics such as anxiety, low mood, confidence, self-harm, basic CBT, attachment, etc. Some resources also focus on skill enhancement, including (1) mental health first aid training for family support works (FSWs); (2) trauma-informed practice; (3) the Graded Care Profile training (an assessment tool designed to help workers identify when a child is at risk of neglect by assisting them to measure the quality of care being given to a child in respect of capacity, communication, interaction and well-being); and (4) Brief Solution-Focused Therapy (Yearlong course – manager only) or 2-day Signs of Safety practice (including brief therapy). The team also have access to 1 : 1 consultation with a clinical psychologist who provides clinical supervision, addresses mental health concerns and helps with unsticking stuck cases by considering the mental health issues. Clinical psychologists also facilitate workshops on various mental health and emotional well-being topics.

Site-08_Community NHS Trust team received a 3-day Accredited Mental Health Training (Association for Psychological Therapies) and CBT training from a Clinical Psychologist (although they are not CBT therapists). They also undertook Solihull training; a relationship-based training focused on attachment and brain development with three main principles, including containment, reciprocity and behaviour management. The Solihull training is an initial 2- to 3-day training, followed by an annual top-up. The team also completed mindfulness training and have access to online training, including trauma-based training and understanding of the teenage brain.

Other teams offering targeted mental health services reported receiving very little or no additional mental health-focused training apart from qualifications and their ongoing continuing professional development (CPD).

Contextual features of mental health services

To understand the contextual features shaping implementation, synthesised data from the documentary review, SPQ and stakeholder interviews have been used to map macro-, meso- and micro-contextual features. This analysis considered how national policy, societal contextual features and the COVID-19 pandemic at a macro level impacted the non-specialist mental health services at a meso level, and then ultimately how the ICALM study was impacted at a micro level. The mapping of macro-, meso- and micro-contextual features is a useful heuristic device for conducting process evaluation, but it inevitably led to arbitrary separation of what, in practice, is complex.

A major challenge experienced by the services at a meso level is a huge volume of referrals, exacerbated during the COVID-19 pandemic, firstly leading to backlogs at central referral points such as Site-10_Central referral point in Suffolk and secondly (and consequently) leading to long waiting times. The Site-10_Central referral point in Suffolk received approximately 9000 referrals a year, compounded by increased demand following COVID-19, ranking amongst the top three trusts in the country for referrals. Towards the end of the ICALM study, to resolve the increase in referrals, Site 10 was looking to temporarily increase resources by using agency workers to help clear the backlog. However, similar efforts in the past have flooded other services which were already under pressure, such as the school nursing services. Site 10 was also working to identify bottlenecks in the services, for example, attention deficit hyperactivity disorder (ADHD), and putting more resources to triage them, a process also envisaged to have knock-on effects on capacity of teams aligned to manage those specific conditions.

We have our urgent which, yeah, 24 hours or 72 hours. The target for a green, which will be a low mood, very limited risk, would be 28 days. We are not there at the moment. We are at about five months. But when I started six months ago, we were at 11 months.

The [Site-10_Central referral point] model is unsustainable, and they are overwhelmed. The way the [service]works right now, anyone can refer themselves in and everyone must be triaged – they have about 2000 people waiting to be triaged. Because they’re so overwhelmed, they’re now only triaging Red and Amber cases – so the green ones we used to work with aren’t getting referred due to the backlog and prioritising Amber and Red cases.

ADHD was a big problem for us. It still remains a big problem for us but we’ve streamlined some of the service. So we’ve just put some people behind the problem in terms of trying to help it … there is a knock-on effect on PMHW, because actually if anxiety is picked up during an assessment for ADHD, anxiety must be tackled first, which then would fall down to the PMHW teams. So as much as we’ve targeted one area, we’ve also pushed it elsewhere …

The increased pressure on Site-10_Central referral point and, consequently, services such as Site-04_Wellbeing Service and school nurses resulted in significantly reduced capacity to utilise pathways of referral into ICALM and on the practitioners to undertake and/or complete their IPC-A training due to very limited numbers of mild low mood being accessed and passed onto appropriate services.

Macro-contextual features of mental health services

The findings that follow focus on how societal contextual features, the COVID-19 pandemic and policy at a macro level have shaped the delivery of mental health services and the ICALM intervention at a meso level. Apart from policy documents presented and referenced, this primarily represents views of stakeholders.

Societal contextual features

Stakeholders attributed the high volume of referrals to a combination of (1) social contextual factors (Table 14), including increased population awareness of mental health challenges, inappropriate or changing use of specific mental health language by parents and other health professionals and (2) the COVID-19 pandemic (Table 15).

In recent years, the national effort to increase mental health awareness and reduce the stigma around mental illness through national campaigns, television and social media49 has contributed to an increase in people seeking mental health support, in part a desired effect. However, variations in credibility of information sources and differing levels of education amongst the population with respect to mental health have contributed to misinformation and inappropriate use of mental health language at the point of referral to the services. There is also a lack of understanding of the role of society and a lack of or inconsistent guidance for advice and signposting services, in supporting positive well-being and mental health of CYP, leading to an increase in public perception of mental health services providing gold standard interventions or ‘fixes’. Thus, in recent years, the services have experienced a continuously increasing demand for assessments (and ‘a diagnosis’) for certain mental health conditions, for example, ADHD amongst CYPs and parents.

The lack of knowledge of (and support for) front-line staff (e.g. primary health care and school personnel) regarding mental health conditions and support pathways also contributes to the inappropriate use of mental health language and inadequate information provided on referral forms. The Site-10_Central referral point receives referrals from professionals and the general population and has a small team of clinicians who must triage each case before identifying the appropriate service. Having inadequate information on referral forms means that practitioners have to do a lot more information gathering from referrers to aid accurate decision-making, thus creating significant delays in the referral process.

Developing and establishing a common language with regard to mental health difficulties and education with regard to the role of society in supporting positive mental health is therefore crucial in managing some of the current challenges faced by the services. There is also a need for more interventions to help build resilience in CYP and manage emotions. To ameliorate the poor completion of referral forms, some services are in the process of adapting online referral forms to facilitate the completion of all required information. There is also a need to support front-line services, especially general practices where the bulk of the referrals originate, to ensure that referrals made are appropriate and have all the information needed.

Currently, there is a local transformation plan underway which aims to create more streamlined services for CYP. The plan is based on the THRIVE model and aims to create more seamless and closer working between Early Help/Social Care teams, primary mental health services (tier 2 services) and Secondary Services (tier 3 CAMHS). The plans also aim to introduce mental health specialists (link workers) into general practice to support under-18 services. Link workers will offer psychoeducation where needed and signposting and assessment advice to practitioners as well as promote mental health education amongst health practitioners within surgery. The implementation of the transformation plan has, however, been delayed by the COVID-19 pandemic.

The COVID-19 pandemic

The study commenced approximately 1 month before the earliest reported cases of COVID-19 in the UK (January 2020). The impact of the pandemic and ensuing restrictions on mental health services in general and the ICALM study are summarised in Table 15. During the COVID-19 pandemic, from SPQs and interviews, the services reported a sharp increase in mental health difficulties and associated levels of risk amongst CYP with (social) anxiety, low mood, self-harming, OCD-type behaviour, germ-phobic behaviour, emetophobia due to fear of sickness, fear or vomit and agoraphobia. Most services, including those delivering targeted mental health support and interventions, reported receiving referrals that were far beyond their capacity and service threshold with regard to level of need. This created major challenges in identifying suitable cases for IPC-A, with participating services reporting CYP to be more concerned with addressing anxiety than low mood.

We certainly opened it up to where young people were presenting with low mood and anxiety, we certainly did start trying to recruit people. I guess it’s when they’re clearly saying they’re getting tummy ache before school, and in school holidays they’re absolutely fine. It’s literally a worry about going to school. We’ve not offered it (IPC-A) to those young people because I think, you know, there’s not a feeling that their mood is a significant factor, it is really significant and very precise worry about school.

For most services, the COVID-19 pandemic also led to reduced capacity to deliver support or interventions due to high levels of staff sicknesses and school closures (and the need for homeschooling). During this time, managers reported having to prioritise the well-being of staff and continuity of service by not participating in activities that were seen as ‘extra’, including research (i.e. ICALM). For some Early Help teams, the focus of the service provision shifted to more safety planning for CYP at home due to the sharp rise in adult mental health challenges and potential safeguarding concerns.

We just had to be quite strict about how we prioritised our time management, all time during the pandemic. And anything additional we just had to be really clear that we weren’t going to take on, for our own wellbeing and therefore the continuity of the service. Nobody has had the capacity I think, or few people have had the capacity to do extra anything, and a trial ends up feeling like extra.

At the start of the pandemic, teams also had to adapt to more flexible ways of delivering services, whereby both assessments and interventions were delivered remotely via Zoom, MS Teams or telephone. By the end of the study, most services had adopted a blended model of delivery whereby assessments and interventions were being offered online or face to face depending on CYP preference or need. This mode of delivery increased accessibility, enabling the services to reach CYP living further from the service centres. The virtual delivery of the sessions worked well for some CYPs, with some schools also setting up rooms to support attendance. However, not all CYPs engaged with the virtual sessions due to a preference for face to face and/or increased severity of cases. The COVID-19 restrictions and shift to a virtual way of working also meant that teams had limited access to CYP homes, thus unable to pay close attention to more serious cases that would have ordinarily required intense face-to-face contact and multiple visits. This posed a challenge to the teams in predicting serious incidents, as some families stopped engaging with services or were not allowing professionals in their houses owing to COVID-19.

Policy

A challenge of implementing the different key mental health policies relevant to CYP set out in Table 16 which address some of the social contextual challenges highlighted above, has been a lack of specificity about how to allocate funds to different resources, including the costs for different staff resources. This leads to a disconnect between workforce strategy and policy objectives about meeting different young people’s needs. An additional challenge here is that workforce capacity-building strategies, which are often focused on training and recruiting less experienced clinical staff, do not typically include commissioning senior clinical supervision roles, with senior clinicians typically allocated to seeing high-priority children as a short-term strategy to reduce waiting times, rather than a longer-term view to capacitate the workforce to meet young people’s needs over time.

I think mental health services for young people is very under resourced to meet the need… And so, everybody is really [under] huge pressure, and this is why my service as well as many others are finding themselves picking up cases from the Site-10_Central Referral Pointwaiting list trying to kind of deal with that backlog but it’s often not the services that families want.

The lack of clarity about how to operationalise mental health policy at a service level is most recently evident in the NHS Long Term Plan, which has objectives to improve access to care by providing a 0–25 service and 24/7 crisis service. However, there is a lack of specificity about how to provide these services, how to implement these services, how many staff are required to support how many children and how senior clinical roles will be deployed to supervise new staff/trainees. Without such specificity, trainees will not be able to develop their skills to build the required specialist workforce over the longer term, perpetuating insufficient capacity across the whole mental health system to meet the ever-increasing demand.

Problems of capacity are not isolated within mental health services, and other service sectors are also overwhelmed with demand for care. As a consequence, young people have experienced being referred across multiple mental health, voluntary, social care and health services, driven by individual providers placing boundaries around which types of problems they treat and do not treat in order to manage that demand. This is reinforced by a lack of system strategy and shared definitions of mental health problems and conditions across services. Oversight, strategy governance and co-ordination of service provision within and across sectors are therefore critical for ensuring young people’s needs are met – a key gap highlighted in the 2017 Care Quality Commission (CQC) report.

A lot of the time it’s [volume of referrals] driven by process and system issues, and the fact that services are overstretched and they’re trying to move people on a lot of the time, and not rightly or wrongly.

I will slow think [Signs of Safety term] and I will not take anything on [accept a referral] unless I know my FSP can do it. If it’s just wellbeing [mental health], I won’t take it if it is not on a wellbeing pathway and if it’s not, then my job is to find out why not – talk to the GP, talk to CAMHS and get their input. If, following consultation, with mental health services, we can implement their advice, we will do that. Because once you’ve got their input, they’re responsible for the actions.

A central policy agenda has been to reduce waiting times in specialist mental health services. However, this agenda set within a context of overwhelming demand has arguably created a public perception that mental health services provide a gold standard set of interventions for young people that are worth waiting for, that all mental problems require specialist support and that the solution lies in enabling access to those services. This suggests that greater effort is required to promote the philosophy of mental health as ‘everyone’s business’,50 namely how effective treatment for mental health problems can be provided across the health and social care system as well as within schools and colleges.

I think we all have to take responsibility as a society don’t, we. We’re quite rightly say you have to stand up for your child’s rights, if children’s services or health are failing you, you have to bring that to someone’s attention. I guess as a society speaking as a parent you take a choice, do I do the tough job as a parent or do I give it to somebody and take away responsibility. I think we have an increased amount of that.

Within this context, the challenge of asking mental health services to implement the IPC-A intervention and feasibility trial becomes clear. While IPC-A as an adaptation of IPT is included within NICE guidance for CYP, it is not a priority of Health Education England, and clinical supervision for CWPs is not available. Targeted investment in IPC-A would be required if IPC-A was to be implemented as a treatment for mild to moderate depression, a significant challenge given the overarching problem of insufficient resources to meet increasing demand. Similarly, evidence-based mental health interventions such as IPC-A are not seen as core elements of Early Help services for CYP or competency/ training and development frameworks for youth services; hence it is difficult to introduce provision in this era of reduced capacity and funding alongside trying to manage increasing demand.

Meso-level contextual features of mental health service provision

The meso-contextual features of mental health services and their impact on ICALM are described below and summarised in Table 17. At a service level, the findings highlight an interaction between enormous demand, unclear service specifications, a lack of capacity and resources, a lack of oversight of service co-ordination and a lack of research prioritisation. The findings reinforce the need to develop an effective workforce strategy, without which services make decisions to minimise risk and manage resources, focusing on those young people with the most severe problems, resulting in young people with mild to moderate mental health problems not being treated or worsening. There is also a need to create more streamlined and co-ordinated service pathways to ensure that CYP are not being passed between multiple services.

Increase in complexity of cases initially classed as ‘mild to moderate’

The backlog at the Site-10_Central referral point led to the prioritisation of red and amber cases and long waiting times for green cases, for example, low mood and anxiety. The prioritisation of red and amber cases impacted referrals into the ICALM study by inhibiting referral of green cases into services which had IPC-A-trained staff. In some cases, the delay in obtaining support had led to CYP (and parents) finding other ways of coping or seeking help elsewhere. However, other green cases on the waiting list deteriorate, leading to increased complexity and severity of cases being referred to the services. This consequently leads to an increased need to escalate cases from Early Help teams to targeted services and from targeted services to tier 3 services.

With a large proportion of cases having an element of mental health difficulty, a major challenge for Early Help teams, who currently do not take the lead in delivering mental health support, is the delay in accessing help from tier 2 targeted mental health services. Most early help services, recognising the dangers of waiting lists due to the increased likelihood for cases to deteriorate and increase in complexity, have highlighted the need for transparency with CYP and parents about the likelihood of getting a therapeutic service and the type of support on offer (i.e. short term).

I’ve learnt from 20 years of doing the job that a waiting list is a very dangerous thing to have, because if you have someone on the waiting list who’s got low mood, and they’re tenth on the list, if someone comes in and they’ve got, you know, a more severe sort of need, they trump, so that person goes to 11, then they go to 12. Waiting list gives false hope, what happens in [the] meantime. Waitlists allocate a worker, which looks like they are receiving help when they are not and leads to inability to get other services.

If families really understood the state of mental health services in Suffolk are such that unless the needs are particularly high the likelihood of you actually getting a therapeutic service is very, very low. I think if families knew that perhaps they would look around to other types of service for support. But I think if they’re just put on a waiting list they’re just there with their fingers crossed just waiting for that phone call, which probably isn’t ever going to come.

The long waiting times for green cases and increase in complexity were some of the barriers highlighted by the services in identifying appropriate cases for the ICALM study. As such, interventions targeting ‘mild’ presentations, for example, IPC-A, became less helpful to the services owing to the range of mental health issues in young people, including some extremely challenging and/or dangerous behaviours towards themselves or others. The long waiting times also led to poor engagement following the implementation of a new referral pathway for green cases from Site-10_Central referral point to the services.

It didn’t work [new pathway] …out of the people that we identified that could be appropriate [for ICALM], I don’t think any of them wanted to do it or didn’t engage or couldn’t get through by phone. And that’s a large part of the problem when people have got wellbeing issues, they are more likely to be the people that are hardest to engage.

Restrictive or unclear service specifications and treatment pathways

The primary focus for Early Help interventions is safety, with health and well-being as a secondary focus. Therefore, apart from offering IPC-A (delivered as part of the ICALM study), Early Help teams’ mental health support primarily consists of promotion of mental health and well-being, confidence and motivation-building interventions and re-engaging with community networks and schools. According to the threshold matrix used by Early Help teams to assess whether referred cases fall within the Early Help service remit, CYP displaying signs of deteriorating mental health and episodes (e.g. low mood/mild depression or those with low-level mental health or emotional issues requiring intervention) should be classed as level 2 and are suitable for early help or targeted support services. This recommendation differs from what is happening at a service provision level, whereby Early Help teams refer cases with mental health difficulties, including low mood, to targeted mental health services or work alongside mental health professionals where advice is given on how they can help. The lack of standard treatment for mild to moderate mental health difficulties in some of the services therefore highlighted the challenges of running a RCT in this setting, as at times it was difficult to establish what TAU for low mood was being delivered by the services.

Early Help support is primarily delivered by non-mental health professionals from various backgrounds with basic mental health training; thus, they are currently not fully equipped to deal with a wide range of mental health difficulties. For example, apart from having access to online resources on mental health, some teams described only receiving a 2-day Signs of Safety training course aimed at equipping them to conduct Signs of Safety-based assessments. ICALM adopted a model which aimed to recruit and train Early Help practitioners, who are not professionally trained but work with mental health, to deliver IPC-A. IPC-A, being relational in its focus, should theoretically fit with the philosophy of the Early Help approach. However, the limited knowledge and expertise in teams that were not mental health focused presented challenges in identifying appropriate young people for the IPC-A intervention. Considering that a large proportion of the cases seen by Early Help teams will have some form of mental health difficulty, the lack of training and low involvement in tackling mental health challenges presents a missed opportunity that could help reduce pressures on mental health services, and indeed, Early Help services.

Apart from the limited role in supporting mental health difficulties, the service specification for Early Help is expanding to include more social service roles, for example, supervised contact of Looked After Children. There is also a current review of the role of FSP, which might lead to more social services roles, thus impacting the direct work the services do with young people and other interventions which are not part of their core offer, for example, IPC-A. This provision of both Early Help and statutory services, coupled with the reduced capacity of the workforce, had an impact on ICALM. Managers reported the challenges of protecting time for IPC-A-trained therapists while prioritising delivery of their core offer, which is not specifically mental health focused.

Due to the increased complexity of cases and the variability and subjectivity of assessment processes used, both Early Help teams and targeted services did not appear to be operating using a clear and/or up-to-date service specification with regard to mental health. Targeted services seemed to be more flexible and confident than Early Help teams in supporting CYP that would not ordinarily fall within their remit. These services described offering ‘holding interventions’ until appropriate help was identified through triage/patient flow/transfer meetings where senior managers from different services discuss cases that get stuck between the services. However, even with the transfer meeting, the teams highlighted that there are still gaps in the services whereby CYPs are not falling within any service remit or have a condition/illness which has no clear support pathway. The lack of clarity in referral pathways, which some teams have described as a ‘system level dysfunction’ is therefore a major challenge for the services.

Capacity

Capacity challenges in the mental health sector have been a long-standing issue which has been exacerbated by the COVID-19 pandemic. The numbers of staff varied between the services, with Early Help teams such as Site-01_Early Help and NEET Team having 14 staff members (including managers) supporting 390 families a year and targeted services such as Site-04_Wellbeing Service having 18 practitioners supporting approximately 1250–1500 cases a year. Factors driving the insufficient capacity include lack of funding and recruitment/retention challenges. The teams attributed some of the recruitment challenges to perceptions of some mental health trusts as ‘failing trusts’, perceptions of mental health as being under pressure and understaffed, the nature of the role, stigma around mental health and risk and poor job advertisements leading to audience misunderstanding of what roles involve.

A major challenge for targeted services was the poor retention of CWPs or PMHWs, who are often employed on short-term contracts, thus offering them no job security. This poor retention and high turnover were a significant barrier to the progression of ICALM, where several staff that had received training to deliver IPC-A either left the services or moved on to other roles. For the purposes of the ICALM study, these challenges especially affected services delivering targeted services (i.e. services with mental health support as their core offer), as only a small proportion of their staff members do not have a prior mental health qualification (CWP’s) and hence are eligible to train and deliver IPC-A. For example, Site-04_Wellbeing Service initially trained 12 CWPs to deliver IPC-A, but at the time of the interviews, only one practitioner remained in the service, with others either leaving or moving on to focus on a core profession. The high numbers of referrals also impacted on CWP’s capacity to deliver IPC-A, as they prioritised their primary role within the services, which was to carry out assessments with CYP, parents and referrers.

Conducting randomised controlled trials in non-specialist mental health services – the ICALM study research processes barriers

Given the paucity of RCTs in tier 1/2 services, where most cases of mild depression present in the UK,17–19 the ICALM study set out to address the need for evidence-based interventions in this setting by conducting a feasibility RCT of the IPC-A intervention. Our experiences with implementing the IPC-A intervention and conducting the ICALM feasibility trial highlight some key challenges and learnings (outlined below) in integrating this type of research into tier 1/2 services:

Research prioritisation

At a service level, process evaluation findings highlight a lack of research prioritisation compounded by an interaction between enormous demand for the services, a lack of capacity and resources and a lack of oversight of service co-ordination. Capacity challenges were then exacerbated by the COVID-19 pandemic, during which time managers reported having to prioritise the well-being of staff and continuity of service by not participating in activities that were seen as ‘extra’, including research.

We just had to be quite strict about how we prioritised our time management, all time during the pandemic. And anything additional we just had to be really clear that we weren’t going to take on, for our own wellbeing and therefore the continuity of the service. Nobody has had the capacity I think, or few people have had the capacity to do extra anything, and a trial ends up feeling like extra.

Randomisation

The capacity challenges faced by some targeted services posed practical challenges in establishing the delivery of two study arms, that is, for each service to have some practitioners solely delivering IPC-A and others delivering TAU. As such, some targeted services which offered interventions for low mood prior to participating in the ICALM study ended up collaborating with each other to deliver either IPC-A or standard care. For example, Site-06_Charity referred their CYP randomised to the intervention arm to Site-05_Charity, and Site-04_Wellbeing Service referred their CYP randomised to usual care to Site-07_Charity.

The lack of familiarity with research processes, in particular randomisation, was also a barrier to initial study set-up. Observations of the training and feedback noted that practitioners struggled with the concept of randomisation with some managers highlighting practitioners' concerns about putting CYP forward for the ICALM study, knowing that they might not be allocated to the IPC-A arm. It is also important to note that some sites (e.g. Site-03_Early Help and NEET Team) had been involved in the IPC-A pilot study, which was a single-arm study. Stakeholders in these sites felt that the pilot study worked well, as the referral process had been easier and more straightforward compared to the RCT.

Practitioners are struggling with randomisation and the idea that young people might get IPC-A or they might not. [Prompt: Do you think that your team’s involvement with the single arm study might have influenced this?] Yes, that was much simpler because they all got the intervention.

Another key lesson was the importance of how equipoise was communicated during the training to services who were not familiar with this type of research and, consequently, how practitioners were introducing the research to potential participants. Initial feedback following the training indicated that, although teams were mostly positive, some felt that the trainers had been overly positive about the success rate of IPC-A. The presentation of the training did include slides on the evidence base for IPT (i.e. first-line treatment in adults and adolescents for moderate to severe depression and persistent mild depression; IPT is better than waiting list/standard treatment and IPT equivalent to CBT) and IPC for adults (remission rates: IPC: 59%/antidepressant: 45%) as well as some findings from the IPC-A pilot study (RCADS) which could have been viewed as positive and might have impacted understanding and communication of equipoise. Following the initial training, and based on the initial feedback, changes were made to the training with emphasis placed on the young person.

Although we do not have access to detailed information about how the research was being communicated to potential participants, interviewed participants allocated to TAU and IPC-A felt that communication about the study, assessment processes and randomisation by the research team was clear and straightforward. Most young people allocated to the IPC-A arm were pleased and indicated that they had hoped for this outcome at the point of randomisation. Although some young people allocated to TAU felt disappointed due to the perceived novelty of IPC-A, most admitted being more concerned about getting the help they needed.

Challenges in establishing treatment as usual

One of the objectives of the process evaluation was to describe TAU for low mood within the participating services. Our findings, highlighting a variation in the types of interventions offered to young people with low mood, with some tier 1/2 services having no standard intervention for low mood, accentuate the complexities in setting up RCTs in this setting as some services were delivering both TAU and IPC-A while others only delivered IPC-A and referred young people randomised to TAU arm to other services (which were also participating in the study). The study team’s perspective at the start of the study was that young people were presenting with low mood at all services that were tasked with addressing the young person’s issues using their service mode and interventions, whether targeting mood or systemic issues. As such, IPC-A was well suited in all services to fit with their treatment philosophies. When under pressure from excessive demand, they resorted to screening out low mood in some services, for example, Early Help, as this was no longer seen as their core business.

If the young person’s issue is only low mood and they are randomised to the TAU then we refer them back to wellbeing specialists e.g., school nurses and wellbeing services and/GP practices. If they are randomised to IPC-A then whatever case they have should be within the early help remit i.e., low mood and other issues, and if the other issues can be dealt with by the Family Support Practitioners, then that’s ok.

Challenges in identifying eligible children and young people

Each team, including those within the same organisation, carried out their own triage and assessments for eligibility. Due to the variation in assessment tools used by the teams and the subjectivity of the assessment, it was difficult to establish whether a case classed as too complex in one service would also be classed as such in another service.

There was also potential for missing some eligible cases, as not all teams in the same organisation were involved or had capacity to deliver ICALM. Other managers carrying out assessments for other sites within the organisation were not aware of the referral criteria for the ICALM study and hence were not looking out for eligible participants who could have been referred to a Family Support Team. There was also a difference in how managers, within the same organisation but with different teams, understood and implemented referral processes, with some delegating the task to practitioners and others taking the task of screening referrals upon themselves and then referring eligible cases to practitioners.

I guess what you need is actually all the managers within our service to understand, you don’t necessarily need them all to be involved but they all need to understand that when a case comes in, they’re looking at it for potential allocation under those parameters to specific workers. So, for you to understand it, each locality and there’s six localities is split up into three teams. So, you have three managers for each locality and those managers are in separate localities, so it may have been lots of cases came into family support that would have been suitable, but they went to different managers, and ICALM managers never had sight of them in the first place. So, say 100 cases went into that locality and they were divided between those three teams only 33 cases would have been checked in your study, whereas 66 may have had them in and went out elsewhere. Logistically I think it was difficult for us, but I think it probably wasn’t set up quite right.

Description of sessions offered to treatment as usual participants

As discussed earlier, not all services had a TAU offer for depression/low mood, in particular teams involved in the provision of social care services who found it difficult to justify TAU for ‘depression’ cases. Therefore, in such services, TAU consisted of referrals to targeted services. As such, recordings of TAU were limited and consisted of three sessions from one therapist and young person. In these observed sessions, TAU used CBT-based approaches – focusing on discussing how the brain develops and how different parts of the brain may relate to the young person’s behaviour. This treatment aimed to retrain the young person’s brain to challenge anxious thought processes. The young people were introduced to strategies which challenged these thoughts, reduced their symptoms and improved their mood. Techniques used within these sessions included breathing and distraction techniques, keeping a diary, rating enjoyment from day-to-day activities and a ‘predictometer’ whereby the young people are asked to predict what they think will happen in a situation and then compare this to what actually happened.

TAU data were also provided by participating services in SPQs and interviews with stakeholders and one TAU therapist. Support provided to young people with low mood by participating in services included: promotion of positive mental health and well-being; emotional well-being and resilience building; CBT-informed work; psychoeducation; and work to develop coping strategies. Numerous services utilised guided self-help to support young person’s needs, for example, using behavioural activation approaches:

Our guided self-help for low mood intervention is mostly based around behavioural activation. So, when there is that lack of motivation, lack of energy, when they’re perhaps not doing as much as they would usually be doing, it’s about getting them to do more and to recognise that when they do more that energy, that motivation, that improvement in mood tends to follow on from that.

As described by young people’s interviews, TAU sessions most often consisted of eight weekly sessions, lasting from half an hour to an hour, and using virtual or face-to-face delivery. Descriptions from young people on the treatment they received give more insight into how TAU is perceived:

She talked to me about stuff and ways of coping with anxiety, and how to think of it in a different kind of way. Not completely negative and kind of facts instead of what I’m thinking.

It was basically, like someone that was there to listen, talk you through everything. It was kind of like having a really good friend, what I needed at the time I guess, like a friend that I could actually talk to. Talk about feelings. That was helpful.

I would say ‘hey, today I’m feeling quite sad’ and then she would go ‘I think that you’re feeling a bit sad today’ and that would be the end of the conversation really. [Laughs] I know how I’m feeling, I’m telling you.

In comparison, TAU and IPC-A interventions were similar in duration (6–8 weeks) and session length (up to an hour). With regard to content, while both IPC-A and TAU tended to encourage young people to develop strategies to improve their mood, TAU did not focus on interpersonal relationships to achieve this. Instead, TAU focused on the individual and strategies that young people could carry out alone to reduce their symptoms, for example, keeping a diary or breathing techniques.

Interpersonal counselling for adolescents’ intervention

The acceptability of interpersonal counselling for adolescents

Interpersonal counselling for adolescents was acceptable to both young people and parents, with qualitative data from interviews with four young people and four parents indicating that the intervention was seen to offer positive benefits for young people. Apart from low mood, initial symptomatic experiences of young people receiving IPC-A included isolation, social anxiety and agoraphobia. Following IPC-A, young people reported being more open to family, socialising with friends and being able to engage in ordinary activities such as going shopping, walking the dog, etc. Several parents reported seeing improvements in young people’s well-being after the second or third session, with one feeling that IPC-A had helped to ‘get their child back’.

She’s like a different child in a positive way… I knew it would take time, I know it’s not a magic wand, but we went from being very indecisive, worrying about what time I was coming home from work…to actually being my happy daughter again who would laugh about her indecision. I mean no tears at school…no getting a phone call ‘Can you come and pick her up, she’s so distraught’ nothing. Her friends have started coming here again…tomorrow we’re off to the cinema together, she’s not worried about the open space, and that somebody next to her might cough. So, for me it was like a miracle to be fair.

Structure and delivery

Interpersonal counselling for adolescents was delivered over six to eight 60-minute sessions, either virtually or in person, depending on need or preference of the young person. Key aspects of IPC-A, including its structure, duration and focus on specific areas, received both positive and negative feedback. Young people reported having a positive experience during the sessions and felt that the structure and duration of the sessions were important for providing stability. Some therapists also enjoyed delivering the intervention, with some highlighting the collaborative nature and the structure of the intervention as positive aspects.

I thought they were good [IPC-A sessions], especially the length of them I thought was really good because it was long enough to get something covered but not long enough that I was being drained afterwards. I could still continue with my day.

I absolutely love it… I loved the structure. Although there was some flexibility in it, but I liked the fact that we focused on one problem area because often in our service you have one area of problem and other things crop up, and you tend to dive away from the main problem area, so I like the fact that we made it quite clear that we’ve identified the problem area.

Other therapists and stakeholders, however, felt that the duration of the intervention (six sessions) was too restrictive, especially where young people needed more support. Other stakeholders, particularly those who were already offering other interventions for low mood, also felt that although IPC-A fitted well with young people experiencing specific problems in the four focal areas (i.e. big changes, relationship disputes, grief and loneliness/isolation), the intervention was too narrow as low mood can be impacted by a lot of other factors. These stakeholders felt that selecting a particular area of focus was difficult because young people are changeable each day and struggle to make decisions.

So, your low mood can be impacted by the fact that you’re not eating properly. That you’re not sleeping properly…. we’ve got kids who are sitting on our list, or were sitting on our list, who have multiple things going on. So, when you tell a young person to pick one, they’re going to go ‘What do you mean, I don’t even know which one is the biggest issue today’.

This feedback raises an important point about how the IPC-A model was understood by some stakeholders/managers. The IPC-A model is designed for the practitioner and young people to work together collaboratively in selecting a focus area rather than asking the young people to pick an area. This misinformation around the model of IPC-A might have also contributed to recruitment issues, especially where managers were responsible for screening referrals.

Issues around the structure of IPC-A sessions and their impact on implementation and theoretical fidelity are further discussed below under Implementation and theoretical fidelity.

Content

Three young people felt that the content of IPC-A had contributed to the improvement of their symptoms and helped them to have a better outlook, of which, two described IPC-A as helping them to focus on more positive rather than negatives aspects of their lives. However, one young person who felt that her condition had not improved much since IPC-A, argued that the intervention focused too much on positive aspects, without giving space to understand the negative aspects and support the young people through it. The feedback from this young person highlights the complexity of delivering brief, focused interventions.

I know it sounds kind of weird to say it this way but it’s like they’re focusing too much on the good. They need to try and break down the bad… for example, if I was to say I’ve had an emotionally bad day but there have been a few good highlights, usually my counsellor will talk about mostly the good and a little bit about the bad at the very end. And then of course that would leave me feeling down afterwards. So, it would be better if they were to start off with the bad, go into a bit of detail about it so they’d understand my position and how it affected me during the entire day and how it made those little good things that might not be good things for other people, but may feel like more to me and make me feel a bit uplifted when I leave the session.

The young person also reported doing weekly tasks/goals outside the sessions, such as keeping diaries of positive/negative experiences and speaking to their parents or head of year as a way of developing positive relationships. While some young people liked having goals, others reported struggling to complete tasks. Apart from noticing young people doing some tasks at home, most parents were not aware of the contents/discussions of the sessions. However, one parent reported being involved in the sessions, initially at the request of the young person who needed their support in explaining previous experiences. This parent also reported receiving helpful support from the therapist with regard to how best to support the young people at home.

If I’m finding something particularly difficult as well, obviously I say to [name of young person], ‘Oh can you just ask [name of therapist] to grab me for two minutes at the end, there’s just something I need to …’ and I’ll have a chat with [name of therapist] and say, ‘Look, something’s occurred, I don’t know if [name]’s mentioned it, but which is the best way to go?’ And then [name of therapist] will say, ‘Oh I’ll send you an email’, yeah. I’ll leave it a few days and I’ll say, ‘Oh [name of young person], [name of therapist] sent these through, and we need to have a try at these’ and we’ll read through it and whatnot. And then quite often that’s just what I need to be able to broach whatever it is that I’ve got niggling, concerning me, because I’ve noticed it.

Suitability for the services

Some services already delivering interventions for low mood felt that IPC-A was a useful alternative approach for low mood that enabled them to provide the most adequate support for that young people tailored to their need. However, the structure of IPC-A and the need to choose an area of focus were perceived to create some tension in other services whose offer was more general.

It [IPC-A] does involve quite specific work, methodology. And our specialism is much more around meeting young people where they are. We have a range of different kinds of ways of working, including things like cognitive behavioural based things or a creative way of working … It’s not always easy for us to hold a very specific offer within our specialism which is a more general offer as opposed to taking them through a very prescriptive method. There are so many agendas imposed on young people currently from school and other parts of their lives the feedback we get…is that they felt that they were listened to and met exactly where they were. So, that’s our primary offer.

Due to the challenges faced by the services in identifying suitable young people and delivering IPC-A, some Early Help managers felt that interventions such as IPC-A would be more suitable for services set up to tackle mental health, such as U18 Well-being Service and school nurses. Others felt that introducing early interventions such as IPC-A in schools would be a better approach, as Early Help usually refers CYP experiencing mental health difficulties to emotional well-being services or school nurses. Some stakeholders expressed the challenges of implementing an individualised intervention such as IPC-A in a service dedicated to supporting families as a unit. These stakeholders felt that, due to the level of young people engagement required, offering IPC-A at the beginning of a case when FSPs are dealing with the whole family and working to develop an intervention plan that has more to do with safety than therapeutic need was challenging. During the study, Family Support Teams resolved this challenge by ensuring that IPC-A was delivered by practitioners who were not case-responsible (i.e. responsible for the family unit). Practitioners felt that in Family Support Teams, IPC-A could work better if offered at the ‘tail end of a sequence of interventions’ (Stakeholder-04, Site-01) when young people are in a more stable situation and environment. Alternatively, some stakeholders felt that IPC-A could be better suited in a school-based setting where ‘young people are worked with individually, and they have that safe space with a professional where they feel safe’ (Stakeholder-04, Site-01).

We’ve found it’s better used when you’re not case responsible. So, if I’ve got a colleague who is case responsible for a family if – so [name of IPC-A trained therapist] was one of our key people, if say my colleague [name of Family Support Practitioner] is working with a family and he identifies that there’s low mood and depressive symptoms in an adolescent he would be able to refer to [name] and then it’s more effective because the roles aren’t blurred. Otherwise…we have tried being case responsible and doing the IPC and it can just be a bit confusing.

We could probably put IPC as an intervention for every single child on our caseload, but it probably isn’t an intervention that the child is ready to engage with because there are so many other pressing issues on the intervention plan. And it’s often a result of either other family members or other domestic situations that are going on with that child that is impacting and causing their low mood or their poor mental health.

Implementation and theoretical fidelity

Fidelity ratings and contamination

Therapists submitted recordings of their sessions to their supervisor to be rated; these sessions included the initial, middle and final sessions (three total). Items were rated on a 0–3 scale: 0 = not attempted, 1 = attempted but incomplete and/or superficial, 2 = completed to a good standard and 3 = completed to an excellent standard. An adequate range of IPC strategies must be demonstrated over training, and for each recorded session, an average score of two or above (for all rated items) was needed for a recording to pass. These ratings indicate how well therapists adhered to the IPC-A manual, which prescribes tasks for each session – differing between initial, middle and final sessions.

Tasks in the initial sessions review the young person’s symptoms, explain depressive symptoms and describe depression as an illness – and not a fault of the young person. The young person’s current coping strategies are also explored, and an interpersonal inventory is conducted. This interpersonal inventory aims to uncover what is going on and who is important to the young person’s life. The therapist and young people may then work together to agree on a problem area to focus on in future sessions by identifying what may trigger and maintain the young person’s depressive symptoms. Middle sessions also involve a review of symptoms and lead to tasks and techniques which are specific to the agreed-upon problem area. The young person’s progress with goals made in each session should also be reviewed and new goals for the next session agreed upon. Final session tasks include reviewing the young person’s progress and improvement by comparing their symptoms and goals across sessions and at the start of IPC-A. Dependent on this progress, the therapist must discuss the possibilities of what should be done once IPC-A has finished. These possibilities range from discharge from the service, maintenance work or referral to more specialist services.

Treatment-as-usual sessions were also rated to assess if IPC-A training led to contamination. No clear systematic use of the principles of IPC-A was identified – suggesting no contamination effects from IPC-A training for the observed TAU therapist, although it is not possible to generalise this with any certainty; see Table 18 for mean ratings for each case, IPC-A and TAU.

Interpersonal counselling for adolescents’ sessions: thematic analysis

Both therapists' and young peoples’ approaches to IPC-A were further explored using a multimethod qualitative approach to identify patterns within the data. Experiences of therapists, young people and parents involved in the ICALM study were considered alongside IPC-A sessions – ‘cases’ refer to the therapist and young people dyads within these sessions; see Table 19 for overview of each case. Thematic analysis was used to explore both session and interview data. Conversational analysis was then used on session data to support findings of the thematic analysis.

One main theme emerged from the data relating to the role of therapist and young peoples’ and interactions between the two in effective delivery of IPC-A: engagement from young people as crucial for successful, collaborative intervention. Within this theme, several subthemes were developed which consider the mechanisms and contextual features of IPC-A sessions and how these can be utilised to encourage engagement and progress for the young person. These subthemes were interactional styles and participation within intervention sessions; importance of feeling heard; orienting discussions to young person’s perspective; and reliability of interpersonal relationships: mini goals and advocacy. It is worth noting the difficulty in defining what is a mechanism of action and what is a contextual feature of IPC-A sessions. Using these terms, we aim to highlight components which are important to the successful delivery of IPC-A. See Table 20 for a summary of findings and recommendations.

Context for implementation and theoretical fidelity: IPC-A training and supervision

Throughout the findings, the impact of IPC-A delivery on implementation and theoretical fidelity is considered. All therapists in the ICALM study took part in IPC-A training, in which they were trained on how to deliver IPC-A to young people within their service. The main focus of training was to introduce therapists to the structure of IPC-A, followed by supervised practice. Within IPC-A, there are initial, middle and end phases – each of which considers different tasks. During training, therapists are taught the components of and outcomes for each session across each phase. Middle phases require different components depending on the problem area agreed by young person and therapist; however, general IPC-A techniques are utilised across each problem area – all of which are addressed in detail during training.

Perceptions of training were positive from most therapists, with some finding it affirmed their current practices:

It’s done in a different way but ultimately, it’s the same sort of process in the way of offering that young person that containment and trying to stick to the worries. So, yes, but I still got a lot out of it because I felt it was reaffirming some of what I did.

Training was also described as containing a lot of new information, leading to the suggestion that some perceived training as ‘dense’.

I mean it was two full-on days. I felt there was a lot of jam packed into two days. I did feel a little bit like a rabbit in the headlights when we’d finished. It was like crikey I really don’t know what’s going on now. It was quite daunting. And I thoroughly enjoyed it I have to say.

I think it would benefit possibly at certain stages for a little bit more participation activities from the trainees, because at certain points there’s quite a long period of information being taken on board.

I remember it feeling like it’s been long day and it’s a long day of a lot of talking as well. So, it’s not like it’s a long day with right let’s do some break out things, it was just long days of being talked at. So, I did find that really hard at times.

Density of training and time between training and delivering the intervention appeared to lower therapists’ confidence in being able to deliver IPC-A as described in their training. Many therapists felt they needed to revise their training closer to the delivery owing to delays with referrals and disruption owing to COVID-19 – for whom refresher sessions were arranged.

I mean, the training was good. I’m a person that if I’m not using it, I tend to lose it. Yeah, so it was always agreed that when we have a case, then we’d review that training, so we knew where we were kind of at.

I was worried actually it’s a very busy world when you’re doing our jobs and my fear was when I’m going to get that referral in and start working with that young person there’s a danger that I’m not going to be able to remember because it’s quite scripted in a way to deliver.

This latter quote supports the suggestion that this perceived ‘density’ of training and structure of IPC-A may have led to therapists feeling pressured to deliver the IPC-A intervention according to its structure rather than flexibly adapting it to the young people's needs. This therapist goes on to describe the structure of IPC-A as detrimental to their delivery and ability to build rapport with the young people, owing to the pressure of sticking to the structure.

And especially that being the first session as well, you’re beginning to build up that relationship, but at the same time you’re thinking, oh, my goodness, I’ve got – I’ve done this, have I ticked that, have I done that?

This tension between delivery of IPC-A according to its structure and engaging with the young person’s needs highlights a tension between implementation fidelity and theoretical fidelity. In this case, there appears to be a focus on implementing the intervention as trained (implementation fidelity) over the young person’s needs and desired outcomes from the intervention (theoretical fidelity). Therapists are told within their training that the needs of the client are more important than the needs of the IPC-A. However, therapists’ delivery of these IPC-A sessions is also supervised and subject to feedback from their supervisor. Therapists have weekly supervision sessions alongside their IPC-A sessions, in which to discuss cases and any areas of difficulty. Supervision is seen as ‘an extension of training where therapists continue to learn and are helped with practical implementation of strategies’ (IPC-A Trainer). Once IPC-A sessions have finished, supervisors rate therapists’ IPC-A sessions using the rating system described above (see Fidelity ratings and contamination). These ratings decide if the therapist can pass their IPC-A training, with each therapist needing to pass two case studies to complete their training. Given this context, implementation fidelity may be given priority by therapists, potentially undermining the mechanisms of change the intervention is designed to activate.

Supervision sessions were seen as invaluable by therapists, many of whom were not confident in their delivery of IPC-A – partly owing to the structure and amount of content to be delivered during IPC-A sessions. Through discussion with their supervisors, therapists found validation in their delivery and affirmation that they are delivering IPC-A as needed.

But I just wasn’t confident enough in that session to say I think this is what the problem area is. So, I went away, took some supervision. Had my supervision and I should have gone with my gut feeling because I was absolutely right.

Increased confidence in their delivery meant that therapists were able to avoid fixating on delivering IPC-A according to the manual and be more flexible in their approach, for example, listening and reacting to the young person’s needs. Guidance from this weekly supervision also allowed therapists to increase their confidence by better understanding young person’s needs through the IPC-A model. Therapists’ perceptions of supervision illustrate the important role it plays in facilitating therapists’ confidence to deliver IPC-A not only well but potentially at all.

And I couldn’t have continued without it in all fairness because I need that for my confidence as well to make sure I was doing the right things.

She was like my little personal cheerleader for someone to come back and go to ‘I think I did that right, but I’m not sure’, and she’d be like ‘oh yes. And you’ve done this, and you’ve done this’, and yes, I needed [supervisor’s name] in a box, I think, to take around with me.

Supervisory meetings play an important role in facilitating both implementation and theoretical fidelity by validating therapists’ delivery, increasing their confidence and providing constructive feedback when therapists need redirecting, ‘lots of lovely constructive feedback, and also the feedback when things didn’t go so well, or where I wasn’t sure what I needed to do’ (Therapist-IPC-01, Site-08). By increasing therapists’ confidence in their delivery, supervision allows therapists to avoid focusing on the structure of IPC-A to consider the young person’s perspective and needs – a crucial element in successful interventions. Had therapists been able to take on a higher number of cases, it had been anticipated that confidence in delivering IPC-A would have increased.

Engagement from young people as crucial for successful, collaborative intervention

A key component of IPC-A is collaboration between the therapist and young person. Collaboration is highlighted throughout the IPC-A manual and training and is used as a tool to help young person identify problems and solutions; for example, what their triggers are, how they can recognise the onset of symptoms and find strategies to ease their symptoms. For collaboration to occur, both the therapist and young people need to actively engage with the sessions and tasks. This need for engagement is also reflected in parent and therapist perspectives that young people must be ready to seek help and engage with support before starting these sessions and for support to be effective (see e.g. Appendix 1, Table 22 data extracts).

These perspectives placed young person’s motivation to participate in the intervention as a central component for ensuring effective delivery of the intervention. However, it raises the question of how active engagement might manifest within the sessions themselves and what therapists might do to help young people actively participate in discussions. One of the most beneficial aspects of accessing therapy, as perceived by parents, therapists and young people, was the ability for the young people to talk and have someone listen without judgement.

In contrast, if a young person does not feel heard, there may be a sense that engaging with interventions is pointless, as discussed by one parent describing responses to their child trying to find support in school: ‘That just made his mood just go even lower because he thought what’s the point. If I’m not going to be listened to then I’ll just carry on as I am, and therefore he then wouldn’t talk’ (Parent, Case 2). This capacity for young people to feel heard, and that their voice matters, can start before deciding whether to take part in the ICALM study, as seen here in a discussion between the young person, parent and the young person’s school about support options available to them:

Interactional styles and participation within intervention sessions

Most apparent throughout recorded IPC-A sessions were different interactional styles between therapist and young people, representing different ways that young people respond to the IPC-A intervention. These can broadly be categorised as (1) active participation, young person works with the therapist to find solutions; (2) active resistance, young person resists attempts by the therapist to collaborate, often followed by renegotiation; and (3) passive participation/resistance, young person defers to therapist. Passive participation may reflect an inability to engage with the session or not knowing how to participate, rather than choosing not to engage. For example, the young person may express that they ‘don’t know’ as a form of resistance or an inability to participate owing to their memory or difficulty of the task. It is up to the therapist to navigate these interactional styles to help the young person to engage with IPC-A and improve effectiveness of the intervention.

Contrasts and variations in interactional styles seen across and within sessions for each case suggest that the effectiveness of IPC-A relies on mechanisms which allow the young people to feel comfortable talking and sharing with their therapist. Exploring how and where these interactional types emerge and change throughout IPC-A sessions offers insight into how contextual features and the tools of IPC-A can be used to encourage active participation from young people. One of the main components in encouraging young people to engage with sessions appeared to be therapists’ ability to give the young people space to not only talk but also feel heard, resulting in increased rapport between the two and facilitating the collaborative effort needed for effective IPC-A.

The importance of feeling heard

During their initial session, the therapist and young person in Case 1 illustrate how active participation from the young people facilitates the therapist’s understanding of their symptoms and experiences, enabling a collaborative exchange that leads to them identifying a problem area (‘big change’) to focus on in future sessions. This level of participation was later reflected on by the young person who reported, ‘I cried quite a lot I think, just like for the first one probably when I was just talking about just everything’ (YP-IPC-04, Case 1). An extract from their first session (see Appendix 1, Table 23) illustrates how the therapist works to show the young person that they have heard and understand them – a key mechanism which enabled this young person to actively participate in sessions.

However, to better understand the importance of feeling heard in facilitating active participation from young people, we needed to examine sessions in which young people were more resistant or unable to engage with therapists’ attempts to deliver the intervention session. In Case 3, the young person reported feeling misunderstood by their therapist and the frustration that came with this:

[Interviewer] And how did you get on with the therapist that you were seeing?

[YP] OK. I think I got a bit frustrated with her sometimes but overall, she was all right in the end.

[I] What frustrated you at the beginning then?

This perception of their sessions may also be reflected in ratings of techniques used by the therapists in Case 3 and Case 1 (and Case 2). For example, Table 7 shows that in the initial session, the therapist in Case 3 averaged lower than the therapist in Case 1 (a score of 2.1 vs. 1.4, which are a pass and fail, respectively). Techniques in which Case 3’s therapist scored lower included not asking the young person about their variation of symptoms (e.g. changes across the week) and creating links between young person’s interpersonal relationships and depressive symptoms. This lack of exploration of the young person’s symptoms may have led to feeling misunderstood, as the complexity of what they had been feeling may not have been fully addressed. Examples of this can be seen in Case 3’s first session, which provides examples of interactional difficulty or tensions between the young person and therapist, locating instances where the young person may have felt misunderstood, or unheard, and the interactional style this led to. These interaction styles included passive participation within the sessions or active resistance towards participating (see Appendix 1, Table 23).

Therapists are rated on their use of exploratory techniques which should be used to actively encourage participation. By showing a sense of curiosity and acknowledging a young person’s contributions, therapists should encourage the young person to expand on productive topics without imposing unnecessary structure. In this case, the therapists’ focus on the structure of IPC-A (and implementation fidelity) and imposition of this may have limited their acknowledgement of the young person’s contributions and exploration of why the young person is unable to describe where their feelings are coming from.

Active resistance is also demonstrated by the young person in Case 3’s initial sessions, which allowed us to explore the difference in context between active and passive resistance. One way in which active resistance appears to be expressed differently from passive resistance is that the young person does not only respond using one-word replies but instead actively responds to the therapist by justifying their reasons for not wanting/being able to engage in an activity. In the observed context of Case 3, this actively resistant interactional style may be prompted by feeling more than misunderstood but also not believed in or even judged by their therapist. This approach, or perceived approach, by the therapist also contrasts with the intended implementation of IPC-A, in which the therapist’s role is to be non-judgemental and on the young person’s side. Further, the rapport between therapist and young person may suffer owing to the misunderstanding in communication between them.

These points of contrast across cases, illustrating varying forms of participation and interactional sequences, reveal the central importance of therapists working to understand and acknowledge young people’s perspectives and the crucial role played by differing interactional styles for facilitating young people’s engagement, with a non-judgemental approach allowing young people to feel heard and share openly without having to defend or justify their disclosures. These points of contrast also illustrate the complex relationship between implementation and theoretical fidelity, in which both may impact the other for better or worse. For example, implementation fidelity does not always ensure theoretical fidelity.

Orienting discussions to a young person’s perspective

One of the most important mechanisms in enabling young people to actively participate in sessions appeared to be the therapists’ ability to be flexible and work to reframe discussions or renegotiate terms in a way that the young people can relate to – orienting discussions to the young person’s perspective.

Therapists’ initial understanding of the young person’s experiences is reliant on how the therapist elicits the young person’s perspectives and how the young person chooses to answer. Depending on the approach taken, misunderstandings may manifest between therapist and young person, creating interactional tension between the two parties. Misunderstandings could also lead to the young person not feeling heard, despite not having shared this information. In Case 1, for example, tension is seen during the initial session when the therapist misinterprets the young person’s relationship with their mum – interpreting their relationship as less close than they are. The young person responds to this by elaborating that their relationship is very close, leading the therapist to readjust their interpretation of the relationship to accommodate this new understanding; in particular, that the young person’s mum is supportive and listens to how they are feeling. See Appendix 1, Table 23 for an example extract showing the consequence of the therapist’s adjustment, with the therapist changing tack.

Listening to the young person’s perspective and reframing the conversation to acknowledge their perspective appears to help the young person actively participate, despite what appears to be a difficult task which requires pauses for thought. Focus then turns to how the young person can change their interactions to discuss their feelings more deeply, for example, going beyond what their feelings are to what has made them feel that way. The young person actively responds to this approach, showing their understanding and the value they see in it – rather than passively accepting the suggestion.

This interaction is followed by an offer to role play this conversation, which is turned down by the young person but later scripted in another session. In this later session, the therapist shows their awareness of young person’s hesitance around ‘role play’: ‘I don’t want to call it role play but it is a little bit like that, just to see how you manage to open up that conversation’. The young person is encouraged to read their script from the screen and have the therapist be their response – rather than starting the conversation on the spot. Again, this therapist can be seen to orient to the young person’s perspective by showing their awareness of issues around ‘role play’ and helping them to script what has been shown to be an uncomfortable, hard conversation for the young person to start. The effectiveness of this technique is seen in the young person’s ability to go on to complete their mini goal. Upon completing this mini goal, the young person is met with praise from the therapist and acknowledgement of its difficulty, ‘I think the more you avoid those emotionally charged situations, the more difficult it becomes for you to then approach those situations and those subjects. So, the fact that you managed to do that, that’s huge, that’s absolutely massive’ (Therapist-IPC-01, Case 1).

Despite the effectiveness of role play, young people appear resistant to trying it out, perhaps owing to their familiarity with talking to the chosen person or the discomfort of improvisation – which scripting conversations appear to relieve. Role play is also offered in Case 2 and again is turned down. The same therapist shows their awareness of hesitation around role play by offering it as a last resort, ‘if you find that you can’t, then what we can do is perhaps do a little bit of – I know this sounds really scary – but a little bit of role play and we can try and sort of act together’ (Therapist-IPC-01, Case 2).

The importance of orienting discussions and sessions to young person’s perspective is also reflected in Case 3. In comparison to the previous case, feelings of misunderstanding for this young person may stem from their inability to relate to the term ‘depressive symptoms’, as they do not think it fits with what they are feeling. Failure of the therapist to recognise and respond accordingly to the young person’s perspective leads to the young person feeling misunderstood, resulting in passive participation from the young person (see e.g. Appendix 1, Table 23).

Initially, the therapist’s response appears inadequate for orienting to the young person’s point of view and enabling the young person to actively engage with the session. This failure to explore and reframe the discussion according to the young person’s perspective appears to result in the young person struggling to actively engage and instead showing passive resistance. This passive resistance is illustrated in their use of unmarked tokens,58 such as ‘yeah’, ‘mhm’ or ‘don’t know’. This interactional style changes within the session when the therapist recognises the issue and renegotiates the terms used with the young person – trying again to orient to the young person’s perspective and successfully enabling the young person to actively engage with the task (see Appendix 1, Table 23).

This renegotiation of terms also highlights a potential tension between implementation and theoretical fidelity for therapists. Implementation fidelity, for example, might encourage the therapist to continue discussing depressive symptoms, owing to the theoretical focus of IPC on the interactional relationship between depressive symptoms and interpersonal relationships. However, for this young person (Case 3), continuing this approach would be inappropriate and result in resistance throughout the sessions and little progress for the young person. Changing these terms shows the therapists’ ability to adapt flexibly to the needs of the young person, helping them to engage and progress with the intervention, and in turn, enhancing theoretical fidelity. Despite the main diagnostic category for the study population being depression/low mood, the therapist’s role is not to diagnose young person with depression. A key mechanism of change within these sessions is the therapist’s ability to work flexibly with the young person to collaborate and work towards a solution on their terms.

Contextualising IPC-A using specific scenarios, relevant to the young person’s experiences, was also seen as another mechanism which involved orienting to the young person’s perspective and encouraging this young person’s (Case 3) engagement. By contextualising IPC-A in this way, the therapist was able to gain information from the young person about their interpersonal relationships, which had proved difficult owing to the young person’s difficulty engaging with previous tasks, for example, creating a timeline of their symptoms.

An important mechanism of IPC-A, therefore, appears to be in the therapist’s ability to orient themselves to the young person’s perspective and reframe discussions accordingly – often requiring flexibility with the structure of IPC-A sessions. Beyond this understanding of the young person’s perspective, therapists must also encourage the young person to change their approach and engage with IPC-A techniques.

Reliability of interpersonal relationships: mini goals and advocacy

Many, if not all, tools of IPC-A rely on active participation from the young people. Most prominent of these are mini goals that are set at the end of each session for the young person to complete before their next session, where they will be reviewed. In accordance with the IPC-A manual, these goals should be agreed on collaboratively, interpersonally and as something the young person has control over. The aim of these goals is to be achievable and help the young person towards their ultimate goals.

Responses to and success in these mini goals can affect the young person’s future engagement with mini goals outside of the sessions. In the cases seen, mini goals were often reliant on others responding appropriately to the young person’s attempts at completing these goals. Engaging in mini goals and not receiving the response they need or expect can be disheartening for the young person and could lead to them feeling less motivated to engage with IPC-A outside of (or within) sessions. This is best illustrated in this example from Case 3, in which the therapist and young person are reviewing the young person’s mini goal to talk more deeply about how they are feeling with their mum (see Appendix 1, Table 23 for extract).

Use of mini goals and advocating for the young person can help to better develop the rapport between the therapist and young person when used effectively. For example, in creating mini goals the young person and therapist make an agreement that the young person will engage with the task outside of the session, and the therapist will follow up with this task in the next session. Following up is a key element of this tool, as much like feeling unheard, if the young person attempts the task but is not asked how it went, they may have little motivation to attempt these mini goals again. Advocating for the young person may also work to reinforce the therapist’s words with actions, going beyond saying that they are listening to their needs to taking steps towards making sure these needs are met by others, where possible. In Cases 2 and 3, both therapists take the opportunity to advocate for the young person by discussing the young person’s needs with important interpersonal relationships (parents and teachers) without revealing details of their sessions.

In Case 2, given the outcome of the previous mini goal, the therapist emphasises the need for the young person to find the right moment to carry out their mini goal: ‘I would like you – if you can – I think it would be a really good idea to, when you’re nice and calm, is to go and visit [teacher’s name] or ask to see him at a certain time’ (Therapist-IPC-01, Case 2). Reframing the context of the mini goal to include finding an appropriate time may foster further consideration of the other person’s perspective, leading to a more balanced view of relationship disputes. The therapist also offers to extend their reach to act as an advocate for the young person to ensure that responses to their reaching out are treated appropriately, in this case by talking to their teacher beforehand in preparation for their chat with the young person. Again, therapist’s ability to act as an advocate for the young person may help the young person feel supported by both their therapist and interpersonal relationships. The latter is crucial to IPC-A’s theoretical fidelity and young person’s progress once their IPC-A sessions have finished. See Appendix 1, Table 23 for an extract from Case 2 which highlights the change in response to the young person's reasons after the therapist is able to advocate for them to their parent.

A key component of the IPC-A is the two-way link between depressive symptoms and interpersonal relationships. Improving young person’s interpersonal relationships through effective advocacy should lead to improved depressive symptoms. Discussing expectations around these interpersonal relationships also appears to be an important tool in understanding the young person’s perspective on IPC-A and how it can work to help them.

Effective use of mini goals and advocacy can change interactional styles and help young people to actively engage in IPC-A, both within and outside of sessions. Use of these tools adheres to the intended implementation of IPC-A and can be seen to foster the theoretical ideals of IPC-A, allowing young people to further develop their interpersonal relationships and the support received from these.