Aims, design and theoretical frameworks

This chapter sets out the overarching aims and objectives of the study. It describes the overall design and theoretical frameworks on which the study builds. The study is set out in three phases, the detailed methods and results of which are outlined in Chapters 3–7.

A full protocol56 for the study was published early in the study process following the guidance provided by the Standards for Reporting Implementation Studies (StaRI) statement.71

Overall study aims and objectives

The overall aims of the study were to identify critical factors affecting the implementation of an online-supported self-management intervention for relatives of people with recent-onset psychosis or bipolar disorder into routine clinical care, and to use this information to inform an implementation plan to facilitate widespread use and inform wider implementation of DHIs.

The objectives were to:

• measure the uptake and use of REACT by NHS EIP teams and relatives
• identify critical factors affecting implementation of REACT
• identify resources required (and cost implications) for successful implementation of REACT in EIP teams
• investigate the impact of REACT delivered by EIP teams on relatives’ self-reported outcomes
• develop a user-friendly REACT implementation plan and related resources to facilitate widespread use and dissemination
• use the findings from this study to further develop theories of implementation of digital interventions in real-world practice.

Study design

This study used a theory-driven multiple case study design,72 taking a mixed-methods approach that integrated quantitative assessments of outcome (delivery, use and impact of REACT) and qualitative assessments of the mechanisms of implementation through observation, document analysis and in-depth interviews. The study was theoretically informed by normalisation process theory (NPT)73 and the non-adoption, abandonment, scale-up, spread and sustainability (NASSS) framework.74

Our cases were six NHS trusts in England. Case studies can be particularly useful when trying to understand the implementation of a complex intervention in a real-world setting in which the process or context cannot be controlled. REACT was a ‘complex intervention’75 because it depended on the actions of individuals across different contexts and individuals adapting their behaviour over time. The intervention also produced multiple outcomes that needed to be understood. Implementation was made more complex by the dynamic context in which the intervention was situated, with competing demands on the system. A mixed-methods approach bringing together quantitative and qualitative assessments was therefore required to understand this complexity.

We also designed the study to have extensive input from stakeholder groups at each trust to ensure that the implementation plans were collaboratively designed and refined.

Theoretical frameworks

The IMPART study was theory driven while remaining strongly embedded in practice. Theory helped to guide our data collection and to frame our analysis, ensuring that we were drawing on previous learning about what was likely to be important, but also contributing our findings to a body of knowledge that could inform other studies.

However, we were careful to use theory only where appropriate, and actively seek data that did not fit our theoretical frameworks. There were many models, frameworks and theories that we could have used to guide our work and ensure that our findings could be interpreted within a theoretical framework that supported their potential to be generalised.76 Our aim was to choose those frameworks that were most appropriate for each of our aims and the context in which this work took place.

Normalisation process theory is a theory of action that focuses on a person’s actions rather than individual attitudes and beliefs (www.normalizationprocess.org; accessed 6 December 2019). NPT began as a model (normalisation process model) of the factors that promote or inhibit the routine work of embedding a new health technology into practice. The key constructs identified were interactional workability, relational integration, skill-set workability and contextual integration. The model has since been developed into a theory that includes the normalisation process model as constituting ‘collective action’ and adds concepts of ‘coherence’ (how actors make sense of a set of practices), ‘cognitive participation’ (the means by which they participate in them) and ‘reflexive monitoring’ (how these practices are then appraised).77 Each construct represents a generative mechanism of social action. That is, each construct represents the different kinds of work that people do as they work around a set of practices, whether these are a new technology or the trial of a complex intervention (Figure 2).

FIGURE 2 Model of the components of NPT. Reproduced from May C, Finch T. Implementing, embedding, and integrating practices: an outline of normalisation process theory. Sociology 43(3), pp. 535–54.73 Copyright © 2009 by BSA Publications Ltd. Reprinted by permission of SAGE Publications, Ltd. (Figure not reproduced here; see the original article.)

Normalisation process theory is concerned with three core problems:

1. Implementation – the social organisation of bringing a practice or practices into action. This is defined as a pattern of organised, dynamic and contingent interactions in which individuals and groups work with a complex intervention, within a specific context or health system, over time.
2. Embedding – the processes through which a practice or practices become (or do not become) the routine everyday work of individuals and groups.
3. Integration – the processes by which a practice or practices are re-enacted and sustained among the social matrices of an organisation.

Normalisation process theory was chosen for use in the IMPART study for several reasons. First, NPT focuses specifically on the work carried out by staff to understand the process by which a complex health-care intervention is implemented, embedded and integrated (or not), and as we anticipated that more of our data would come from talking to staff, this matched our focus of concern. Second, NPT is a formal theory and, as such, facilitated the development of testable hypotheses in phase 1 of the study to help focus the collection and analysis of data in phase 2. Third, NPT has clear implementation constructs and a website with clear guidelines on understanding and guiding the implementation process. This allowed all members of the multidisciplinary team to learn about the theory independently (as well as within the group) and ensured a clear reference point for those working with the data and the coding framework. Finally, NPT has also been extensively applied in eHealth settings,78–80 allowing us to more easily compare our findings with previous studies.

Normalisation process theory proved a very useful model for guiding our collection and analysis of data from staff working in EIP teams. However, it was not easily adapted to help understand the experiences being offered by REACT or the wider context in which staff were operating. Both of these were essential to building an explanation of the implementation of REACT. To integrate our findings across all elements of the study, we therefore drew on the NASSS framework,74 which aims to help predict and evaluate the success of a technology-supported health or social care programme.

The NASSS framework was developed from an extensive systematic review of previous technology implementation frameworks, and a series of six empirical case studies, each testing a different type of technology-supported programme (e.g. video consultations, pendant alarm systems, care organising software) in different health-care settings. The framework outlines seven key domains that are important to consider in determining the success of implementation: the condition or illness, the technology, the value proposition, the adopter system (staff and patients/relatives), the organisation, the wider social context and the evolution (interaction and mutual adaptation) of these domains over time.

The NASSS framework was published after the study protocol was funded and, therefore, did not inform the study at the outset. However, while conducting data analysis and interpretation, we found that it provided a useful framework for explaining the findings as a whole. Therefore, the domains from the NASSS framework were used to structure and present our findings in Chapter 8. A diagram of the framework is presented in Chapter 8 (see Figure 19).

Three phases of the IMPART study

We used our iterative case study design flexibly, adapting our approach in response to activity on the ground. While maintaining a focus on specific trusts at each time point, we maintained good links with all trusts and listened to what was happening at each. The study was divided into three phases. The design is shown in Figure 3.

FIGURE 3 Flow chart showing the design of the IMPART study. IPv, implementation plan version; SRG, stakeholder reference group. Reproduced with permission from Lobban et al.56 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/. The figure includes minor additions and formatting changes to the original. (Figure not reproduced here; see the original article.)

Phase 1: hypothesis generation

In phase 1, we outlined an implementation theory identifying the factors that we hypothesised would influence successful implementation of REACT and how they would lead to successful outcomes. Our study theory was informed by NPT, with specific hypotheses further informed by:

• a systematic review of relevant implementation studies of DHIs for people with psychosis or bipolar disorder and/or their relatives to identify factors affecting implementation81
• qualitative analysis of data relevant to implementation from the REACT feasibility trial,53 including interviews with EIP staff who have worked as REACT supporters and relatives who used REACT
• stakeholder workshops including staff and relatives at each participating trust
• synthesis of these data informed by our clinical and theoretical expertise in this area.

Phase 2: hypothesis testing

In phase 2, we used a case study design to test our hypotheses about which factors would influence the implementation of REACT. We developed an implementation plan to address these factors, and made it available in successively more developed forms across three waves.

Phase 2 was conducted across two geographical regions (North of England and South of England), with three NHS trusts participating in each. All six trusts were given the REACT toolkit and initial implementation plan [implementation plan version (IPv) 1] at the start of phase 2. The implementation plan included an online manual of detailed instructions outlining roles and responsibilities for key staff implementing REACT and face-to-face training sessions within each trust. Guidance for relatives on using REACT is embedded within the toolkit.

We collected detailed case study data in two of our participating trusts (one in each region) as wave 1. Key factors affecting implementation were identified and then discussed with stakeholders in the two trusts in wave 2. The research team worked collaboratively with these stakeholders to design IPv2 in these trusts. Further data were collected to test the impact of this and to identify additional factors affecting implementation. A third iteration (IPv3) was then developed with staff in the final two wave 3 trusts. Data from this wave, combined with ongoing longitudinal data from the four trusts in waves 1 and 2, informed the final version (IPv4).

Phase 2 required a mixed-methods approach, in which we collected quantitative data to describe the levels of implementation and outcomes within each trust, and qualitative data to identify the key factors that could explain these patterns of implementation. Consistent with the case series design, our data were first analysed within each trust before we attempted to analyse and explain similarities and differences between trusts.

The methods are described in detail in Chapter 4, and the design and objectives are summarised in Table 2.

Developing the implementation plans

In this study, we have used the term ‘implementation plan’ (or sometimes ‘implementation strategy’ or ‘implementation intervention’) according to the definition by Proctor et al.82 to refer to a wide range of ‘methods or techniques used to enhance the adoption, implementation, and sustainability’ of an intervention (in this case, REACT). Implementation strategies have been widely reported in the literature across a wide range of contexts, interventions, and user groups, but identifying what is likely to work for whom, in what context and in relation to a specific intervention remains a challenge. Attempts have begun to meet this challenge, with the development of taxonomies to define and classify the array of strategies to facilitate further testing (e.g. Powell et al.83 and Leeman et al.84). Although offering helpful suggestions, this work did not yet help us to identify the best strategies to implement REACT. Therefore, we relied on a pragmatic analysis of our data from phase 2, and the NPT framework, to guide our successive versions of the implementation plan.

To develop successive implementation plan iterations in a time frame to suit the project and the needs of trusts, we needed the agility to make sense of the ‘bigger picture’ and then develop, implement and evaluate each new iteration. As far as possible, this was linked to specific data sources (interviews, workshops, documents and meeting observations) but did not rely on a full thematic analysis or coding of transcribed data, which would have been unfeasible in the time available. Regular multidisciplinary research team meetings and stakeholder reference group (SRG) meetings within the trusts ensured that this analysis remained grounded in the data. This was referred to as ‘level 1 analysis’, and is distinguished from level 2 analysis, which was much more detailed and involved coding of interview, document and meeting data using a framework analysis85 informed by NPT and our study hypotheses generated in phase 1.

We did not use NPT to guide our data collection or analysis with relatives, although we did consider this. Although the NPT framework could be usefully adapted to accommodate the very important role of the service user in delivering complex health-care interventions, we felt that this was beyond the scope of the study. We therefore approached our data more inductively and used an open thematic analysis to identify emerging themes.

Consistent with the accelerated creation-to-sustainment (ACTS) model described by Mohr et al.,86 our approach to developing the implementation plans recognised that the success and sustainability of any DHI depended on understanding people’s experiences and the health service setting in which the DHI was intended to be used. Asking health professionals to engage with something new when resources (e.g. time and staffing) are scarce requires us to develop flexible and low-effort ways to facilitate this. New practices, particularly digital ones such as REACT, are often introduced with the promise of time and cost saving or enabling services to meet targets that are not currently being met. However, in the short term (and before these benefits materialise), staff are asked to invest time to learn new skills, become familiar enough with a new technology to be able to use or recommend it, and often to interact with service users in a different way.

The implementation of a DHI may also be one of a range of research activities that a trust or service is currently involved in. Therefore, our implementation plans were developed using collaborative iterative cycles developed through face-to-face workshops, and drawing on appreciative inquiry approaches,87 to recognise and enhance aspects that were working well while providing a safe context to identify what needed to be changed.

Phase 3: finalising the implementation plan

The aim of phase 3 was to synthesise all of the data across all six trusts to identify the key factors affecting implementation and develop a national implementation plan for REACT IPv4 (objective 4), but also to draw out the generalisable learning for the delivery of DHIs into NHS services (objective 5). We used local ‘data analysis days’ to engage staff in the analysis of trust-level data; key staff involved in REACT roles across all trusts in integration of findings across trusts; and the whole project team including carer relatives in a final 2-day ‘explanatory synthesis event’ in which we produced the final explanatory synthesis outlined in Chapter 8. We also presented the iterative development of the implementation plans and drew on all of the study data to agree a final version of IPv4.

Ethics

Ethics approval for the IMPART study was granted by the Health Research Authority and the East of England – Cambridge South Research Ethics Committee (reference 16/EE/0022). All EIP staff in each of the participating trusts were made aware of the study, and organisational research and development (R&D) approval was given for the research team to monitor the use of REACT and be present in the trust. All participants (staff and relatives) who provided additional individualised data (interviews, workshops, relatives’ questionnaires) provided informed written consent prior to participating in the study.