Our conclusions focus on two main areas: the implications of the findings for health care and recommendations for future research.
Implications for health care
Our in-depth analysis of the process of implementing a complex intervention targeted at compassionate care raises questions about the extent to which interventions of this kind should in fact target and seek to influence and restructure relationships, norms and resources in the wider system. They suggest that health-care leaders who interpret their role as mobilising structural capacity to support the relational work of frontline staff may well improve the relational capacity of teams and their individual members. We have defined elements of this mobilisation in relation to the CLECC intervention, and our planned enhancements to the original intervention are set out in Box 2 . The enhancements clarify the role of leaders outside the ward team in supporting ward teams to implement CLECC by engaging leaders in the programme, involving them in the learning activities and creating opportunities for them to engage and reflect with frontline staff. We also propose tying in CLECC with wider staff education strategies in the organisation to help its wider integration, thus enabling the possibility that its goals become embedded and reflected across educational provision.
Proposed enhancements to the CLECC implementation period include appointing PDNs with solid experience as educators, and a more detailed specification for study days, to ensure consistency between sites in adhering to the CLECC principles. The appointment of team members as CLECC champions and inviting teams to develop sustainability plans will encourage staff to identify concrete contextually specific activities that will support CLECC going forward, along with an articulation of the roles, responsibilities and resources required to sustain CLECC.
Other proposed enhancements to CLECC focus on the period beyond the initial implementation period. Sustainability plans will encourage staff to agree on explicit expectations for discussing and developing an understanding of CLECC principles on an ongoing basis, and may embed this culture further. New team members will need induction to understand what CLECC is and what their role is in relation to CLECC. This particularly applies to ward leaders joining a ward where CLECC is in place, and careful attention and mentoring will be needed to support them as they develop their role.
Recommendations for research
The complexity of the intervention and the clear relationship between context and impact reflected in the findings, in addition to the continuing need to establish the efficacy of interventions of this kind, require a mixed-methods approach to future evaluation within the context of a programme of research to:
- identify the organisational contexts in which optimal impact and sustainability of the CLECC intervention are most likely
- further establish the feasibility and validity of the QuIS tool in relation to the experiences of patients in acute care settings
- evaluate the effectiveness and cost-effectiveness of the CLECC intervention relative to usual care.
A programme of research would enable each of these objectives to be addressed. First, identifying the contexts in which optimal impact and sustainability are most likely is an important goal for the next stage of this research. Evaluating the processes of implementation in relation to contextual features in a wider range of acute care contexts will enable us to generate and test a typology of organisational types (at hospital and ward levels) that specifies their receptiveness to interventions of this kind. The nature of this type of evaluation is likely to be largely qualitative, drawing on observations of practice and interviews with key stakeholders over time to describe variations and identify relationships between implementation processes and context, and the resultant impact and sustainability. Qualitative and quantitative contextual data gathered from the feasibility study reported here (on, for instance, ward leadership, staff perceptions of care and staff well-being) could be added to equivalent data gathered from other organisations in a future study and combined to inform the development of the typology and the identification of contextual features relevant to implementation processes, impact and sustainability. The opportunity to investigate the wider dissemination and embedding of ideas and practices originating from the intervention but spreading beyond the target team to the wider system can also be exploited by qualitative exploration within clinical departments and within the wider health-care system.
Meeting this first objective is achievable through the study of context, implementation, impact and sustainability of the CLECC intervention in acute hospital nursing teams sampled to ensure heterogeneity. The findings regarding the influence of differences in contextual features at the institutional and team levels can be used to inform this sampling, namely institutional norms regarding the legitimacy and nature of nursing work, staff learning and staff support; interpretation of key stakeholder roles (including nursing managers and PDNs); and ward-level characteristics, such as staffing levels in relation to patient workload and stability of workforce and team leadership over time. Conducting this study over a 2-year period will enable us to capture the impact of variations over time and also to build a picture of longer-term sustainability.
Although our findings on the use of the QuIS tool are promising in relation to its inclusivity of hard-to-reach patient groups, further work is merited to inform its use as a primary outcome measure in future experiments in acute settings, in particular on its validity in relation to patient experience. Our work indicates that there is an association between QuIS ratings and patient experiences in acute hospital settings, but this relationship needs testing on a wider scale, in particular with patients who have a cognitive impairment. The second objective of the proposed programme of work can be met by a study that evaluates staff–patient interactions through QuIS rating, as used here, but also through patient ratings of the same interactions. These sets of ratings can then be compared. Further work will be needed to establish a means by which patients with cognitive impairment can rate interactions. If the proportion of negative interactions is the primary outcome measure in a future study, understanding which interactions are rated by observers (and, when possible, patients) as negative, and why, is an important next step, as is working with patient representatives to establish their views on the size of a meaningful reduction in negative interactions. Further study could also be used to develop more effective procedures to blind observers from experimental allocation in advance of an experimental study. In addition, the high intracluster correlation that we found at an observation session level merits the exploration of the cause of this variance and the feasibility of different approaches to data collection that might reduce its impact, for instance, shorter observation sessions. This further evaluation and testing of QuIS across these parameters would be a valuable foundation to its further use as an outcome measure in acute settings.
Drawing on our findings about the feasibility of experimental approaches to evaluating compassionate care interventions, our third objective for a programme of work focuses on the delivery of a definitive multicentre trial to establish CLECC’s effectiveness and cost-effectiveness. Our findings indicate that, given variations in the implementation journey over time, outcomes should be captured over a long period of time, at least 12 months, to ensure that sustainability is tested. Evaluation in a range of organisational contexts will improve the generalisability of findings and so a multicentre trial is merited. If QuIS is selected as the primary outcome measure, the high ICC at the observation session level indicates that it would be more efficient to conduct more observation sessions of shorter duration. The length of time taken to recruit a patient group with more complex needs militates against conducting sessions of < 1 hour. We can explore this further in the QuIS study proposed above but, for the calculations that follow, we assume that observation sessions are 1 hour in length. Detecting a 50% reduction in the rate (odds) of a negative interaction (i.e. a reduction from 10% of all interactions to 5%) at 90% power would require observation of 582 interactions per group, that is 1164 interactions in a parallel group trial. Allowance for clustering is achieved through use of a multiplicative factor [1 + (n cluster – 1) × ICC]. Patients in our feasibility study had an average of six interactions with staff per hour. If the cluster (observation session) is six (interactions), that is, the observation sessions are 1-hour long, the factor is [1 + (6 – 1) × 0.411] = 3.055. We would therefore need 1164 × 3.055 = 3556 observed interactions to detect a difference, that is, 593 observation periods of 1 hour each in total, rounded up to 300 per group.
If individual wards are observed for 20 hours each at each assessment period (the amount used in the feasibility study), 30 wards would need to be engaged in a trial (15 in each group). Our estimates of the costs and feasibility of implementing the CLECC intervention in each NHS organisation and the work required to set up and oversee the study in each organisation indicate that five wards each in six different NHS hospitals would enable this level of data collection to be achieved. These calculations assume that one patient is observed at a time but take no account of clustering within wards. In reality we were generally able to observe more than one patient at a time, which would provide additional data that would be more than sufficient to compensate for the relatively small increase in sample size required because of clustering at the ward level, given the low ICC at this level.
Our findings indicate that measuring patient views on care and staff self-rated empathy are useful and feasible as secondary outcome measures, although careful attention would need to be paid to maximise staff survey response rates. The study design would need to ensure that intervention and control conditions do not run in the same organisation at the same time. A waiting list control may be helpful here, with all study wards eventually receiving the intervention, but this would double the intervention costs and funding constraints are very likely to reduce the feasibility of this option.
As outlined above, cost-effectiveness can be evaluated through an impact inventory, including comprehensive data on the costs and benefits of the interventions, distinguishing training and implementation. We also propose a series of exploratory cost-effectiveness analyses linking cost to each of the primary and secondary outcomes.
To summarise, our findings indicate that further intervention development and evaluation work of the CLECC intervention through a programme of research is now merited.