This study aimed to assess the feasibility of implementing the CLECC intervention in acute hospital settings and to assess the feasibility of conducting a CRT with associated process and economic evaluations to measure and explain the effectiveness of CLECC.
The objectives were to:
- determine the feasibility of implementing the CLECC intervention and sustaining the resulting work practices
- inform the design of a definitive evaluation of the effectiveness of CLECC
- inform the measurement of costs and benefits of CLECC in a definitive evaluation.
Findings showed that the CLECC intervention is feasible to implement in practice with medical and surgical nursing teams in acute care hospitals. We found strong evidence of good participation by nurses and HCAs, and staff reported benefits throughout CLECC’s introductory period and beyond. Further impact and sustainability were limited by the focus on changing ward team behaviours rather than wider system restructuring. The pilot CRT proceeded as planned and randomisation was acceptable to teams. There was some evidence of contamination between wards in the same hospital but not between wards involved in the study. QuIS performed well, with a high recruitment rate and good inclusion of people with cognitive impairment. At follow-up there were higher total positive and lower total negative QuIS ratings for intervention wards than control wards. More control ward patients than intervention ward patients had lowest (i.e. more negative) scores on the PEECH connection subscale. These differences, although supported by the qualitative findings, are not significant. No significant differences in nursing empathy were observed, although response rates to the staff questionnaire were low. We also identified the costs associated with using CLECC and recommend that an impact inventory is used in any future study.
Each of the study objectives is addressed in more detail below.
Sections of this chapter are reproduced in Bridges et al. 121 This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See http://creativecommons.org/licenses/by/4.0/.
Feasibility of implementing and sustaining Creating Learning Environments for Compassionate Care121
The process evaluation aimed to identify and explain the extent to which the planned intervention was implemented into existing nursing practices and to draw conclusions about optimising the sustainability of future CLECC intervention. We found that, on all of the wards, many of the individual elements of CLECC were welcomed by staff and were possible to implement during the implementation period. We also found that sustaining this work beyond this time was difficult for some ward teams to achieve. Findings point to refinements needed to CLECC to improve the prospect of its impact and sustainability. The findings applied across all the ward contexts, in spite of some important differences between wards at the outset including specialty, staffing levels and ward leader experience.
Although CLECC had limited coherence for some staff, it was welcomed by teams and served as a broader stimulus to collective action. CLECC developed cultures in which reflection, learning, mutual support and innovation were legitimised within the work team, and in which expertise was seen to be distributed more widely between managers, RNs and HCAs. CLECC moved all the teams further along the continuum to becoming more expansive learning environments,29 but implementation was mediated for all by the context of working in an acute hospital environment. Staff highlighting what they valued about CLECC illuminated the stark realities of team working in such settings. The struggle to find the time to participate in CLECC reflects the pressure on staff to be constantly engaged in material patient care activities. Staff valued the cluster discussions because, otherwise, there was little opportunity to support each other’s well-being. They appreciated the study days because they could get to know each other as people. They valued CLECC because otherwise they were lone workers, sharing working time and space with other team members, but not actually working as a team at all. The intensification of nursing work owing to rising patient complexity, in parallel with the application of increasingly stringent financial efficiency quotas, is well documented.124–126 These findings paint a rich picture of the consequences for staff experiences at work and explain associations between hospital work team climate and staff well-being reported elsewhere.26,27
Staff at all levels were able to identify the benefits to patient care of ward staff engaging in CLECC activities, echoing other findings that the creation of unmanaged spaces for work team members to ‘take shelter’ provides the potential for valued learning and social support for difficult work with clients.39,125 The findings confirm that intervening at the work team level can be successful, confirming an association conjectured from other research.26–28,127 In spite of high workloads, CLECC empowered staff to reflect on local norms governing team practice, and on the relationships and resources that aligned with them, and to make some changes, confirming the original programme theory (set out earlier in Chapter 1 , Creating Learning Environments for Compassionate Care ) and indicating that collective agency can play a part in shaping relational capacity at the individual and work team levels.125,128 However, we also found that implementation was uneven between teams, particularly over the longer term, reinforcing the value of paying attention to the sustainability of complex interventions beyond initial set-up.129–131
Factors outside the direct influence of the ward teams mediated the impact and sustainability of the intervention, in particular the institutional norms that legitimated staff’s participation (or not) in CLECC activities and the interpretation by more senior figures (including PDNs, matrons and senior hospital managers) of what CLECC was and their role in supporting it. Although CLECC draws on the principles of democratic working, and we saw how HCAs in particular were enabled to take a lead in some CLECC activities, its longer-term success relied on cognitive participation from more senior members of the hierarchy. The authority that ward staff had to control how they spent their time, to innovate and to afford their own and colleagues’ well-being some priority varied between teams and was dependent on the signals, or ‘invitational qualities’,128 from these more senior figures as to what was legitimate or not. These findings that nurses do not control the conditions in which they work echo extensive research on the curtailment of professional autonomy in publicly funded health care, and the particular position that nursing as a profession occupies.2,7,8,126,132–134 Matrons are the point at which organisational drivers, often business imperatives, must align with professional imperatives and the needs of frontline teams and their patients. The hybrid role and competing identities for nursing managers of this kind have been highlighted elsewhere126,135,136 and it is unsurprising that we identified different approaches to managing this key role. Although the current CLECC activities relating to senior manager participation appear to have aided coherence, findings suggest that additional activities targeted at improving their cognitive participation may be needed. Contextual features that appeared to be relevant to CLECC’s implementation journey included institutional norms regarding the legitimacy and nature of nursing work, staff learning and staff support; interpretation of key stakeholder roles (including nursing managers and PDNs); and ward-level characteristics, such as staffing levels in relation to patient workload, and stability of workforce and team leadership over time.
Our findings indicate that a higher and more sustained impact for interventions, such as CLECC, may be possible only through more substantial restructuring that reshapes the conditions in which people are able to act.53,54 We support Parker’s39 assertion that caregiving organisations need to be designed to enable caregivers to access functional work teams within which they can interpret their experiences, and we have identified a number of concrete but modifiable barriers that merit attention in such design, including lack of time and institutional rules that undermine the value of staff-to-staff social support. They also include more clearly defining the role of nursing managers in signalling the legitimacy of staff providing each other with emotional support, supporting nursing teams to meet and learn together. Future versions of the CLECC intervention should include new activities to engage nursing managers in the implementation period, involve them in the learning activities and create opportunities for them to engage and reflect with frontline staff.
These findings reflect the fact that relational work in caregiving organisations depends not just on individual caregiver agency but also on whether or not this work is adequately supported by features of the wider system. Relational capacity may thus be regarded not only as a property of individual practitioners, but also as a modifiable and situated property of work teams. The limitation that institutional norms of legitimate nursing work placed on staff finding time to meet together raises the prospect of lack of relational capacity at a wider system level, and suggests that wider restructuring beyond middle manager roles may in fact be needed to effect substantial and sustained change.
These findings offer possibilities for actively restructuring work team conditions to enable the relational aspects of caring and working. Complex interventions, such as CLECC, can be theorised as ‘time-limited series of events, new activity settings and technologies’ in systems, the focus of the intervention being to generate new structures of interaction and new shared meanings.53,54 Adopting this perspective means that the focus of change efforts is not just on the behaviour of individual staff, but also on restructuring of relationships, norms and resources in the wider system that may play their part in the success or otherwise of the intervention. The findings point towards a number of organisational conditions in which high-quality care is most likely and in which interventions of this kind are most likely to succeed, and these are set out in Chapter 12 . Further research across a wider range of organisational settings will enable these emerging theories to be refined to enhance their transferability. This further research will also enable us to more closely describe the links between context, implementation processes and outcomes associated with implementation of the CLECC intervention.
Informing future Creating Learning Environments for Compassionate Care evaluation
Our systematic review16 found that any of the interventions we investigated might be deemed worthy of further investigation based on their positive outcomes but that none could be recommended for routine implementation, given the lack of theoretical basis and description for many interventions, the pervasive positive bias that is associated with weak study designs, and the lack of evidence for impact on patient outcomes in most studies. Adherence to recognised and emerging standards for developing and evaluating complex interventions, such as the UK MRC framework,88 and fuller reporting of interventions and outcomes would address many of the issues noted in our review. We concluded that many researchers in this field have been unable or unwilling to use experimental designs within the context of mixed-methods approaches to evaluation.
The findings from this study indicate that the use of an experimental design to evaluate the effectiveness of compassionate care interventions within the context of a mixed-methods study is feasible, as is a focus on outcomes that are patient based. Ward teams were successfully randomised to intervention or control, and staff were amenable to the prospect of randomisation to either experimental condition. All wards recruited remained in the study throughout data collection and all clusters randomised to the intervention went on to receive it. Blinding of patients and visitors to ward allocation appeared successful, although strategies to blind researchers gathering data need further development in a future trial. Although we identified some differences between individual clusters and individual participants in the trial, none was sufficiently substantial to raise concerns of baseline imbalances between intervention and control conditions.137,138
We found evidence of pathways through which the CLECC intervention had the potential to influence practice in other wards in both of the participating NHS trusts. This may be a sign of CLECC’s success in transforming and embedding in the wider system, but also indicated that future studies of effectiveness should not run intervention and control conditions in the same organisation over the same time period. This will be an important consideration in designing any future definitive experiments.139,140
Older patients with a range of cognitive abilities are a traditionally hard-to-reach group in research, especially when they are unwell and in hospital. Even though they are more prone to negative experiences of hospital care, they are often excluded from research.6,141,142 Although cognitive deficits may limit some people’s ability to share their experiences, our findings indicate that devising recruitment and data collection methods that maximise support and inclusion can be successful. It is estimated that at any one time, up to 25% of beds in acute hospitals are occupied by people with dementia, with the figure likely to be higher on specialist older people’s care wards.143,144 Overall, 25% of patients observed in this study had evidence of cognitive impairment, suggesting that our sample was representative of the wider hospital population. Of all the patient questionnaires returned, 12% were completed by patients with cognitive impairment and so we were less successful in achieving representativeness here.
Our findings that structured non-participant observation appears to be the most promising method to describe the experiences of older people with cognitive impairment in the general hospital setting echo those of Goldberg et al. 141 Participating in an observation does not require any particular state of health, abilities or performance from the patient in question, whereas, for instance, answering questions in an interview about one’s care experiences requires as a minimum orientation to place, language skills and attention.142 We did find that recruitment of people with cognitive impairment to the study took more time, as did completing questionnaires with them, and this has resource implications for future research with this patient group. In relation to patient involvement in general, this method overcomes the reluctance of patients to evaluate care while it is ongoing that was noted in other studies.64
Our development of the QuIS tool for use in acute settings worked well and our earlier work confirmed an association between patient-reported experience and observed staff–patient interactions.99 However, our earlier work did not include people with a cognitive impairment and validation of QuIS ratings with this patient group may be a necessary next step in the tool’s development. Although a clear design effect was apparent with QuIS at the observation session level that will need to be accounted for in the design of a future trial, benefits to the use of this tool were notable.139 Acceptability of the tool to patients and staff was high, and reliability between observers was acceptable. We did not find any evidence that staff changed their behaviour as a result of being observed but this possibility cannot be eliminated. However, the findings across both assessment periods that a proportion of interactions observed were negative indicate that, even if staff planned to give consistently good care while being observed, they were not successful. Although we cannot eliminate the possibility of observer effects, the effect in a future trial will be present on all wards in all conditions and so differences between wards can still be attributed to the intervention. We know from other work that the quality of interactions with staff is very important to older people and shapes their experiences in hospital settings and so its successful measurement is a good indicator of compassionate care.6 Our findings from this study confirm that observation-based measures are more inclusive of patient groups vulnerable to negative experiences in hospital. Overall, they support the selection of the quality of staff–patient interactions, measured by an observational tool (such as QuIS), as a candidate primary outcome in a future trial.
The response rate to nursing questionnaires (36%) was very low, particularly at follow-up stage on the intervention wards, perhaps owing to research fatigue. There was a perception that questionnaires were lengthy to complete and that staff were too busy. The low response rate means reduced certainty that the responses to the questionnaire represent the views of the nursing staff as a whole, although the findings of burnout and low empathy within the staff groups on every ward were important. The response rates are typical for surveys of this kind with this population. For instance, in a European study145 of nurse staffing levels, an estimated 39% of 2917 registered nurses working on NHS medical and surgical units in England completed a questionnaire similar to the one used in this study. Deploying a shorter questionnaire and negotiating time for their completion with managers may enhance response rates in a future study.
Small but non-significant differences between experimental groups at follow-up in quality of interaction and patient evaluation of emotional connection with staff are promising findings, particularly in the context of qualitative findings that indicate benefits to patient care perceived by staff. Data were gathered ≥ 4 months after the end of the implementation period, indicating that if there is an effect that can be attributed to CLECC, it is sustainable beyond the period in which CLECC was being actively facilitated by PDNs. We found no evidence that nursing staff empathy may be improved because of CLECC but these results in particular have to be viewed in the context of the low response rate to nursing surveys.
These findings are an important contribution to a field in which the use of experiments is relatively rare, and the results reported here will inform study design, sample size calculation and outcome measure selection and use in a future trial.
Our qualitative findings illuminate the importance of context in shaping implementation and outcomes and strongly indicate that future measurement of benefit should be part of a mixed-methods evaluation.
Informing measurement of Creating Learning Environments for Compassionate Care costs and benefits
Our findings have established the feasibility of estimating the cost of a CLECC-type intervention. Intervention costs were calculated as training costs (PDN time and staff time attending study day) and ongoing implementation costs (cost of staff engaging in CLECC activities on the ward). The extent to which training costs would be an additional cost to existing training would need to clarified in a future study, including any further training costs incurred from shifting team membership and the need for refresher sessions for existing staff. Through qualitative interviews with staff, we were able to establish that, aside from initial CLECC training costs, the implementation of concrete CLECC activities by ward teams was not perceived to require additional resource use. However, given the size of our sample and its qualitative nature, we consider that staff time spent on CLECC activities would still need to be recorded and costed in any larger trial. Data should also be gathered on the amount of training delivered through a register of attendance at classroom training, action-learning sessions and cluster sessions. The findings also indicate candidate activities associated with supporting CLECC that merit attention in a future evaluation.
A patient-level outcome measure, such as EQ-5D-5L, was shown not to be feasible, mainly because different patients with different ailments and severity were involved at baseline and at follow-up. Use of a patient-specific outcome requires either that the same patients in both intervention and control groups are measured over time or that adequate adjustments for the casemix of severity can be made. Neither seems likely. Neither QuIS nor any of the secondary outcomes were promising candidates for cost-effectiveness analysis, owing mainly to lack of single summary measures. Although QuIS is used as an audit tool in Scotland and has been applied to some services in the English NHS, it does not at present lend itself to use in cost-effectiveness analyses. Provision of data on QuIS and other ward-level scores may be of use in other evaluations or in developments towards a summary measure.
Consequently, we consider that any proposed economic evaluation of a CLECC-type intervention should comprise two elements. The first is an impact inventory, including comprehensive data on the costs and benefits of the interventions, distinguishing training and implementation. The second element proposed is a series of cost-effectiveness analyses linking cost to each of the primary and secondary outcomes. This approach has value for the evaluation of novel complex interventions with uncertain resource implications.
In summary, an impact inventory would provide a comprehensive listing of the resources, cost and benefits of CLECC with a focus on those to do with providing the interventions, but set within a wider context that includes effects on staff and on patients. Cost per change in each of the primary and secondary outcomes could also be estimated and compared with other later studies.
Strengths and limitations
This detailed and thorough mixed-methods study makes an important contribution to the evidence base on the design and evaluation of compassionate care interventions. Based on data gathered from a range of English NHS ward contexts, the findings are relevant to those seeking to influence and evaluate nursing practice in acute hospital settings in similar contexts. The qualitative findings indicate that staff welcomed CLECC and perceived positive benefits to their own well-being, to improved team working and to more compassionate patient care. As detailed earlier, we have theorised that the impact and sustainability of CLECC can be enhanced by attention to wider system restructuring, and refining these theories using data gathered in a wider range of organisational contexts will be an important next step. In particular, closer attention to defining the contexts in which CLECC is most likely to succeed will be a necessary focus in the next stage of this programme of research. We found variations in intervention fidelity attributable to a variety of contextual features. These factors merit refinement of elements of the intervention for future use, but also deserve continuing investigation in future studies because there may be other important features, identifiable only through a larger-scale study of context. Continuing qualitative investigation, in the form of interviews with frontline staff and service managers, and more detailed observation of CLECC implementation in practice, will enhance an understanding of the influence of context on implementation. It will also provide explanations for findings regarding CLECC’s efficacy.
The pilot RCT findings lend some support to the nurses’ views as regards the benefits to patient care, but larger-scale evaluation is needed before definitive claims are merited. Insufficient information was available at the outset of this study to enable power calculations that informed sample size and so there is no certainty that any apparent positive effects are not produced by chance alone, rather than the impact of the CLECC intervention. Potential issues of lack of researcher blinding to experimental allocation and contamination pathways between intervention and control wards also mean that caution should be applied in drawing conclusions of efficacy from the findings presented.
We have generated useful findings about the performance of a range of outcome measures in relation to compassionate care and have demonstrated the feasibility of using patient-based outcome measures in this field. Our findings indicate the strengths of observation-based evaluations of care delivered, but further research to assess the validity of these evaluations in relation to the experiences of people with cognitive impairment is warranted.
The research to date has focused on nursing teams in hospital settings and no claims are made about the generalisability of these findings to other types of team or other settings. We propose that, with some modifications to account for different contexts, the CLECC intervention may be of value to other teams in other settings but research of the kind reported here will be an important foundation to its use and evaluation in new contexts.
Our findings indicate that further evaluation is merited and point the way to how such future evaluation should be designed and carried out. Chapter 12 draws together the conclusions from the study and sets out implications for health care and recommendations for this future research.