Discussion and conclusions

Principal findings

To our knowledge, this was the first, large, randomised evaluation of a complex preventive psychological intervention conducted in the challenging setting of critical care units in the NHS. The POPPI trial, a parallel-group, cluster RCT conducted in 24 adult general critical care units, indicated that, among adults who stayed at least 48 hours in the critical care unit and received level 3 intensive care, the delivery of a complex, nurse-led preventive psychological intervention provided no significant difference in the primary clinical outcome (PTSD symptom severity at 6 months) when compared with usual care (treatment effect estimate –0.03 points on the PSS-SR; 95% CI –2.58 to 2.52; p = 0.98). There was considerable statistical uncertainty with regards to the cost-effectiveness results. On average, the POPPI intervention showed decreased costs and slightly improved QALYs, leading to a positive INMB at 6 months (£835, 95% CI –£4322 to £5992). The probability that the POPPI intervention is cost-effective (at a willingness-to-pay threshold of £20,000 per QALY) is ≈60%. When extrapolated to the lifetime, the INMB was larger, although with even greater uncertainty.

There was no significant interaction between the effect of treatment allocation and PTSD symptom severity at 6 months in any of the prespecified subgroups: age, sex, socioeconomic status, duration of delirium, anxiety (measure by the STAI-6), surgical status, how well the intervention sites adhered to the intervention (overall intervention implementation score) or heterogeneity of treatment effect (predicted PSS-SR score). There were no significant differences in any of the six secondary clinical outcomes, including anxiety and depression. At 6 months, just over 20% of responders scored > 18 points on the PSS-SR (i.e. the threshold warranting further investigation for probable PTSD), indicating substantial ongoing psychological morbidity in this patient group.

We suggest the following reasons why this preventive, complex psychological intervention may have not been effective in reducing longer-term psychological morbidity:

• Contrary to evidence suggesting that focusing on acute stress and its causes early in the ICU may help,113 patients may have still been too ill to absorb and put into practice the therapeutic messages taught in the stress support sessions. This is shown by the high numbers of patients being discharged from critical care prior to regaining the necessary cognitive capacity to consent. The POPPI nurses frequently reported patient-centred barriers to intervention delivery, including patient tiredness, confusion and dexterity problems, with patients focusing on short-term goals, primarily leaving the critical care unit. If an intervention was targeted later in a patient’s recovery trajectory, it could include more challenging techniques (e.g. exposure to traumatic memories), which may be needed to reduce longer-term psychological morbidity in this population.114
• Patients did not receive the full dose of the three stress support sessions. Although protocol adherence was high, with > 80% [this ranged across sites from 50% (lowest adherence) to 94% (highest adherence)] of patients receiving at least two sessions, approximately one-third did not receive session 3. As shown in Chapter 6, there was a short-term reduction in anxiety, measured using the STAI-6, for patients who received all three sessions and had data. The main reason for not receiving subsequent sessions was hospital discharge; therefore, a future intervention that follows patients into the community may be necessary.
• As described in Chapter 6, Introduction, some of the POPPI nurses did have difficulties in delivering the sessions as they were intended, such as struggling to keep patients on track with the session content. This may have been because critical care nurses are non-experts with regards to psychological support. Qualified psychologists may be required to deliver such interventions, or non-experts may require more in-depth training, guidance and support to help with more complex patients and situations.
• The translation of knowledge from an online training course into practice was varied at sites. Uptake was high (> 80% of staff) with regards to completion of the online training, but this may not have translated as required into practice. The POPPI nurses reported the challenge of changing long-standing practices in units and how all units have staff who are difficult to engage in any new practice. Some units were restricted in the degree to which they could implement change by the environmental limitations of the unit. In very open-plan small units, staff even found it difficult to address the more simple elements such as reducing noise levels and orientating patients with whiteboards or clocks. Moving forward, if an environment intervention were to be implemented in the NHS critical care setting, it is clear that some units would be able to adapt to the intervention more fully than others, but this should not be equated with staff resistance to compliance with recommendations.

Summaries of key research recommendations

• Conducting research in this area is challenging and future research needs to factor in, when relevant, several challenges identified during the conduct of this cluster RCT. These include the short time period during which patients have mental capacity prior to discharge from the critical care unit, which affects the ability to consent and deliver a critical care unit intervention; the potential stigma associated with psychological intervention(s), which can negatively affect the process of attempting to gain consent; and releasing bedside nurses from their busy work schedules in order to attend training, deliver psychological interventions and participate in debriefing and support.
• Prior to development and evaluation of any subsequent psychological intervention in the critical care unit, there is much to learn from post hoc analyses of the rich quantitative and qualitative data from this cluster RCT:
  • Further exploration of why the nurse-led complex psychological intervention in the cluster RCT did not produce an effect. Potential areas identified above worth exploring include whether or not the intervention was the right intervention; whether or not the timing and duration of the intervention were correct; whether or not the right people to deliver the intervention (non-experts) were used; and whether or not the necessary amount and type of training and support were delivered.
  • Further exploration from the cluster RCT data of the patient factors, including risk factors, mediators and subgroups of patients at a greater risk of acute stress or longer-term psychological morbidity who may be more likely to benefit from a future psychological intervention.

Implications for health care/practice

The results of the cluster RCT do not support the adoption of this complex, nurse-led preventive psychological intervention into routine practice.

However, the cluster RCT results do indicate that high levels of both acute stress in the critical care unit (64%) and longer-term PTSD symptom severity (20%) exist, and that these levels are in line with previous studies.16,115 This area, therefore, remains a priority area for critical care and those involved in the longer-term follow-up of patients.

The POPPI cluster RCT has unearthed a number of important health-care factors that are vital to consider when assessing how best to alleviate acute stress and prevent psychological morbidity among critically ill patients. There is huge room for improvement with the routine assessment of delirium. Less than 50% of patients who were recruited had evidence of a CAM-ICU assessment documented in their medical records, despite all 24 cluster RCT sites reporting, at the point of site identification and selection, that they routinely use this tool. The lack of routine screening with CAM-ICU in the POPPI cluster RCT may reflect a perceived unreliability of the tool among critical care staff, or again, may be a function of patients being discharged from the critical care unit very soon after regaining mental capacity. Another key consideration identified is that, if critical care staff were to provide psychological support to patients, they would need to have fully protected time away from other clinical duties, to allow for the additional workload that this would entail. This is difficult to achieve in the confines of a research study, but those organising and managing clinical practice would need to thoroughly consider this issue, to ensure that high-quality support is provided without causing additional stress to those delivering the support. With regards to the fact that it was observed to be difficult to change aspects of unit culture (shown by the huge variation in the success of translating the online training into clinical practice), this has a potential knock-on effect for how a large part of mandatory training is delivered in the NHS, to make sure that knowledge is actually being translated into practice. The final health-care factor identified was regarding the earliness of discharge from critical care. Interventions need to take into account the fact that patients are often discharged without regaining mental capacity.

Strengths and limitations

Strengths

This large, parallel-group, cluster RCT was rigorously conducted in a challenging setting. There were a number of potential strengths of the conduct of the cluster RCT:

• The intervention was rigorously developed and refined, adapted from relevant theories and techniques to alleviate acute stress and prevent longer-term psychological morbidity for critical care patients, according to the MRC guidance,45 which was led by experts with vast psychology and adult critical care experience.
• The RCT built on a detailed feasibility phase, which tested and refined delivery of the trial processes, the education package, the support tools and the complex psychological intervention.
• Site set-up was completed efficiently and on time, according to the pre-planned schedule, with only one site opening shortly after its scheduled start.
• The RCT was pragmatic, conducted in a representative sample of NHS adult general critical care units, with delivery of the intervention (including key staff selection) determined locally, as would be the case if the intervention were to be adopted into routine NHS care.
• A high proportion of eligible patients were identified and approached for consent (69%), with a total of 1458 patients recruited across 24 critical care units over the 17-month recruitment period.
• Loss to follow-up on the important patient-centred psychological outcome, in this complex patient group at 6 months, was low. There were no important differences in baseline characteristics between patients who did and those who did not respond.
• We used the recommended approach of multiple imputation to address missing data, and imputed missing values, conditional on all the information observed, including the vast data available through the CMP.
• All analyses were conducted according to a prespecified, published statistical analysis plan.70
• We included prospectively designed integrated economic and process evaluations; the process evaluation was essential given the complexity of the intervention. The economic evaluation ensured that resource use data were collected on both primary admissions and re-admissions for each patient recruited, and harnessed information from four sources: the electronic case report form, linked data from the CMP database and NHS Digital, and responses to the follow-up health services questionnaires at 6 months.

Limitations

There were a number of potential limitations of the conduct of the cluster RCT:

• As for many cluster RCTs, there is a risk of selection bias, with potentially different cohorts of patients recruited in different groups.116 In the cluster RCT, there was an indication that refusals of consent were higher in the intervention sites during the intervention period than during other periods. We attempted to mitigate this issue by providing the option for patients to consent to receive solely the follow-up questionnaire, but this was rarely taken up. Even though slightly differential recruitment rates were seen, mainly attributable to the increase in refusals of consent, no important differences in baseline characteristics between time periods were observed.
• The nurse-led, preventive, complex psychological intervention could not be blinded to those caring for patients, but the risk of bias was minimised through central random allocation after sites had already started recruiting to the cluster RCT (during the baseline period). The use of a patient-reported primary outcome measure was not subject to observer bias.
• Despite all sites that signed up indicating that they routinely used the CAM-ICU to monitor delirium, actual monitoring with the CAM-ICU was poor, with over half of the recruited patients not being assessed for delirium.
• Although the stress support sessions were designed to be commenced in the critical care unit, the proportion of patients actually receiving their first session in the critical care unit was lower (40%) than expected.
• We were unable to assess fidelity to the stress support session delivery by audio-recording sessions. Probably owing to the nature of the interaction, it was reported by the POPPI nurses that this aspect was felt to be intrusive and would make patients feel uncomfortable. Alternatively, more detailed ethnographic methods could have been used, but this may not have resolved patients feeling uncomfortable.
• To better understand delivery of the intervention, additional patient feedback could have been beneficial. Although patient feedback was sought during the intervention feasibility study, and that feedback confirmed acceptability of the stress support sessions, only a voluntary mechanism to provide feedback was provided in the cluster RCT. Unfortunately, this opportunity was not taken up. Feedback from relatives may have also been of interest; however, this was not sought.
• HRQoL data at baseline were self-reported by patients using a visual analogue scale, which was chosen because a visual analogue scale is a simple tool that is quick to administer and easy to complete for patients in critical care. There are concerns over whether or not critical care patients are able to provide valid ratings using simple visual analogue scales. On the other hand, there are concerns over whether or not valid HRQoL data for patients can be obtained from proxies (e.g. carers or health professionals).117 Considering this, we performed a sensitivity analysis considering alternative values of HRQoL at baseline, and found that the cost-effectiveness results are not sensitive to alternative values of HRQoL at baseline.
• As there is currently no NICE-recommended value set specific to the EQ-5D-5L, we decided to value the descriptive EQ-5D-5L patient data, collected as part of the cluster RCT, using a published EQ-5D-5L value set for England,98 which is one of the first value sets reported that is relevant to this population. An alternative approach would have been to use available crosswalk value sets (e.g. from the NICE-recommended EQ-5D-3L value set to the EQ-5D-5L) via a ‘mapping algorithm’;118,119 however, this approach was not adopted as further research is ongoing to establish an optimum mapping algorithm. The choice of value set (EQ-5D-5L value set versus EQ-5D-3L to EQ-5D-5L mapping) may have important implications for the cost-effectiveness results.120
• The estimation of lifetime cost-effectiveness inevitably required us to make assumptions, in particular about the costs in the time period beyond the observed data. However, the study made maximum use of the available trial data to inform these assumptions.