Feasibility, piloting and refinement

Objectives

• To test the feasibility and acceptability of the education package, support tools and delivery of the intervention to patients and staff.
• To test the feasibility of the proposed procedures for the cluster RCT (including confirmation of the recruitment and retention rates).
• To refine the education package, support tools, intervention and cluster RCT procedures based on results of the feasibility studies.

The scope of the two feasibility studies is presented in Figure 2.

FIGURE 2 Scope of the POPPI feasibility studies. (Figure not reproduced here; see the original article.)

Methods

Research governance

A joint protocol for two feasibility studies was prepared and submitted to a Research Ethics Committee (REC) for approval. Applications for adoption onto the National Institute for Health Research (NIHR) Clinical Research Network (CRN) portfolio and the International Standard Randomised Controlled Trial Number (ISRCTN) registry were made.

Recruitment

Sites and POPPI nurses

Two NHS adult, general critical care units (one located in a teaching hospital and one in a district general hospital) were invited to participate in the intervention feasibility study. Two different units (also one in a teaching hospital and one in a district general hospital) were invited to participate in the RCT procedures feasibility study. All four sites were asked to identify principal investigators (PIs).

For the two intervention feasibility study sites, three self-selected POPPI nurses were identified locally to participate in the education package and to support and deliver the POPPI intervention, using the following criteria:

• expert practitioner in critical care
• excellent communicator with good interpersonal skills
• interested in improving critical care patients’ psychological outcomes
• willing and able to support the rest of the team in delivery of all elements of the POPPI intervention
• available to attend a 3-day training course and 1 day for feedback and assessment.

Within both feasibility studies, all sites obtained local research and development (R&D) approval and were visited by the trial team for a site initiation visit, prior to the commencement of patient screening and recruitment. Regular contact was kept between the trial team and the local research teams throughout the recruitment periods for each study. In addition, in the intervention feasibility study, the expert trainers (health psychologist and senior nurses) provided the POPPI nurses, both in person and over the telephone, ongoing one-to-one debriefing and support on the stress support sessions.

Patients

Identical screening procedures were used in both feasibility studies. On admission to the critical care unit, all patients were added to a screening and enrolment log. Once a patient had stayed 48 hours in the unit, they were screened by the local research team for the following criteria:

• aged ≥ 18 years
• received 48 hours of level 2 or 3 care
• English speaking
• no pre-existing –
  • chronic cognitive impairment (e.g. dementia)
  • psychotic illnesses
  • chronic PTSD
• not previously recruited into the intervention feasibility study/RCT procedures feasibility study (as relevant).

If the patient met all of the above criteria, daily screening of the following criteria commenced:

• current Richmond Agitation–Sedation Scale (RASS) score of between + 1 and −1
• current Glasgow Coma Scale score of 15
• not terminally ill or receiving end-of-life care
• currently able to communicate orally
• able to give informed consent [e.g. not deemed delirious by the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU)].

If any of the daily screening criteria were not met, the patient would be rescreened each day until either fully meeting the criteria or being discharged from the critical care unit. Once the patient met all the daily screening criteria, they would be approached for informed consent in the unit. The final screening status of the patient, including reasons why patients were not recruited, were recorded on the screening and enrolment log.

To maintain the profile of the feasibility studies, posters were displayed in staff areas and at other relevant locations within the units and pocket cards, which summarised the eligibility criteria, were distributed to unit staff.

The informed consent process included provision of information, both written [in the form of the prepared information leaflet and patient information sheet (PIS)] and verbally in conversation with authorised members of the local research team. Eligible patients were encouraged to ask questions and given the opportunity to discuss the study with both family and/or friends. After the local research team member was satisfied that the patient had read and understood the information, they were invited to sign the consent form.

Delivery of the POPPI intervention: intervention feasibility study

At intervention feasibility study sites, consented patients were assessed using the IPAT. If the patient was assessed as acutely stressed (scoring ≥ 7 points), they were offered three stress support sessions, which were to be delivered by the specially trained POPPI nurse, alongside the relaxation and recovery programme.

Feasibility assessment

To deliver the education package

Assessment of feasibility to deliver all elements of the education package encompassed the following activities.

POPPI online training

The POPPI online training was evaluated by critical care staff members completing the course as follows:

• an assessment of key learning (knowledge assessment)
• feedback on the course.

Three-day training course

The 3-day course on the intervention was evaluated by each of the six POPPI nurses (three nurses across two sites) as follows:

• a pre and post assessment of self-efficacy
• self-assessment of key learning objectives (knowledge assessment)
• feedback on the course and course materials.

One day for feedback and assessment

The 1 day for feedback and assessment on delivering the intervention was evaluated by the six POPPI nurses and expert trainers as follows:

• an assessment of self-efficacy
• an expert trainer assessment of efficacy (skills assessment)
• a focus group discussion.

To deliver the POPPI intervention

Assessment of feasibility to deliver the three elements of the intervention encompassed the following activities.

Creating a therapeutic environment in critical care

Feasibility to deliver the first element of the intervention (creating a therapeutic environment in critical care) via the POPPI online training was assessed as follows:

• the proportion of critical care unit staff members undertaking the course
• the proportion of staff completing and passing the end-of-course assessment.

Three stress support sessions for patients identified as acutely stressed

Feasibility to deliver the second element of the intervention (three stress support sessions for patients identified as acutely stressed) was assessed as follows:

• the proportion of eligible and consenting patients assessed with the IPAT
• the number of stress support sessions delivered
• the timing of delivery
• the duration of each stress support session
• feedback from the one-to-one debriefing and support sessions with the expert trainers.

In addition, the acceptability of the three stress support sessions was measured as follows:

• by patient-rated stress thermometer scores (collected before and after every stress support session); and
• by patient-completed satisfaction questionnaires (for those receiving at least two stress support sessions and recruited after the one day for feedback and assessment).

Relaxation and recovery programme for patients identified as acutely stressed

Feasibility to deliver the third element of the intervention [the relaxation and recovery programme (for use on tablet computer by patients between stress support sessions and on DVD at home after discharge with patient booklet)] was assessed by the POPPI nurse and patient feedback.

Final focus group

At the end of the intervention feasibility study, two focus groups were held with the POPPI nurses at each site to discuss their experiences.

Patient follow-up: randomised clinical trial procedures feasibility study

At RCT procedures feasibility study sites, consented patients were sent a follow-up questionnaire at 5 months post recruitment by the Intensive Care National Audit & Research Centre (ICNARC) Clinical Trials Unit (CTU) via its indicated, preferred route (via post or e-mail – indicated at consent). (Note that the intervention was not delivered in this study.) The questionnaire pack included the proposed instruments to be used in the cluster RCT: the EuroQol-5 Dimensions, five-level version (EQ-5D-5L);56 the PTSD Symptom Scale – Self-Report questionnaire (PSS-SR);57 the Center for Epidemiologic Studies Depression Scale (short form) (CES-D-10);58 and a health services questionnaire (providing information for the proposed integrated economic evaluation in the cluster RCT). For questionnaire packs sent via post, a pen and a self-addressed, stamped envelope were included. After 2 weeks, non-responders were telephoned to check receipt and to provide an option of responding over the telephone.

Both the response rate and the completeness of the primary outcome measure, the PSS-SR, were assessed – the latter through detailed inspection for missing data across the 17 items that make up the scale.

Prior to follow-up commencing, the primary outcome questionnaire was changed from the Post-traumatic Diagnostic Scale to the PSS-SR because of issues regarding copyright and requirements for formatting. We also changed the feasibility follow-up time period from 1 to 5 months to allow for closer replication of the cluster RCT (follow-up at 6 months). This ensured that we would have increased confidence over the expected follow-up rates when planning the cluster RCT.

We conducted a small, methodological substudy to elicit whether or not the burden of questionnaires (i.e. number of questionnaires) influenced the response rate by randomly allocating patients to receive either a full questionnaire pack (i.e. PSS-SR, CES-D-10, EQ-5D-5L and health services questionnaire) or solely the single PSS-SR required for the primary outcome.

Results

Research governance

The joint protocol for the two feasibility studies was submitted to the National Research Ethics Service (NRES) Committee South Central B – Oxford (reference number 14/SC/0149) on 25 February 2014. Favourable ethics opinion was granted on 23 April 2014 following response to requested minor modifications. Two further minor modifications occurred in May and September 2014 (these related to the timing and questionnaires used as part of the patient follow-up in the RCT procedures pilot study). The feasibility studies were adopted onto the NIHR CRN portfolio (16479) and registered on the ISRCTN Registry (ISRCTN61088114).

Recruitment to the intervention feasibility study

Sites and POPPI nurses

The two adult, general critical care units (sites) recruited were University College Hospital, London (a teaching hospital) and Watford General Hospital (a district general hospital). Both sites identified a PI and three POPPI nurses. Soon after delivery of the 3-day training course for the POPPI nurses, R&D approval at both sites was achieved, site initiation visits occurred and site activation and the start of patient screening commenced. In total, the sites were screened for 5.5 months.

Patients

Of 1246 patients screened, 315 were deemed potentially eligible, of which 99 eligible patients consented to participate in the intervention feasibility study; the Consolidated Standards of Reporting Trials (CONSORT) flow diagram can be seen in Figure 3. All patients were approached for informed consent while still in the critical care unit. Two patients provided written consent after discharge from the unit.

FIGURE 3 Intervention feasibility study CONSORT flow diagram. (Figure not reproduced here; see the original article.)

Relative to the day of admission, the median number of days for gaining consent was 5 [interquartile range (IQR) 4–7 days]. With an inclusion criterion of ‘received ≥ 48 hours of level 2 or 3 care’, of the 99 consented patients, 45 (45.5%) were consented within the next 48 hours (i.e. within 96 hours of admission) and 78 (78.8%) were consented within their first week in the critical care unit.

Feasibility assessment

To deliver the education package

POPPI online training

All POPPI nurses completed the online training prior to or soon after the 3-day training course. Local POPPI teams enumerated all clinical critical care unit staff and provided e-mail addresses for each staff member to the POPPI trial team, as registration for the POPPI online training. On the day following the 3-day course, each staff member was sent an e-mail invitation to undertake the online training. Uptake was monitored and staff were regularly prompted in person and via e-mail about completion.

In total, 283 staff members across both sites completed the POPPI online training. Five key learning objectives were self-assessed and each explored how much knowledge the individual undertaking felt that they had acquired during the online training course and was rated on a five-point scale from ‘not at all’ to ‘a lot’.

Feedback on the POPPI online training was provided by 260 (91.9%) critical care unit staff members who completed it. Four aspects were rated on a five-point scale: (1) stimulating to boring, (2) useful to useless, (3) well designed to not well designed and (4) just right (in length) to too long. Those completing the feedback were also asked to indicate any preference for their most and least liked aspect about the POPPI online training.

The POPPI online training was rated as stimulating/very stimulating by 72.7%, useful/very useful by 85.8%, well designed by 84.0% and just right (in length) by 80.0%. With regard to preferences about aspects of the course, factual information was the most liked aspect of the course (41.8% rated this as their most liked aspect), followed by videos from patients (34.8%), videos of staff–patient interactions (12.5%) and quizzes/assessments (10.2%). With regard to the least liked aspect of the course, most (69.3%) indicated there was no aspect that they least liked.

Overall, comments in response to open questions around content were positive and suggestions for improvement included ideas for additional content; additional clarification around the part dealing with how to assess patients with the IPAT; a clearer description of all the elements of the POPPI intervention and, specifically, the stress support sessions to be delivered by the POPPI nurses; clarity on where to access post-discharge information available for patients; graded, tougher assessments at the end of the online training course; and promotion of wider and repeated teaching opportunities (in the form of seminars and encouraging repeated use of the POPPI online training). Feedback was also provided on technical issues, emphasising the need to make access to the course as simple as possible and the absence of sound cards/speakers in NHS computers making the videos redundant and thus frustrating users (the use of subtitles or transcripts was suggested).

Other feedback provided to the POPPI trial team included the need for key messages, the need for key messages relevant to practice, a preference for practical rather than theoretical knowledge, the need for autogeneration of a certification following course completion and the need for improved design (with more colours, more graphics and larger but less text).

Three-day training course

All six POPPI nurses attended the 3-day course on the intervention. The 3-day training course was held from 20 to 22 October 2014 in central London. At the 3-day course, each POPPI nurse received training materials in both paper and electronic [on a USB (Universal Serial Bus) stick] format. At the end of the course, each POPPI nurse returned to their site with a tablet computer containing the relaxation and recovery programme for use by patients.

Pre and post assessment of self-efficacy was completed by all six POPPI nurses. Seven aspects were assessed; each explored how confident the POPPI nurse felt about each aspect and was rated on a five-point scale from ‘not at all confident’ to ‘very confident’. Although the sample was small, improvement in self-efficacy, in terms of increased confidence, was seen across six of the seven aspects assessed.

Self-assessment of key learning objectives was completed by all six POPPI nurses. Eight key learning objectives were assessed; each explored how much knowledge the POPPI nurse felt that they had acquired during the 3-day training course and was rated on a five-point scale from ‘nothing’ to ‘a lot’. Although the sample was small, acquisition of knowledge for all eight key learning objectives was indicated.

Feedback on the course was provided by all six POPPI nurses. Five aspects were rated on a five-point scale from stimulating to boring; useful to useless; relevant to irrelevant; well conducted to poorly conducted; and motivating to not motivating. All six POPPI nurses rated the 3-day training course as stimulating/very stimulating, useful/very useful, relevant/very relevant, well conducted/very well conducted and motivating/very motivating.

In response to open questions, positive feedback on the course included involvement of ex-patients (hearing their and other patients’ stories), incorporating practical sessions with one-to-one role plays, the openness in discussing shared experiences and knowledge, relevant course materials and the training team’s knowledge of theory and techniques. Suggestions for improvement included the challenge of role playing, the potential to use actors (rather than each other) in role plays, the challenge of the course content and materials (greater insistence on pre-course preparation was suggested), and some refocusing of course timings to the more challenging elements.

One day for feedback and assessment

All six POPPI nurses attended the 1 day for feedback and assessment, held on 13 January 2015 (for POPPI nurses from Watford General Hospital) and 15 January 2015 (for POPPI nurses from University College Hospital, London) in central London.

Assessments of self-efficacy were completed by all six POPPI nurses. As before, for the 3-day training course, seven aspects were assessed, exploring how confident the POPPI nurse felt about each, and were rated on a five-point scale from ‘not at all confident’ to ‘very confident’. For all six POPPI nurses, levels of confidence in each aspect were either sustained or improved.

Two expert trainers rated each of the six POPPI nurses conducting the second of the three stress support sessions (using an actor and conducted in a private room). Five of the six nurses passed their skills development assessment. During the assessment, feedback and support were provided. Feedback provided to the POPPI nurses was around continuous revision (through continuing use of the POPPI nurse training manual), the mode of delivery (less hesitancy) and the timing of delivery (increasing the time taken). Support from the expert trainers reinforced the value of practice and their confidence in their newly acquired skills.

One POPPI nurse, who did not pass on the day (predominantly as a result of nervousness around the notion of assessment) was referred for further training and support from the expert trainers. This occurred on-site and the POPPI nurse passed on a subsequent visit from the expert trainers on 22 January 2015.

At the 1-day focus group, reflections on the 3-day training course indicated that earlier engagement with the theory would have helped and would have made the course less hard and tiring. The POPPI nurses indicated that, during skills practice, they preferred the approach of dividing each stress support session into essential steps and then mastering each of these, in turn, prior to putting them all together to deliver the complete session. Demonstrations (by the expert trainers at the course or by videos of expert trainers) and the use of actors in role plays were suggested improvements. Clarity was requested over when to approach patients in the critical care unit, namely that the 48-hour mark was just the point at which to begin daily screening and that determining when a patient was ‘ready’ could be difficult. They said the consensus was that patients welcomed the opportunity to talk. One suggested approach was to leave the information leaflet on the first visit. All found the IPAT straightforward to use. All POPPI nurses said that training in delivery of the stress support sessions had made them much more aware of what patients might be experiencing and made it easier to start conversations with patients. It was felt that methods to achieve greater awareness by other staff of the POPPI study/POPPI nurse role, particularly relating to the stress support sessions, would help. The POPPI nurses felt that engagement with the study by other key staff in the unit (those at a senior level and those with responsibility for research, etc.) would be key to successful delivery of the POPPI cluster RCT. Issues raised about the tablet computer and DVD were around the music playing continuously (rather than as a track at a time) and directing patients to patients’ stories that were relevant to them. Regular teleconferences on wider study progress with other key staff, as well as individual debriefing and support sessions for the POPPI nurses, were seen as important. The POPPI nurses felt that the content of the online training course needed to be reinforced by others (team leaders, educators, etc.) in the unit. It was felt that creation of the therapeutic environment should be done by others, leaving more time for the POPPI nurses to deliver the stress support sessions.

To deliver the POPPI intervention

Creating a therapeutic environment in critical care

Uptake of the POPPI online training on creating a therapeutic environment was monitored closely. Overall, uptake for the 338 enumerated staff members across the two critical care units was 283 (84%). By job role, uptake was 92% for doctors, 83% for nurses and 76% for other allied health professionals. Overall uptake for each of the two critical care units was 81% and 87%, with uptake by doctors, nurses and allied health professionals following a similar pattern in both units.

Of those staff members taking the POPPI online training, 280 (98.9%) attempted the six-question assessment at the end of the course. Of these, 277 (97.9%) passed the assessment (achieving a score of ≥ 80%) and 3 (1.1%) failed the assessment and opted not to retake it.

Of those staff members passing the assessment, 128 (45.2%) passed first time and 138 (48.8%) passed second time. For 11 (3.9%) staff members, three or more attempts were required to pass. All staff members who passed the assessment received a certificate.

Three stress support sessions for patients identified as acutely stressed

All 99 (100%) eligible, consented and recruited patients were screened for acute stress with the IPAT. Of these, 40 (40.4%) were assessed as acutely stressed and being at a high risk of psychological morbidity (IPAT score of ≥ 7 points). The median score for acutely stressed patients was 10 (IQR 9–12). The median score for all patients was 5 (IQR 3–9) and for low stress patients was 3 (IQR 2–3).

For patients identified as acutely stressed, the median length of stay from consent to critical care unit discharge was 1 (IQR 0–4) day and to hospital discharge was 10 (IQR 6–26) days.

Of the 40 eligible patients identified as acutely stressed, 18 (45.0%) received all three stress support sessions, 12 (30.0%) received two, four (10.0%) received one and six (15.0%) received none of the stress support sessions (Figure 4).

FIGURE 4 Number of stress support sessions received by patients (n = 40). SSS, stress support session. (Figure not reproduced here; see the original article.)

Of the six patients who received no stress support session, one declined and one was discharged from hospital prior to delivery. The remaining four participants received no session(s) because of unavailability of a POPPI nurse to deliver the session(s) (see Site issues and resolution).

Of the four patients who received one stress support session, the second session was scheduled for two of them, but then postponed because of the patient’s conditions (one patient was too tired and was then discharged from hospital before the second session was rescheduled and the other participant deteriorated clinically and was no longer eligible). Of the other two patients, one was discharged from hospital before the second session occurred, and the second did not receive the second session, because they were busy when approached at the scheduled time. Later, the POPPI nurse was unavailable to deliver the session (see Site issues and resolution).

Of the 12 patients who received two stress support sessions, five were discharged from hospital before the third session could be conducted; two declined; three patients received the second and third sessions combined, in anticipation of hospital discharge; and two did not receive the third session because of unavailability of a POPPI nurse (see Site issues and resolution). Five of the 12 patients were provided with the relaxation and recovery DVD and the Getting Well, Staying Well booklet, due to be given during session three, prior to leaving the hospital.

Relative to the day of admission, the median day for gaining consent was 5 (IQR 4–7) and the median day for delivery of the first stress support session was 9 (IQR 7–12). Of the 34 patients who received at least one stress support session, the time from consent to first session was ≤ 2 days for 20 patients (58.8%) and, cumulatively, ≤ 4 days for 27 patients (79.4%). The first stress support session was delivered in the critical care unit for 15 (45.5%) of the patients. Of the 18 patients who received all three stress support sessions, 13 (72.2%) received them within 1 week, as promoted by the POPPI study. All stress support sessions were delivered to patients by the same trained POPPI nurse (i.e. one of the three POPPI nurses assigned to each patient).

The median duration for each stress support session was 35 (IQR 33–40) minutes for the first session, 30 (IQR 25–35) minutes for the second and 30 (IQR 30–40) minutes for the third session.

Ongoing debriefing and support indicated that the POPPI nurses found delivery of the stress support sessions rewarding and challenging. Issues requiring resolution arose around engaging patients, ensuring that patients understood what the sessions did (and did not) encompass, identifying when to refer patients on for further support and knowing how to handle patients revealing very serious matters unrelated to the critical care unit stay. At each debriefing and support session with POPPI nurses, the expert trainers encouraged the value of ongoing revision, reinforced the importance of seeking support from the expert trainers, acknowledged the particular challenge of delivering the content of the second stress support session, stressed the value of listening to the patient and confirmed the importance of sharing concerns about the patient with their clinical team.

Site issues and resolution

It became clear that, as time progressed, the POPPI nurses at one of the two sites were unable to commit the necessary time to the intervention feasibility study for a number of reasons: one reason was personal (compassionate leave), one reason was professional (attendance/involvement in another course/activity, confounded by the unavoidable delay in commencing the intervention feasibility study) and one nurse never really engaged. This clearly affected their availability for consent (see Figure 3) (patients were not consented if stress support sessions could not be offered) and ability to deliver the stress support sessions (see Figure 4).

To address this, the intervention feasibility study was put on hold at one site while four new POPPI nurses were rapidly identified and trained (i.e. attending the 3-day training course and the 1 day for feedback and assessment). The results for their pre and post assessment of self-efficacy and self-assessment of key learning objectives are included, along with those of the initial six nurses, in Table 2.

All four nurses passed their skills development assessment by two expert trainers. The feasibility study recommenced, and a further 28 eligible patients were recruited prior to the feasibility study end. Of these 28 eligible patients who consented to participate, 27 (96%) consented while still in the critical care unit. Relative to the day of admission, the median day for gaining consent was 5 (IQR 4–7). With an inclusion criterion of ‘received ≥ 48 hours of level 3 or 2 care’, of the 28, 12 (42.9%) were consented within the next 48 hours (i.e. within 96 hours of admission) and 22 (78.6%) were consented within their first week in the critical care unit.

All 28 (100%) eligible patients were screened for acute stress with the IPAT. Of these, 11 (39.3%) were identified as acutely stressed (IPAT score of ≥ 7 points). The median IPAT score for these patients was 9 (IQR 8–13). For these 11 patients, the median stay in the critical care unit from consent to unit discharge was 1 (IQR 1–2) day and to hospital discharge was 10 (IQR 5–20) days.

Of the 11 eligible patients assessed as being acutely stressed, seven (64%) received all three stress support sessions, two (18%) received two stress support sessions, one (9%) received one stress support session and one (9%) received none.

The one patient who received no stress support sessions withdrew consent. The one patient receiving only one session became confused and agitated and future sessions were deemed inappropriate. The two patients receiving two sessions were discharged from hospital before the third session could be conducted.

Of the 10 patients receiving the first stress support session, the time from consent to the first session was ≤ 2 days for all. For five patients (50%), the first session was delivered in the critical care unit. Relative to the day of admission, the median day for gaining consent was 5 (IQR 4–7) and the median day for delivery of the first stress support session was 11 (IQR 9–12). For the seven patients who received all three stress support sessions, all were delivered in 1 week by the same POPPI nurse, as promoted by the POPPI study.

The median duration for each stress support session was 30 (IQR 24–30) minutes for the first session, 30 (IQR 30–35) minutes for the second session and 25 (IQR 25–30) minutes for the third session.

Further feasibility indicators, based on the combined data for the 51 patients identified as acutely stressed and the 10 trained POPPI nurses, are presented in Table 2.

Patient acceptability

Patient acceptability of the three one-to-one stress support sessions are based on the combined data for the 51 patients identified as acutely stressed in the intervention feasibility study.

Stress thermometer scores were collected at the beginning and end of each stress support session. Stress thermometer scores ranged from 10 (very stressed) to 0 (very calm). The mean scores at the start and end of each stress support session were:

• stress support session 1 – start, 5.4 [standard deviation (SD) 2.6]; end, 5.0 (SD 2.7)
• stress support session 2 – start, 4.7 (SD 2.8); end, 3.6 (SD 2.4)
• stress support session 3 – start, 3.5 (SD 2.7); end, 2.7 (SD 2.7).

On average, each stress support session resulted in a small negative mean difference, –0.4 (SD 1.7), –1.1 (SD 1.8) and –0.8 (SD 1.4), respectively, indicating a small reduction in immediate acute stress. In addition, patients’ stress thermometer-rated acute stress also showed a negative mean difference [–2.6 (SD 3.3)] from the start of the first stress support session to the end of the third session for patients receiving all three sessions.

Following the 1-day course for feedback and assessment, when possible, patients recruited and receiving at least two stress support sessions were asked, independently of the POPPI nurses, to rate their satisfaction with the stress support sessions. Ten aspects were assessed, with seven rated on a five-point scale from ‘strongly agree’ to ‘strongly disagree’ and three rated on three different five-point scales: from ‘definitely too few’ to ‘definitely too many’, from ‘much too short’ to much too long’ and from ‘not at all useful’ to ‘very useful’.

Of 23 patients receiving at least two stress support sessions, 15 (65.2%) were given satisfaction questionnaires prior to hospital discharge and all 15 (100%) completed them (Figure 5).

FIGURE 5 Acceptability of the stress support sessions by patients (n = 15). (Figure not reproduced here; see the original article.)

In general, patients reported satisfaction with the stress support sessions and the majority felt that the number and duration were about right (with a minority preferring a greater duration) and, overall, that the support provided was useful. Some patients reported that the tablet computer was not easy to use.

A free-text box on the patient satisfaction questionnaires elicited comments:

Every hospital should have it.

It seemed a life-saver and really helpful to know the organisation is treating the subject so strongly.

I am glad to hear that notice has been taken of the many vital warning alarms in ICU [intensive care unit] . . . which increase patient stress.

I hope that POPPI will eventually be used in all ICUs [intensive care units]. Thank you to xxx for the support.

I feel the programme is such a good idea.

Stressful not knowing what’s going on.

Too many people coming in and out.

Comments on the intervention:

Really enjoyed the sessions.

. . . proves that I am normal but under the influence of morphine!

xxx was excellent and had explanations for what seemed a very dark world.

I have been given strategies that I will use in everyday life.

I had no idea how stressed I was feeling until I started talking to xxx about my experience.

Nurse was very good – felt less stressed about leaving hospital than last time.

Found the tape/music very relaxing.

Tablet computer difficult to use – didn’t always pick up commands.

Could have done with one more session.

Relaxation and recovery programme

The POPPI nurses reported variation in the use of the elements of the relaxation and recovery programme by patients. For example, some patients related best to the patient stories, some to the relaxation exercises, some to the meditation and some to the music. Some patients were apprehensive about using the tablet computer and required both encouragement and support in its use. Despite this support, some found it confusing/difficult to use, owing to a combination of unfamiliarity with the technology and reduced dexterity. Patients were pleased to be able to take home the DVD with the same relaxation and recovery programme for their ongoing use.

Final focus group

Following recruitment of the last patients, two final focus groups were held at the sites: one on 5 May 2015 with the POPPI nurses from University College Hospital, London, and the other on 8 May 2015 with the POPPI nurses from Watford General Hospital. The focus groups discussed the experiences of the POPPI nurses delivering the intervention.

The POPPI nurse role

All POPPI nurses indicated that they found the role enjoyable and beneficial (personally) yet also challenging. They indicated that it improved their communication skills and that they got a real boost (‘blown away by’) from delivering a session. Their main challenges were fitting the role around their clinical demands and inflexible rotas. As improvements, they suggested that better information on their role should be provided upfront, that the theory on psychological techniques should be provided ahead of the 3-day training course, that the concept of a ‘POPPI shift’ should be included into rotas and that, potentially, one POPPI nurse could be identified as the main/lead POPPI nurse in a unit and ensure peer support.

Most indicated that they received the right amount of ongoing debriefing and support from the training team, and that it was useful for reflection (particularly on harder cases). They found that reassurance in these debriefing sessions led to greater confidence. They felt that debriefing was best done as soon as possible, with some nurses suggesting that the first call could be booked at the 3-day training course, but several nurses also agreeing that it was best conducted after three full stress support sessions had been delivered by the POPPI nurse. They agreed that such debriefing and support sessions were difficult to arrange around shifts and that the expert trainers and the POPPI nurses would need to be flexible to make these happen. They agreed that one-to-one sessions with an expert trainer were important, but also thought joint and peer debriefing and support would be helpful too.

Education package

Three-day training course

All POPPI nurses indicated that the 3-day training course was very useful and that they felt privileged to be part of it. They liked the involvement of former patients and preferred when the learning was incremental (i.e. showing a video but using stop/start to discuss detail, learning the content of the stress support sessions in steps before putting them all together, etc.). All found the course hard work and tiring, the notion of assessment stressful, and felt that the skills practice would be better if they involved actors. The provision of scripts and a glossary were welcomed.

All POPPI nurses felt that the POPPI nurse training manual provided at the 3-day training course was invaluable as a backup but felt that some reordering around stress support sessions with relevant appendices would be welcome.

One day for feedback and assessment

The use of an actor on this day was welcomed (and they were pleased that their feedback had been listened to). All nurses found the skills assessment daunting, in anticipation, but knew that it was important. All indicated that the more that the assessment could be seen as ongoing debriefing and support, in a safe environment, the better – as all were seeking greater reassurance.

Online training course

The POPPI nurses felt that the online training did raise awareness and encouraged staff to think differently. They liked the fact that it was short (30 minutes duration) and that certificates were received following completion. With regard to content, they felt that the impact of critical care on long-term outcomes should be emphasised even more, and that some ‘raw’ patient stories should be used to engage the course-taker.

Intervention

Creating a therapeutic environment in critical care

The challenges of this element of the intervention included motivating others to do the online training, relieving staff to enable them to complete the online training, the difficulties of using Internet Explorer, version 6 (Microsoft Corporation, Redmond, WA, USA), and the absence of sound cards in NHS computers. Suggestions to improve uptake of the online training were that it should be incorporated into team study days (possibly mandated) and be available as a mobile phone app. For those who were less accustomed to using computers or electronic devices, training could be provided in a booklet or some form of buddying could be put in place.

With regard to microteaching, the POPPI nurses indicated that this had been very informal and that it should be more formalised and include involvement from others responsible for education within the unit.

Routine screening for acute stress with the Intensive Care Psychological Assessment Tool

Whether done by the POPPI nurses or by the research team, most felt that some dedicated training time would be good (and possibly to a wider group than just the POPPI nurses). Some felt that the IPAT questions on hallucinations, delusions and intrusive memories were more difficult to pose and that these should be covered more fully in the online training course, and in any dedicated training.

Three stress support sessions for patients identified as acutely stressed

All POPPI nurses indicated that the vast majority of patients seemed grateful for the stress support session(s) and enjoyed someone sitting and talking with them. All indicated that the language/approach required for the stress support sessions was very different from being a critical care nurse in clinical practice. It was hard to adjust to the practice of getting a patient to help themselves rather than them doing everything for the patient. Nevertheless, they appreciated the focus on patient empowerment. All complained that time, rotas and annual leave were a challenge to timely delivery and that delivery was more stressful when they were busy. Delivering the sessions was more stressful at the start because of their lack of confidence with the seemingly alien clinical approach. Suggested improvements included directing more focus at the second session, seen as the most challenging, and indicating during teaching that it was acceptable to deviate from the script using one’s own language. Two practical improvements noted were, first, the POPPI nurse should check the IPAT individual item scores (not just the overall score) and, second, in the situation in which hospital discharge was clearly close, that the second and third stress support sessions could, and should, be delivered together.

Relaxation and recovery programme for patients identified as acutely stressed

The POPPI nurses indicated that the tablet computers were used less than they imagined they would be. However, patients who did use them found them very beneficial – particularly if the selection of possible options (e.g. music, nature sounds, patient stories) was tailored to the individual patient by the POPPI nurse. All felt that wider knowledge by other hospital staff of the tablet computers’ existence would have increased use (e.g. bedside nurses could play a role in promoting their use). Challenges to their use appeared to be a result of dexterity (either because of age or current, post-intensive care health state). Some technological aspects (e.g. ease of increasing volume, size of icons, touch sensitivity) were also raised as challenges. The prospect of providing some support materials in other formats was also raised.

Randomised clinical trial procedures feasibility study

Sites and patients

Two sites and two PIs were recruited at the adult, general critical care units at Bristol Royal Infirmary (a teaching hospital) and at Medway Maritime Hospital (a district general hospital). Time from R&D approval to activation and start of screening took a total of 5 and 6 days at each site, respectively. Screening, consenting, recruitment and data collection commenced on 11 June 2014 at Bristol Royal Infirmary and 16 June 2014 at Medway Maritime Hospital, and both sites recruited patients for the required 2 months.

In total, 435 patients were screened between 11 June and 17 August 2014. Of these, 275 (63.2%) did not fulfil eligibility criteria or were deemed unable to give consent. Of the 160 eligible patients, 86 (53.8%) were consented (Figure 6).

FIGURE 6 The RCT procedures feasibility study CONSORT flow diagram. (Figure not reproduced here; see the original article.)

The estimated total patient recruitment was 44 patients, based on available data from the Case Mix Programme (CMP) – the national clinical audit for adult critical care co-ordinated by ICNARC – translating, on average, to 22 patients per site or 11 patients per month per site. The two sites recruited a total of 86 patients: 39 were recruited by the unit at Bristol Royal Infirmary and 47 by the unit at Medway Maritime Hospital (both operating as 19-bed, adult, general critical care units).

From the screening and enrolment logs, the reasons that eligible patients were not recruited were as follows:

• declined (n = 21)
• missed for recruitment (n = 48)
• it was deemed that research/questionnaire burden was too much for patients (n = 5).

The median number of days from critical care unit admission to consent was 3 (IQR 3–5) and from critical care unit admission to hospital discharge was 11 (IQR 6–28).

Follow-up

Of the 86 recruited patients, nine (10.5%) died by 5 months. Of the 77 patients surviving to 5 months, 62 (80.5%) responded – 47 by post, eight by e-mail and seven by telephone. Nine (11.7%) patients declined to complete the questionnaire and six (7.8%) were lost to follow-up (Figure 7).

FIGURE 7 Follow-up flow diagram. (Figure not reproduced here; see the original article.)

In the small, methodological substudy to determine if the burden of questionnaires influenced the response rate, overall response was 81.6% (31/38) for the questionnaire pack and 79.5% (31/39) for the single questionnaire.

Overall completeness of the primary outcome measure, the PSS-SR, was very good. For a total of 1054 fields (62 responses with 17 items each), only 24 (2.3%) had missing data. We undertook a psychometric evaluation of the results (Table 3).

Refinements

Selection of the POPPI nurses

We decided it was important to identify four, rather than three, POPPI nurses (three plus one reserve) allowing for subsequent education of a further POPPI nurse, as required. Updated person specification and selection criteria for POPPI nurses (using feedback from the POPPI intervention feasibility study) would be provided to sites to ensure selection and engagement of the best-placed individuals.

Following feedback from the intervention feasibility study, it became evident that it was important to engage earlier with the POPPI nurses prior to the 3-day training course. At this point, we would provide the POPPI nurses with accessible learning materials on the theory and practice of essential psychological techniques underlying the intervention for the POPPI cluster RCT, particularly with respect to the stress support sessions, in an easily digestible form. Additionally prior to the course, the POPPI nurses would undertake the POPPI online training and familiarise themselves with their POPPI nurse training manuals, the DVD with the relaxation and recovery programme and other teaching/trial materials supplied for local, bedside teaching and support. However, all materials would be used only after the 3-day training course. The POPPI trial team would engage those responsible for research and for education within the sites to support the POPPI nurses by identifying and enhancing opportunities for local teaching (e.g. bedside, seminars, workshops) and wider support of the POPPI cluster RCT.

Earlier engagement with the POPPI nurses and with those responsible for research and education within the sites was in response to feedback from the intervention feasibility study about the need for more background and preparation prior to the 3-day training course and the need for additional support for local bedside teaching and support within the unit for creating the therapeutic environment.

A site initiation meeting would also be used to engage the wider team prior to the intervention period. The POPPI trial team would use this visit to each of the intervention sites to cover the activities required of the site as they transition into the intervention period. This would be presented both to the POPPI nurses and to the local research and education teams (including the site PIs – one senior nurse and one senior doctor from the unit). This site initiation meeting would also cover delivery of creating a therapeutic environment in critical care (via the POPPI online training for all critical care unit staff, supplemented by other teaching and educational activities) and introduce the concept of routine screening for acute stress with the IPAT.

The early engagement with the POPPI nurses and the site initiation meeting from the POPPI trial team (covering delivery of two of the elements of the intervention) would reduce the burden of content delivery on the 3-day training course – allowing the course to focus on the practical delivery of the three stress support sessions.

Refinements to the education package

POPPI online training

Technical issues around problems balancing ease of access with security of access (i.e. only to staff at intervention group sites) were resolved early in the intervention feasibility study. The POPPI online training was updated following feedback; updates included a greater emphasis on key messages – both overall and directly for practice (a shift of emphasis from theory to practice); techniques to make it more engaging (e.g. less text, more graphics); and additional content (e.g. on delirium, sources of information post discharge for patients). Reminders of key messages would be sent to those completing the course. We added subtitles and transcripts for audio on video files. The revised, final, online training course lasted 30 minutes and comprised the following five sections:

1. understanding the stresses of intensive care patients
2. reducing stress and fear in patients
3. communicating with distressed patients
4. inspiring patients with confidence and hope
5. summary and assessment.

Three-day training course

The 3-day training course was focused even more strongly on delivery of the three stress support sessions. Teaching of theory would be delivered through quick revision sessions (following earlier engagement and provision of pre-course information). The number of skills practice opportunities (i.e. role playing of stress support sessions) was increased and actors were employed to assist in these. Greater focus and time were given to the more challenging elements of the stress support sessions and skills practice, with individual steps in each stress support session being mastered first, with feedback, prior to practical delivery of a whole session. Case studies and exercises would now cover a wider spectrum of patient situations and include stress support sessions that do not necessarily go as planned. Other content was also revised based on feedback. The same expert trainers would deliver the training course, including the former critical care patients, whose contribution was very popular. Course documentation has been tailored to the newly focused, practical content of the course.

Between the 3-day training course and the 1 day for feedback and assessment, debriefing and support would be available to the POPPI nurses – initially after the first patient to whom they deliver stress support sessions, and then at regular intervals thereafter, as the transition period evolved into the full intervention period.

The refined, slimmed-down, programme for the 3-day training course was as follows.

• Day 1:
  • Understanding critical care patients’ stress.
  • Learning the skills needed to deliver stress support sessions.
  • Overview of the stress support sessions.
  • Observing and practising session 1.
• Day 2:
  • More about unusual experiences in critical care.
  • Learning further stress support skills.
  • Observing and practising stress support session 2.
• Day 3:
  • Revision of key points from stress support session 2.
  • Observing and practising stress support session 3.
  • Using the patient booklet to create personal action plans.
  • Debriefing and support arrangements.

The three common components in every session were starting, building rapport and finishing. Starting a session included introductions, outlining aims, measuring stress and explaining confidentiality, note-taking and risk-reporting. Building rapport involved focusing throughout all the sessions on creating a relationship with the patient based on openness, honesty, warmth and listening. Finishing a session involved summarising, getting patient feedback, agreeing practice, remeasuring stress, agreeing wording for notes and arranging the next session.

We simplified the presentation of each session to consist of three individual components, rather than five, in the support tools. The individual components of the first stress support session were normalise reactions (discuss common psychological reactions and their causes in critical care), encourage communication (help patients open up about worries and concerns) and teach coping strategies (encourage patients to gather more information to address their concerns and use the relaxation programme) – acronym NET. The second session comprised stress reactions (encourage further talk about worries and fears, normalise concerns and notice stressful thoughts), explain stressful thinking (explain how unrealistic fears can create extra stress and identify one stressful thought that the patient agrees to work on) and teach ‘check out my fear’ (a technique for patients to find out if their fear is realistic or not) – acronym SET. The third session included summarise and review (key messages from sessions and persisting issues), action plan (create a personal plan to cope with challenges ahead, using the patient booklet and building on the stress sessions) and future expectations (encourage realistic optimism and hope about progress made and recovery) – acronym SAFe.

Documentation and support tools

All POPPI cluster RCT documentation and support tools for the education package were revisited. Specifically, the POPPI nurse training manual was split into an easy introduction to the theory (supplied in a booklet ahead of the 3-day training course) and a training manual focusing on practical delivery of the three stress support sessions, aligned to the 3-day training course. This resulted in improved linkage to appendices and greater focus on practical delivery in the training manual. The relaxation and recovery programme was improved and tablet computers with better touch sensitivity were identified for use and donated by Samsung Electronics (UK) Ltd (Chertsey, UK). If the materials were provided via DVD during stress support session 2, then more patients could take away the DVD regardless of whether or not they received the full three sessions. The Getting Well, Staying Well booklet will also be provided at the same time.

One day for feedback and assessment

The 1 day for feedback and assessment may be best delivered at sites, rather than centrally; feedback indicates that, if delivered locally, it will more likely to be seen by the POPPI nurses as a continuation of debriefing and support – while retaining an assessment of skills. In this way, assessment will become part of the debriefing and support process and assessment will be a one-to-one, face-to-face confirmation of skills session with the expert trainers. From experience, the feedback element of the day is best focused on local barriers (and solutions) to being a POPPI nurse, with feedback elicited in a focus group setting (with topic guides prepared from knowledge acquired in the POPPI intervention feasibility study).

Review of assumptions underlying the power calculation for the cluster randomised clinical trial

All assumptions underlying the sample size calculation and anticipated recruitment rate were reviewed in the light of the results of the two feasibility studies and the selection of sites for the POPPI cluster RCT (Table 4).

Based on the most recent 12 months’ data from the CMP, the throughput of potentially eligible patients (i.e. with a length of stay in the critical care unit of ≥ 48 hours and receiving some level 3 care) for the 24 selected sites was extremely similar to the average values from all units in the CMP used in the original calculations. This enabled us to concentrate on those patients who would most likely have the potential to benefit from the intervention (i.e. those receiving level 3 care, rather than level 2/level 3).

Of these potentially eligible patients, the percentages who met trial criteria and were able to give informed consent (i.e. regained capacity and were free of delirium) were 61.2% and 48.2% in the RCT procedures feasibility study and intervention feasibility study, respectively, compared with an original assumption of 60%. Therefore, this assumption was lowered to a more conservative 55% (average of the rates from the two feasibility studies).

Of those that met trial eligibility and were able to consent, 71.4% and 73.2% were approached for consent in the two feasibility studies, respectively. We therefore lowered the assumption for this percentage from 80% to 70%.

Of those approached for consent in the RCT procedures feasibility study, 74.8% gave informed consent and were recruited to the study. We therefore lowered the assumption for the percentage of patients approached that will give informed consent from 90% to 75%. We did not use the consent rate from the intervention feasibility study to validate this assumption, as recruitment to the intervention feasibility study also required the patient to consent to assessment with the IPAT and, if identified as acutely stressed, receiving stress support sessions, which will not be the case for the cluster RCT.

The percentages of patients who were alive at 5 months and who returned a completed follow-up questionnaire in the RCT procedures feasibility study were extremely similar to the original assumptions (89.5% vs. 90% alive, 80.5% vs. 80% completing follow-up). We therefore retained the original assumptions. From the returned questionnaires, both the mean and the standard deviation (SD) of the PSS-SR were considerably lower than those assumed in the original sample size calculation (mean 6.1 vs. 14, SD 7.5 vs. 12). We therefore revised the assumptions based on these data, which we also used to recalculate the Reliable Change Index (RCI) for the PSS-SR.

Patients who declined assessment with the IPAT were excluded from the intervention feasibility study. However, in the cluster RCT, such patients would be recruited (provided they consented to complete the follow-up questionnaire), but they would have minimal opportunity to benefit from all aspects of the intervention. Using the screening and enrolment log from one of the two feasibility sites (with detailed reasons for non-recruitment available), we estimated that if these patients had been recruited to the study but declined the assessment with the IPAT and, if stressed, stress support sessions; this would correspond to 16.2% of recruited patients. No allowance for this was made in the original sample size calculation, but we have reduced the minimal detectable difference accordingly.

In the original sample size calculation, it was assumed that 50% of recruited patients would be screened and assessed as acutely stressed. In the intervention feasibility study, this figure was 40.2%. We therefore reduced the minimal detectable difference accordingly. Consequently, for the revised sample size calculation, we wished to be able to detect a minimal difference in the mean PSS-SR of 2.9 points.

Results of review of assumptions

Based on the revised assumptions above, and with the original proposed design of 12 intervention and 12 control sites recruiting for a 5-month baseline period, a 1-month transition and a 5-month intervention period, the anticipated power from the proposed design remained at 91.5% (exceeding the target power of 90%). However, the total anticipated number of patients recruited was lower at 1914 (7.25 per site per month), as were the anticipated numbers assessed using the IPAT and receiving stress support sessions.

As a consequence of this review, there was no need to change the proposed design of the POPPI cluster RCT.

Confirmation of the pre-trial power calculation

The power calculation was completed using the approach of Hussey and Hughes59 to achieve 90% power to detect a reduction from 6 points to 3.1 points (p < 0.05) in the mean PSS-SR score at 6 months, and was based on the following assumptions:

• Mean (6) and SD (7.5) of the PSS-SR score taken from patients in the feasibility study.
• Estimated intracluster correlation coefficient (ICC) of 0.138 – between-site coefficient of variation 0.5, corresponding to between-site SD of 3 (this is a conservative estimate as no multicentre data were available).60 (The inclusion of a baseline recruitment period means that the sample size calculation is less sensitive to the degree of clustering.59)
• Treatment effect of a reduction of 2.9 points on the PSS-SR score based on a RCI for the PSS-SR of 8.6 points61 being observed in 40% of eligible patients in the intervention periods assessed as being at a high risk of psychological morbidity using the IPAT, with 16% of recruiting patients declining the intervention.
• Harmonic mean of the number of patients completing follow-up (52 per site per annum – corresponding to 22 in a 5-month period) based on data from the CMP.

With the design and the above assumptions, the estimated total number of patients recruited (based on CMP data) for the cluster RCT would be 1914 patients from the 24 sites. It was anticipated that 438 patients would be assessed for acute stress, using the IPAT, of which 175 (40%) would be assessed as being at a high risk of psychological morbidity and receive the stress support sessions.

Conclusion

The feasibility to deliver the education package and the intervention, and the procedures for the cluster RCT, were successfully tested. The feasibility studies identified a number of issues that were translated into refinements and collated in Refinements and Review of assumptions underlying the power calculation for the cluster randomised clinical trial. These included the following areas: POPPI nurse selection, the education package, and the support tools and materials. We also used the additional knowledge and results to further inform the design of the cluster RCT, including reviewing the assumptions of the sample size calculation.