Phase 4: consensus and future recommendations

Introduction

To inform the design of a definitive trial of ERM intervention within UK PICUs we consolidated the Phase 1–3 PERMIT study findings and presented them to key stakeholders. In addition, we developed a preliminary proposal for the design of a definitive effectiveness trial, incorporating the key feasibility points from Phase 3.

Study management

The work package was led by BRS. JMen led the PPIE component. Rebecca Wooley and Karla Hemmings (BCTU trial statisticians) performed sample size calculation. RF co-ordinated PICANet data analysis. The study management group participated in consensus conference and reviewed the future trial proposal.

Aims

This study aimed to:

1. provide parent perspectives on the results to ensure clarity in the summary of results for health-care professionals;
2. provide parent perspectives on the summary of results for parents of children who were PERMIT trial participants;
3. refine messages to share with parents/legal guardians who were not study participants;
4. obtain feedback on the importance of a future trial of ERM and the outcome measures we should include in a future trial;
5. propose a preliminary design of a future trial.

Method

This study consisted of three parts: (1) parent consensus meeting; (2) HCP meeting; and (3) study management group trial design.

Parent consensus meeting

We invited all parent/family member participants (n = 17) who consented and provided contact e-mail address details in Phase 3 to a virtual meeting (via Zoom) in January 2022. Invitations were sent in December 2021 and a reminder invitation in January 2022. The meeting included a presentation by BS (study design, rationale and key quantitative results), Natalie Read (results from site debriefs) and JMen (key results from healthcare professional and parent interviews).

The video presentation (30 minutes duration) was recorded, uploaded to a private web-based video-housing channel and a summarised leaflet with key study findings (pdf format) was created.

Following the meeting all parents/family members were sent the slides and video link with a short online questionnaire relating to feasibility and acceptability. No sensitive or identifiable information was collected. Implied consent was indicated on completion and submission of the online questionnaire.

The questionnaire asked:

1. whether they found it useful to see the study results;
2. appropriateness of an opt-out consent in future study;
3. which outcome measures should be included in a future study (ranked according to importance); the amount of ERM a child received (dose);the type of ERM activities a child received;the length of time spent on PICU;the length of time on a ventilator;the LOS in hospital;physical measure such as muscle strength;an outcome reflecting longer recovery (e.g. return to nursery/school);
4. other suggestions of what should be measured (free-text response);
5. whether a future trial of ERM was required.

Healthcare practitioner consensus meeting

We planned to hold an online HCP consensus meeting to discuss a proposed future trial design. However, additional COVID pandemic restrictions and NHS workforce pressures in January 2022 resulted in the cancellation of this meeting, with a plan to reschedule in May 2022 during the national PCCS-SG annual investigators meeting.

Study management group trial design

The study management group met on three occasions to review the findings of the Phase 3 feasibility study and discuss the design and structure of the future trial. Additional methodological expertise was obtained from Birmingham Clinical Trials Unit statisticians and study team at PICANet. Meetings were conducted over Zoom and, following the verbal consent of all attendees, recorded for comprehensive notes to be made to inform trial development.

Results

Parent consensus meeting

An invitation to attend the parent consensus meeting was sent to 17 participants in the Phase 3 feasibility study with the recorded materials. No parents/legal guardians (n = 0/17, 0%) attended the live virtual meeting. However, four parents/legal guardians responded to the online survey. All respondents (n = 4) found it useful to see the key study findings and agreed that an opt-out model of consent was appropriate for a future study.

The outcome measures were rated as follows.

All four respondents rated as very important:

1. the amount of ERM a child received (dose).

In addition, they ranked from highest to lowest the following outcomes:

1. the type of ERM activities a child received (highest ranked);
2. the length of time spent on PICU;
3. the length of time on a ventilator;
4. the LOS in hospital;
5. physical measure such as muscle strength;
6. an outcome reflecting longer recovery, e.g. return to nursery/school (lowest ranked).

Other measures suggested in free-text response included:

For younger children/babies perhaps a measure of stress/anxiety – do erm activities such as reading help with sat [oxygen saturation] stability or overall sats [oxygen saturation] vs. children who do not receive ERM. E.g.: we continued to read night-time stories to try and maintain a degree of normality as well as comfort.

Two parents replied Yes: further research on ERM was required and two said No: reasons being that they felt the evidence was conclusive:

Probably not, the benefits are obvious and the feedback is conclusive I would say. (Parent D) I guess to get this to be a standard of care you will need more evidence it works. But with the positive feedback from parents I personally think enough research has been done – it provides a positive environment for all.

Impact for future paediatric early rehabilitation and mobilisation during intensive care trial

Parent satisfaction is extremely high surrounding ERM and contributed to two parents feeling that from their perspective further research was not required. However, there was recognition that for this to become a standard of care for all patients this would require more evidence, with further work focusing on identification of outcome measures relevant to patients and parents/legal guardians.

Study management group trial design proposal

The study management group, with trial methodologist input, met, reviewed all the PERMIT study findings and discussed an overall proposal for a future trial design.

Importance of research question and professional support for early rehabilitation and mobilisation

There remains a clear need to improve the morbidity and mortality of children requiring care within PICU after PICU critical surgery, illness or injury. We identified evidence from our scoping review that ERM is safe, feasible and, in combination with the adult literature, has the potential to be effective at improving patients’ recovery from critical illness; however, further trials are needed. We identified significant HCP enthusiasm, across the multidisciplinary groups within PICU, regarding the role and potential of ERM intervention. The Phase 1b survey demonstrated wide community buy-in for future studies, the Phase 1c observational study had a high level of interest, investment of time and participation by bedside nursing staff and therapists, and the speed and number of PERMIT champions recruited in Phase 3 was very reassuring. Establishing ERM as a credible PICU intervention through a future definitive study is clearly a high priority with NHS staff.

Intervention

The development of the PERMIT manual and the successful, safe, implementation of the manual within the PERMIT Phase 3 feasibility study has confirmed that a complex, multidisciplinary, PICU-wide ERM intervention can be evaluated. The PERMIT manual includes a six-step implementation guide, learning and training resources, and ongoing site support from ERM experts. It incorporates daily acuity scoring, activity-level guidance and a pause and assess safety structure, which are modified to local resources, environment and staffing structure. Through research staff investment, debriefs and local site involvement, we have established the initial implementation of the manual takes approximately 12 weeks. This time period should be integrated into any implementation training phase in a trial.

Trial design and consent model

As with other trials in PICUs, the most powerful trial would use a stepped-wedged cluster design. Not only would this increase the power of the trial, it would also mean that all PICUs involved would benefit from training in the intervention. This design has worked well in PICU previously; the HTA-funded SANDWICH trial66 of a sedation and ventilator liberation protocol intervention recruited 8843 patients across 18 UK PICUs over 18 months.

Opt-out consent is the preferred model of consent. This was successful in the Phase 1 observational study and strongly recommend in the Phase 3 process evaluation, interviews and PPIE work. This consent model would work within a cluster RCT design and implementation of the PERMIT manual and ERM delivery across the whole PICU.

Population

The PERMIT programme of research has clearly shown that elements of ERM are delivered in some form to nearly all patients in PICU, including the sickest patients on maximal organ support. Therefore, the whole diverse spectrum of PICU patients should have the opportunity to benefit from ERM early in their PICU care pathway and for as long as is required. However, for ERM to impact a measurable patient-related outcome it would be logical that a period of exposure to the ERM intervention is required. Therefore, patients with short stays in PICU (e.g. <48 hours) are unlikely to gain measurable benefit. Our approach of including patients on day 3 of PICU in the Phase 1c observational study and Phase 3 feasibility study allowed identification of this patient group. Trial inclusion criteria should allow selection of these patients for assessment of efficacy of ERM, although the ERM intervention could and ideally should be delivered to every patient within the PICU to ensure maximal embedding of the ERM manual and processes with staff. We therefore recommend that a future trial population includes critically ill/injured patients of all ages (0–<18 years) admitted to PICU. They would be identified and enrolled in the study early in their PICU stay (e.g. within 72 hours after PICU admission). This would allow the interventions in the PERMIT manual to be commenced in patients who could receive an adequate amount of ERM, which may lead to measurable benefit. Patients would be excluded by parents/guardians opting out, or clinicians’ decision that ERM or trial inclusion is inappropriate (e.g. anticipated death in PICU, or planned palliative care and withdrawal of life-sustaining therapies). We recognise that ERM activities may be important and recommended end-of-life interventions in this population (e.g. part of pain relief, enrichment to surroundings patient and family PICU experience); however, these would require different patient-reported outcomes or to be delivered outside of a trial setting.

Comparison

The comparison (control arm) would be standard of care before implementation of the PERMIT manual. We have established that even PICUs with an existing ERM programme can benefit and improve with the PERMIT manual and programme (e.g. Site 1 in the Phase 3 feasibility study). The ability for the PERMIT manual to work alongside and enhance, refine and develop existing programmes would allow increased recruitment of PICU sites to a future study. The research programme and process evaluation are also attractive to PICU sites as additional resources would be available.

Outcomes

Through PPIE and parent/family feedback, the amount and dose of ERM were identified as the most important outcome measure. However, we identified in Phases 1 and 3 that it is very difficult to quantify ERM dose due to (1) a heterogeneous patient population (e.g. 0–<18 years of age, developmental stages, size, ambulatory status), and (2) diverse types of ERM activities delivered. Inclusion of health-related QoL outcome measures is recommended in PICU studies47 and is felt important by healthcare professionals and parents/carers; however, the burden on families to complete these tools was clear in Phase 3. Therefore, a pragmatic measure of improvement in critical illness or injury recovery would be required, with accompanying intervention fidelity assessment (i.e. measurement of increased ERM dose delivery at the site and patient levels). Potentially suitable outcomes include length of MV and LOS in PICU (or days free of ventilation/PICU). The advantages to length of MV as an outcome is that it has a clearly defined beginning and end point (e.g. successful extubation without the need for re-intubation). It is also the primary outcome measure in the ongoing pilot, stepped-wedge trial of ERM in the USA PICU UP! study, which would allow direct comparison.112 The disadvantage is that this will exclude up to 35% of PICU patients with critical illness not requiring ventilation. LOS, as an outcome measure, would allow all patients admitted to PICU to be included in a trial. However, the disadvantage is that the end point is less precise. Discharge from PICU can be to any of step-down to high-dependency unit, ward location, other hospital PICU, neonatal intensive care, home, palliative care facility. The timing of this discharge can be affected by additional factors external to improvement in critical illness and injury (e.g. bed availability on the ward) and is therefore subject to measurement inaccuracy, which may affect trial findings.

With no definitive primary outcome choice, we present a draft proposal based on LOS and length of ventilation (LOV). We need to explore this area further.

Sample size

Approximately 20,000 patients are admitted to PICU each year, of whom around 65% receive invasive MV at admission. The eligible population, using LOV as an outcome, would come from the patients who are on MV on day 3 and stay in ICU for an additional 4 days (this gives us the population who are both most likely to benefit and can receive at least a minimal amount of ERM). The UK national PICANET data report that approximately 19% of patients admitted fulfil these criteria, so 2470 patients will be eligible per year.67 Allowing for ~5% attrition, 2350 patients will be available for analysis. This equates to approximately seven eligible participants per 1-month period per cluster.

There is a total of 26 suitable PICUs across the UK. Provisional interest in participation in a full PERMIT study is high, and it is reasonable to assume that 21 PICUs would want to participate, which could also include the three sites in the feasibility study. This trial design requires that all participating PICUs begin the control phase of the trial when the data-collection period begins. Further analysis will be required to accurately estimate required parameters for a stepped-wedge cluster randomised trial including the intracluster coefficient (ICC) and cluster autocorrelation coefficient (CAC). Provisional data from BCU PICU provided a mean length of time on MV as 19.3 days for those in PICU more than 7 days, with a SD of 28.4. A reduction of 1.5 days on MV is considered to be meaningful to patients and parents. The SANDWICH trial66 estimated the ICC to be 0.005. Personal correspondence from the USA PICU Up! trial and SANDWICH trial66 leaders suggests that the CAC may be close to 1.

Our proposed design is in shown in Figure 17. Similar to the SANDWICH trial protocol (adapted under CC BY 4.0 license http://creativecommons.org/licenses/by/4.0/),113 there will be an initial 3-month period of baseline data collection during which the PICU will not be exposed to the intervention. Following this, every 3 months, three sites will be randomly selected to transition to the intervention. This will start a 3-month training period of the intervention. As we cannot assume that the PICUs are exposed or not exposed to the intervention during the training, no patients will be recruited through this 3-month period. Once a PICU crosses over to the intervention, it will remain exposed to the intervention for the remaining duration of the study. There will be a final 12-month period during which all PICUs will be fully exposed after the last PICU has transitioned to the intervention. This 12-month period is to ensure that the ERM programme is fully embedded following implementation, which is a key requirement expressed by clinicians to be part of PERMIT and was not achieved in the feasibility study. Assuming an average of seven patients per 1-month period, and total follow-up period of 36 months, 80% power would be achieved. This would equate to a sample size of 4851 participants.

FIGURE 17 PERMIT trial design cluster, stepped-wedge. (Figure not reproduced here; see the original article.)

If LOS is used as primary outcome this increases the eligible population to 3800 (19% of 20,000) (Table 30). Allowing for 5% attrition provides a population size of approximately 3600 patients and approximately 11 participants per 1-month period per cluster. The same data from BCH PICU provide a mean estimate of LOS to be 23.7 with a SD of 31.0. Using the same design matrix as for length of time on MV, and an ICC of 0.02 estimated from the PICANet data, the sample size of 7623 would provide over 80% power to detect a 1.5 reduction in LOS.

There will be an initial 3-month pre-implementation block for 21 clusters (e.g. PICUs), stepping into a 3-month training block (staggered for 3 clusters at a time) with 12 months or more post-implementation period.

A further sample size calculation will be performed using the whole PICANET data set with LOV outcome for UK population. At the time of report submission these data were not available because of a cyber-incident on the Leeds University server.

Additional consideration

Chapter 7 concludes with a list of trial elements which will be important to incorporate into a future trial design (see Tables 27, 28 and 29).

In addition, further evaluation of the sample size and primary outcome will be required to justify the size and scale of a definitive trial. We acknowledge that plausible effect size for the primary outcome will also need to be estimated in an internal or external pilot phase of a future study.

Important considerations are needed on the potential confounders which may occur in a large pragmatic trial. The risk of education contamination bias, for example retaining separation between arms of the trial, is important. The huge investment of staff time, resources and effort that we identified during the Phase 3 feasibility study would be replicated in a definitive trial in each PICU. The risk of this occurring outside of the conduct of the trial and at the correct step in the design we believe is low. However, we acknowledge that PICUs in the UK have implemented ERM programmes in various forms, but we also recognise that those PICUs have struggled with maintaining programmes, without well-theorised implementation programmes like PERMIT (e.g. Birmingham and the MOVE4WARD programme).

Finally, the PERMIT feasibility study was conducted with a limited budget and although we provided research support staff funding for data collection and trial conduct at each site, we did not provide resources for any additional ERM delivery by therapists or nursing staff. In addition, we did not undertake an economic evaluation or involve a health economist in the PERMIT feasibility study as per the original commissioning brief. In a future trial, additional NHS support costs will be needed for adequate staff support of both the implementation and the delivery of ERM within clinical care, as identified in Chapter 6: Process evaluation.

Summary of findings to inform paediatric early rehabilitation and mobilisation during intensive care study

• The PERMIT manual is a complex intervention and is best assessed through a clustered stepped-wedge RCT in PICU.
• Following consultation, a draft proposal for the clustered stepped-wedge RCT in PICU is proposed.
• There is strong support for a definitive efficacy trial of ERM from HCPs. Feedback from participants for the PERMIT study, although limited in number, also supports a future trial and suggests important design and measurement elements.
• The primary outcome of LOV is a pragmatic compromise on measurable PICU outcome and likely accurate measure of improvement in critical illness recovery. Further consensus work in developing the primary outcome will be required with the UK PCCS-SG and trialist prior to a definitive study proposal.