Consensus exercise

Methods

This process was conducted over three phases:

• Phase 1: item generation using a ‘So what, now what’ focus group
• Phase 2: online modified Delphi over three rounds of iterative voting
• Phase 3: consensus meeting to confirm prioritisation of items.

Stakeholders

Two stakeholder groups were defined: patients with previous experience of PSD and clinicians with an interest in PSD. Clinicians included those with certificates of completion of training in general surgery and nurse specialists with wound care practice. Participants were recruited via e-mail, national organisations and social media. Snowball sampling was also used for clinician recruitment. Participants were invited to participate in one, two or all three phases of the consensus exercise.

Patient and public involvement

Fifteen patient representatives with relevant experience were recruited to the patient stakeholder group following substudy conception, and contributed to the delivery and analysis. Of these, 6/15 PPI representatives attended the initial workshop and supported the generation of the longlist of recommendations; 4/15 PPI representatives attended the virtual consensus meeting and highlighted the importance of ensuring the final set of recommendation statements were conceivable to a patient audience.

Generation of longlist

In accordance with Delphi methodology, the study consisted of three phases. In phase 1, an online workshop was conducted. This was based on Rolfe’s critical reflection model, ‘What? So What? Now What?’.118 In the ‘What?’ phase, data or information is presented. In this case, researchers presented findings from the cohort study, mixed-methods study and survey-based work. In the ‘So What?’ phase, participants are encouraged to reflect and discuss the information. Participants were asked to consider how the presented data reflected their experiences, and how this matched wider experiences. In the ‘Now What?’ phase, participants discuss how the information should be used to influence the next stage. Participants were asked to frame their ideas as statements related to policy or research ideas. In the workshop, the following data were presented: preliminary PITSTOP cohort data, a systematic review of classification systems,110 a mapping reviewing of PSD,119 the PITSTOP DCE survey109 and the PITSTOP mixed-methods study.30 Participants were asked to consider two questions: ‘How can we use this data to improve and/or inform clinical practice?’ and ‘What are the key research questions generated by this data?’. A longlist of potential practice and research recommendation statements was generated. The steering committee assessed the readability of these statements. Prior to attending the workshop, all participants received an information sheet and completed an online electronic consent form.

e-Delphi consensus

In phase 2, a three-round e-Delphi consensus was conducted. The Delphi surveys were delivered using Qualtrics. In round 1, all participants were e-mailed a participant information sheet and a link to the survey. Upon accessing the survey, participants were asked to complete an online consent form. The following information was captured: age, gender, demographics, ethnicity, e-mail address and stakeholder respondent group (patient, surgeon and specialist nurse). The longlist of recommendation statements was presented in a random order, and each statement was supplemented with a written summary to aid understanding. At the end of the survey, respondents were encouraged to propose any additional statements. Additional items were reviewed at the end of round 1.

In rounds 2 and 3, the remaining longlisted items were presented in random order. Ratings of items were reviewed after the close of each round. Respondents received an e-mail copy of results which included their vote and how that compared to each stakeholder group’s votes.

During each round, participants voted on the importance of each recommendation using a 9-point Likert scale (one being ‘not important’ and nine being ‘very important’). Recommendations were shortlisted if the following was satisfied: (1) > 70% participants within both stakeholder groups rate the recommendation as 7–9; or (2) 90% participants within a single stakeholder group rate the recommendation as 7–9. Recommendations that reached consensus after three rounds were considered at the consensus meeting. All items had to be rated to complete the surveys. Only those who completed a survey round were eligible to participate in the subsequent round. At the end of round 3, all participants were asked if they were interested in participating in the virtual consensus meeting.

Virtual consensus meeting

In phase 3, an online consensus meeting was held to finalise the set of recommendations. Invitations were issued to interested participants, with efforts made to encourage participation from members of the public/patients. A target of > 15 participants with at least three patient representatives was felt to be reasonable for this prioritisation exercise as it reflected proportions recruited to round 1 of the consensus. The meeting was held using the Google Meet™ videoconferencing platform (Google Inc., Mountain View, CA, USA). Electronic consent was taken prior to the meeting. Participants were presented with a total of 34 statements: 15 policy and 19 research. After the presentation, participants were instructed to complete two separate Qualtrics surveys. A constant sum question type was used to calculate the sum of scores for each statement. In the first survey, participants were asked to distribute 100 points between the 15 policy statements dependent on priority. This could be all points to a single item, an even division, or however the respondent felt appropriate, as long as 100 points were distributed. The total points allocated to each item were then calculated, and the five highest-scoring items were considered as top priorities. In the second survey, participants distributed 100 points between the 19 policy statements dependent on priority. The same approach to summing points was conducted.

Results

Longlisting of potential outcomes

Following the initial ‘So What, Now What’ workshop, 33 items were generated for the longlist by clinicians and patients. The flow of items is presented in Figure 8.

FIGURE 8 Flow of items through the e-Delphi. (Figure not reproduced here; see the original article.)

Delphi round 1

Consent forms were completed by 57 potential candidates from both stakeholder groups, and 57 completed round 1. Characteristics of respondents’ participation among stakeholder groups are presented in Table 21. This included 15 patients, 40 surgeons and 1 nurse specialist. A total of 33 items were considered for level of priority; 15 items reached a priori consensus for inclusion for the final consensus meeting. Respondents proposed a further 12 items for review. Outcomes voted on in each round, along with the proportion of each panel rating them 7–9, are presented in Appendix 3, Tables 35 and 36.

Delphi round 2

In round 2, 53 participants completed the survey. This included 14 patients, 38 surgeons and 1 nurse specialist, 95% of those who completed round 1. Respondents were sent a summary of voting patterns from the first round and were asked to vote on 30 items: 18 for reconsideration and an additional 12 statements proposed. Of these, 18 items met the a priori criteria for inclusion in the consensus meeting.

Delphi round 3

In round 3, 51 participants completed the survey. This included 14 patients, 36 surgeons and 1 nurse specialist. This reflected 96% of those who completed round 2 and 91% who completed round 1. One further item was carried to the consensus meeting.

Consensus meeting

The consensus meeting was attended by three patient representatives and 15 clinicians. One clinician withdrew during the meeting due to work commitments. The top five policy statements and research recommendations are presented in Table 22.

Discussion

Overview

Research in surgery has been much maligned over the years,120 and pilonidal disease is no exception.14 This is reflected in relatively weak or vague guidance to support practice. The Delphi we conducted has identified high-priority practice and research topics to guide the further development of the field. Consensus statements on practice topics strongly reflect the findings from previous sections. The top practice recommendation reflects the need to avoid harm related to interventions. This is supported by findings from the mixed-methods study which showed high levels of regret associated with poor wound healing. The cohort study supports the top two policy recommendations as it demonstrates the ongoing morbidity from poor wound healing after major procedures. In contrast, the third policy recommendation does not relate to clinical outcomes, but to the need to train surgeons in new techniques as highlighted by the clinician survey. This is particularly important if the top two priorities are to be achieved, as upskilling of surgeons will be required to facilitate techniques beyond ‘excise and leave open’. The need for a classification tool with clinical reference is clear. In other areas of colorectal surgery, classification tools and systems facilitate decision-making around treatment pathways.121,122 This inconsistency in PSD means that it can be difficult to compare outcomes between patients and surgeons due to an inconsistent baseline description and treatment selection. Finally, mixed-methods and cohort studies highlighted the importance of return to work in this typically young and economically active patient group.

The recommended research priorities provide direction on next steps. Priorities one and three discuss the potential design of a future RCT. These suggest that a pragmatic approach to design, using an umbrella-type approach with interventions grouped into severity or grade of procedure, would be a appropriate.123 We have seen similar approaches in other proctology studies.124,125 The fifth priority also demonstrates the need to understand interventions which work across the treatment pathway. These are not limited to surgical interventions, as adjuvant treatment such as hair removal, use of antibiotics and wound dressings may also play a role in this pathway.119 With this in mind, it may be more appropriate and efficient for a funder to commission a multiarm, multistage trial, with rerandomisation of patients who develop recurrence of PSD. The importance of measuring relevant outcomes in a consistent manner is emphasised, with the need for a core outcome set identified as a priority.126 In addition, participants highlighted the need for the development of patient-reported outcome measures (PROMs). PROMs are important in any core outcome set.127 A multidimensional PROM may include an assessment of return to work, wound healing and recurrent symptoms, all of which have been identified as essential in prior work packages.

Patient and public involvement

The consensus was based on the engagement of patients and members of the public. Initial findings were discussed with patient representatives, and these ideas were used when delivering the initial workshop. This was also attended by patients who were able to express their priorities. This engagement continued at all stages. During the final consensus meeting, patient representatives were invited to comment regularly to ensure their voice was considered in the final ranking.

Impact for policy-makers

This exercise sets out five clear policy statements which could form the basis for the development of future guidelines by informing PICO (population, intervention, control/comparison, outcome) development. The findings of limited training opportunities following qualification as a consultant surgeon may not be limited to this area, and policy-makers should be cognisant of this when commissioning services.

Impact for researchers

Researchers can use the findings of this study to direct future research. The directives here are for pragmatic trials established to address questions along the treatment pathway. Studies should also have a clear patient focus.

Conclusion

This consensus exercise has involved patients and clinicians to identify five key policy and five key research priorities. These should form the foundation of future work in the field.