Methods

This investigation was a follow-on study and used the same theoretical framing and methods as those used by the research team in the original study of PAs in primary care.17 As an applied health service research study, we used an evaluative framework described by Donabedian49 and applied it to the UK setting by Maxwell.50 The contribution of PAs, as new types of personnel, was investigated through the dimensions of effectiveness, appropriateness, equity (fairness), efficiency, acceptability and cost in secondary health care.49,50 The interactions within and between the macro, meso and micro levels of the health-care system in supporting or inhibiting the adoption of PAs as an innovation were also investigated.51 Furthermore, the study was framed by an awareness of theories concerning substitution and supplementation to task shift from one group of professionals to another30,52 and the potential for contest between professional groups.53 Overall, the study employed a mixed-methods approach54 in four interlinked workstreams:

1. Investigation of the extent of the adoption, deployment and role of PAs in hospital medical teams through two national electronic surveys: one to MDs of acute trusts and one to PAs (addressed research questions 1 and 2 at the macro and meso levels of the health-care system).
2. Investigation of the evidence of the impact of PAs and factors supporting or inhibiting the adoption of PAs at the macro and meso levels of the system through a review of published evidence for the specialties that most commonly reported employing PAs in the national surveys (workstream 1) and updating the published policy review from the prior study17 (addressed research questions 1, 3 and 4).
3. Investigation, through case study methodology in six hospitals employing PAs, of the impact, contribution and consequences of PAs in the medical teams. This included interviews with patients, managers and team and service members as well as requests for routine management data and observation of PAs at work. It also included a comparison of patient outcomes and service costs in EDs by PAs and Foundation Year 2 (FY2) junior doctors, where these two professional groups are deployed interchangeably (addressed research questions 1–4).
4. A synthesis of evidence from the three workstreams. This was presented and tested at an emerging-findings workshop with invitees from the research participants, the patient and public forum members and other advisors to the study (addressed the overarching study aim).

The following section describes the process of patient and public involvement. The methods of the study itself are then detailed.

Patient and public involvement

The patient and public voice was important to this study. The public and patient representative forum for the previous primary care study17 stressed that the innovation of bringing in a new group of professionals was of concern to the public. Key issues that the forum emphasised in discussion concerned (1) patient choice of the professional to consult or be attended by, (2) whether or not some groups of patients will receive a ‘second-class ‘or inferior service, (3) how patients will judge if this new professional is competent and (4) the ways in which the public and patients are informed of and understand this new role in different settings. Those issues were incorporated in the design of this study.

Within the study itself, the patient and public voice was interwoven in the following ways. First, Sally Brearley, as a public voice representative, was a co-applicant and member of the research team and the study paid for her time. Second, the study advisory group had two public voice members who were reimbursed for their time, following NIHR INVOLVE guidance.55 Third, two patient and public voice groups were formed: one in London and the other in the West Midlands. Invitations to join were sent to the members of the patient and public group forum involved in the previous primary care study, through the public and patients network of the Centre for Public Engagement (at the Joint Faculty, Kingston University London and St George’s, University of London). Invitations were also circulated through an established patient involvement group of the University Hospitals Birmingham NHS Foundation Trust, Birmingham. Members of these groups who attended research meetings were reimbursed for their time, again following NIHR INVOLVE guidance.55 The patient and public voice groups met first of all to hear about the study and meet a PA, who was able to describe their work and education and answer questions. The early meetings were used to inform the interview topic guides and interview questions. Later meetings brought members together to help inform the analysis and interpretation of interview transcripts. All patient and public representatives were invited to participate in the final emerging-findings workshop (workstream 4).

We now describe the methods in detail.

Workstream 1: investigating the extent of the adoption, deployment and role of physician associates in hospital medical teams

Two electronic, descriptive, self-report surveys56 using Survey Monkey^® (San Mateo, CA, USA) were conducted to map and describe the employment and deployment of PAs in secondary care services in England.

The first survey was of NHS MDs. It was designed by the research team and piloted by a MD. The survey addressed questions of employment and factors supporting or inhibiting that (see Appendix 2). The sample was of MDs in all acute and mental health NHS trusts in England as listed by NHS Choices (the NHS information portal) in December 2015. Contact details were obtained from Binley’s database.57 The invitation was sent by e-mail in December 2015, and two reminder e-mails were sent.

The second survey was sent to the UK voluntary register and/or UK-trained PAs. It was adapted from the primary care study58 by the research team and piloted by a PA board member of the FPA (see Appendix 3). The questions addressed the PAs’ work settings, activities and supervision. E-mail invitations were sent in February 2016 by the FPA to its members and by university course directors to their alumni. Two reminder emails were sent.

Anonymous responses to both surveys were imported into IBM SPSS Statistics Version 23 (IBM Corporation, Armonk, NY, USA). Data from closed questions were used to produce frequency counts and open responses were analysed for thematic groupings.

This workstream was approved by the Faculty of Health, Social Care and Education Research Ethics Committee at Kingston University London and St George’s, University of London.

All respondents were invited to complete a separate Survey Monkey page (following a link from the completed survey) to provide contact details if they wished to be kept informed of the study progress and outputs.

The findings from this workstream informed workstreams 2 and 3.

Workstream 2: investigating evidence of the impact and factors supporting or inhibiting the adoption of physician associates in the literature and policy

We undertook a systematic review and a policy review.

The systematic review

A systematic review investigated the impact of PAs on patients’ experiences and outcomes, service organisation, working practices, costs and other professional groups for the secondary care specialties of acute medicine, care of the elderly, emergency medicine, trauma and orthopaedics and mental health. These were the specialties that PAs were most frequently reported to work in by the FPA59 and this was confirmed in workstream 1.

The review was conducted as per the international Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.60 The protocol was published on the International Prospective Register of Systematic Reviews (PROSPERO) as CRD42016032895.61 In summary, nine electronic databases were systematically searched from 1 January 1995 to the second week of December 2015. The research team undertook periodic update searches, with the last search undertaken on 5 January 2018.

The databases were MEDLINE (via Ovid), EMBASE (via Ovid), Applied Social Sciences Index and Abstracts, Cumulative Index to Nursing and Allied Health Literature Plus (via EBSCOhost), Scopus version 4 (via Elsevier), PsycINFO, Social Policy and Practice (via Ovid), EconLit (via EBSCOhost) and Cochrane Central Register of Controlled Trials. A search strategy is presented in Appendix 4. Lateral searches were also undertaken.62 Relevant studies were identified first through abstract screening and then through full-text reading by two researchers, with any disagreements resolved by a third researcher. The inclusion criteria were:

• Articles had to have been peer reviewed.
• Population – PAs in accordance with the UK definition.20
• Intervention – the implementation of PAs in the following secondary health-care specialties: acute medicine, care of the elderly, emergency medicine, mental health and trauma and orthopaedics.
• Comparison – the comparison group was any health-care professional with whom PAs were compared.
• Outcome – any measure of impact, informed by recognised dimensions of quality (effectiveness, efficiency, acceptability, access, equity and relevance).50
• Study design – any study design that allowed the measurement of the impact of PAs in a primary study.

Articles were excluded if they were not published in English; were from countries not defined by the International Monetary Fund as advanced economies;63 did not provide empirical data; did not provide data for PAs separately from other advanced clinical practitioners, such as nurse practitioners; did not focus on the PAs as the intervention; only provided data before and after a service redesign or educational programme; or only presented literature reviews or commentary.

A data extraction spreadsheet was populated by two researchers independently, with differences resolved by a third researcher. Quality assessment was undertaken with Qualsyst checklists for quantitative and qualitative studies,64 with additional questions from the mixed-methods appraisal tool.65 No study was excluded on the basis of its quality score, but the limitations of lower-quality evidence were considered in the final synthesis.66

Owing to the heterogeneity of the included studies, a meta-analysis was not possible, and the results are presented as a narrative synthesis.67

The policy review

The previous study reported on a review of policy (up to 2013) to identify supporting or hindering factors in the development of the PA profession in the English health-care workforce.17,18 This policy review68 was therefore updated with a particular focus on secondary care using documentary review methods.69 Internet searches of relevant English government, NHS and associated agencies’ websites were conducted periodically throughout the study period to identify relevant policy documents and reports on health-care workforce planning, education, regulation and development. The time period was September 2013 to October 2017. These electronic documents were then searched using the ‘find’ function for any references to PAs. A data extraction framework, as used in the previous primary care study, was used to classify the document, note the presence or absence of PAs and record text relevant to the problem analysis that PAs were the policy solution,68 as well implementation concerning the education, employment and deployment of PAs. A narrative synthesis was then undertaken.

Workstream 3: investigating the deployment and contribution of physician associates at the micro level of the health system

A mixed-methods case study design70 addressed all of the research questions at the micro level of the health-care system through investigation in six NHS trusts that employed PAs in their acute care hospitals. Similar methods have been used in other studies investigating new staff groups in NHS hospitals.71,72 The mixed-methods approach within a multiple case study design70 was designed to both describe and, as far as possible, quantify the impact of PAs in the context of secondary care. The aim was to achieve diversity in the hospitals by geographical location, size and type. At the time of the application for funding, the research team had agreement in principle to participate from trusts in the West Midlands and London that employed PAs. The intention was also to have diversity in the medical and surgical specialties within which PAs were employed. The final decisions regarding the inclusion of hospital trusts in the study was informed by the surveys in workstream 1, as well as the willingness of the trust senior managers and PAs and their consultants to participate. In addition, as PAs are a relatively small staff group and, therefore, potentially identifiable, the design ensured that volunteer PAs came from a specialty in which more than one PA and more than one trust participated. The aim was also to include more than two hospitals employing PAs in emergency medicine (being the acute specialty reported to employ the largest number of PAs).1 The intention was to report the evidence from the multiple case studies in the manner described by Yin70 as the ‘fourth way’, that is to only present cross-case analysis of the issues of interest rather than from each individual case study site. With this understanding of the reporting intentions, the research team approached hospital executive staff to explain the research and gain formal permission to undertake the study; this was gained from a NHS trust board-level executive (chief executive or MD). Initial meetings were then held with PAs and their consultants to explain the study and ask for volunteers to participate. The sites were recruited sequentially but with data collection being undertaken in two or three sites at a time over a period of between 4 and 6 months. Permission to start data collection was in the control of each hospital research governance team and, in some instances, it took 6 months from the point of overall permission being given to agree. Characteristics of the six hospital trusts are provided in Table 1. At the time of the study, the Care Quality Commission (CQC) rated the overall quality of one of the trusts as ‘outstanding’, one as ‘good’ and the other four as ‘requires improvement’.

The characteristics of some aspects of the medical staffing and training in each of the hospital trusts are given in Table 2.

In overview, data collection in each hospital trust included:

• interviews with senior managers (including operations directors and human resources), lead consultants, members of the health-care teams (medical, nursing and support staff), PAs and patients
• requests for relevant routine management information (data and reports) as well as any internal documents, reports or audits on the work or impact of the PAs
• work activity diaries and observation of the PAs at work
• for those with PAs working in the EDs, anonymised patient records to allow comparison of patient outcomes and costs for the treatment of patients by PAs and FY2 doctors in the ED, a setting where individual clinicians are allocated to conduct an assessment and propose a plan for the patient.

An overview of the numbers of PAs volunteering to participate and the data collection in each site is provided in Table 3.

Semistructured interviews

Semistructured interviews were conducted with trust executive-level managers, lead consultants, members of the health-care teams (medical, nursing and support staff), PAs and patients. Topic guides75 appropriate to each group were developed in consultation with the advisory group and the study patient and public forum (see Appendix 5). The topic guides for patients included questions about the patient experience and perceptions of the role as well as acceptability. For managers and senior clinicians, topic guides included questions on factors inhibiting and supporting the employment of PAs as well as the impact on organisation, patient outcomes and costs. The topic guides for professionals included questions on deployment, acceptability and impact on working practices, role boundaries and patient experience. Each group was invited in different ways: senior staff were invited by e-mail and other staff were invited following introductory meetings or introductions by other members of the clinical team. Patients were approached by the clinical team in the first instance. With permission, the interviews were digitally recorded, or notes were taken if preferred. Recordings were transcribed, and the recordings were destroyed at the completion of the study. Transcripts and notes were anonymised. The transcriptions and notes were coded using NVivo version 12 (QSR International Pty Ltd, Melbourne, Australia). Initial coding frameworks were developed by five researchers (CW, MH, LN, JP and VMD). These were developed and agreed with the wider research team as well as the patient and public forum, using a variety of transcripts. Thematic analysis was conducted.76

Routine data and reports

The managers and clinicians were invited to share any relevant organisational documents or management data for all of the services employing PAs that could assist in answering the research questions concerning deployment, patient outcomes and organisation and cost. Examples were suggested, including patient throughput and outcome data, adverse events/serious untoward incidents, patient feedback, audit reports and expenditure on medical locums. The intention, if data were available, was to compare data before and after PAs were employed in a particular service. These data were to be descriptively analysed and, if appropriate, subjected to tests of significance to explore differences in time periods or between providers. It should be noted that no data of this type were offered from any of the cases study sites, a finding that is discussed in more detail in Chapters 4, 8 and 11.

Work activity diaries and observation of physician associates at work

The PAs were invited to complete work activity diaries, better described as logs. These were adapted from the primary care study17 in discussion with PAs on the research team and also with the advisory group (see Appendix 6). PAs were requested to complete these for 7-day periods up to three times, and for periods likely to demonstrate some differences in their activities (e.g. rotational duties of PAs or rotation times for junior doctors). These work activity diaries were designed to provide detailed information on the deployment, work setting and role of the PA. Data were entered into IBM SPSS Statistics Version 23 and Microsoft Excel^® (Microsoft Corporation, Redmond, WA, USA) and were analysed descriptively for summary characteristics and distribution, central tendency, dispersion and measures of the spread of individual variables.

The PAs were invited to volunteer to be observed by a researcher while they were working. This element of the study drew on the ethnographic tradition used in many health service research studies in the UK.77 For any PA volunteering, permission was also sought from the lead consultant for that service. The PAs sought assent for the researcher’s presence from patients they were attending, in the same way as permission is sought for students to be present during clinical care and treatment (see Appendix 7). Observations of any intimate activities with patients or in which the patient was, or was likely to be, distressed were excluded. PAs were observed for up to three sessions; this was intended to capture diversity in their work or the work environment. These sessions of observations were for all or part of their shift hours, including shifts not in the 09.00–17.00 period and shifts at weekends. Field notes of observations of the PAs’ activities and interactions were made at the time and written up in full later. The analysis method drew on that used in other, similar studies of health professional work in the UK.78 Rather than trying to impose a coding structure on what were usually fast-moving, dynamic events, the notes were read and particular ethnographic vignettes were identified from each. These were vignettes judged by the observer and one other researcher as being most relevant to the research questions, the theoretical framing of the study or the themes derived from the interview and documentary data. The reading and judgement of the vignettes was conducted with a lens that looked for both confirmatory and also disconfirming evidence of that obtained through sources such as interviews.

Pragmatic retrospective assessment comparing the outcomes and service costs for patients attended by physician associates and Foundation Year 2 doctors in the emergency department

The ED is a setting where PAs are employed and may work in different physical sections [minors, also known as the minor injuries, and majors (i.e. serious illness or trauma79)] alongside, and substituting for, junior doctors.1,80 Patients, triaged by a clinical professional to these sections of the ED, are assigned and initially assessed by a clinical professional (a PA, a doctor, or, in the case of the minor injuries section, another professional, for example an emergency care practitioner). Consequently, a pragmatic comparison of the patient outcomes and service costs of consultations by PAs and junior doctors (FY2) was undertaken based on the methods and data from two English studies, which compared minor injuries section consultations by nurse practitioners, as the mid-level professional (in the absence of UK data for PAs), with those of doctors.81,82

Revisions to the protocol

Following the surveys and preliminary recruitment of case study hospitals in 2015 and 2016, it was found that additional ED sites (from the four initially proposed) were required to undertake this element. Although the initial proposal had only included a comparison in the minor injuries section, it became apparent in trying to recruit the hospitals and PAs in January 2017 that PAs were rarely working in the ED at all as they were more frequently assigned to the acute medical assessment units. When they were assigned to the ED, it was usually to the majors section. In addition, practical constraints within trusts made the proposed prospective data collection impractical. Hence, a decision was taken and agreed with NIHR to change to a retrospective analysis of anonymised patient records from hospital databases. This had a concomitant effect in that the proposed linked patient satisfaction surveys could not be conducted. The study protocol was amended twice in pragmatic responses to changing NHS environments.

The pragmatic retrospective record review

In consultation with lead clinicians in EDs, a time period of 16 weeks (covering a national FY2 doctor rotation) in 2016 was designated for data collection. This period was sufficient to ensure that the required sample size of records for patients seen by PAs could be obtained (see Sample size). Each patient case attended by a PA or FY2 doctor in that time period was identified by information management staff and assigned a pseudonymised code number, with the link to the patient-identifiable record kept by the trust. The sample for the primary outcome was identified from the identification (ID) numbers using random sample numbers generated in Stata^® version 14.6 (StataCorp LP, College Station, TX, USA) and stratified to ensure equal numbers of cases by type of professional (PA or FY2 doctor) attending the patient and coverage across the period of the rotation. Data items that were requested for the sample included demographics, triage scores, treatment and investigations, prescriptions, diagnoses and outcomes (destinations), time in the ED and reconsultations within seven days. Postcode, to be used to calculate the Index of Multiple Deprivation score as a proxy for socioeconomic status, was requested but information governance review in trusts declined this request as having the potential to make individuals identifiable. Anonymised records were then passed to the research team in a Microsoft Excel spreadsheet and entered onto Stata version 14.6.

For a subsample of 10% of the above sample for the primary outcome, the patient’s full (anonymised) electronic or paper clinical record was supplied by the trust. Each record was assessed by four clinicians in emergency medicine [two emergency medicine registrars (in the final year of their specialty training), one PA (with > 20 years’ experience in emergency medicine) and one emergency medicine medical consultant]. Blinded to each other’s assessments and to the type of professional who was undertaking the patient consultation, they made a judgement as to the clinical adequacy of care. A pro forma (see Appendix 8) was used to guide and record the judgement. The pro forma employed the criteria used in the study by Sakr et al.,81 which compared consultations between junior doctors and nurse practitioners. When disagreement in judgments was found, the decision of the emergency medicine consultant was used.

Outcome measures

This was a pragmatic study designed to provide information for clinicians, the public, service managers and commissioners on a wide range of measures of quality, satisfaction, impact and cost. The primary outcome was specified in order to assist in calculating a sample size and was a proxy for patient safety and clinical effectiveness, but the goal of this element of the study was to synthesise evidence emerging from all the outcomes, along with the other workstreams, and we therefore used the sample calculation as a guide to avoid futility at either end of the scale.

Primary outcome

The primary outcome was unplanned re-attendance at the same ED within 7 days for the same condition. Unplanned re-attendance by patients, seen in any section, within 7 days for the same condition at the same department is one of the NHS clinical quality indicators for accident and EDs in England.83

Secondary outcomes

These secondary outcome measures were derived from two studies investigating the substitution of doctors by nurse practitioners in the minors section of the ED.81,82

• Measures of consultation processes (taken from the clinical record), including length of consultation (time stamp), use of diagnostic tests, immediate outcomes of the consultation (including prescriptions, treatments, referrals and follow-ups) and associated costs.
• The clinical adequacy of care. This was an expert clinical judgement. Criteria included record of medical history, examination of patient, requests for diagnostic tests, diagnosis, treatment decisions, referrals and planned follow-up provided by the PA or FY2 doctor at the initial consultation.

Sample size

Anticipated rate of unplanned re-consultation

In the absence of UK data on PAs in the ED setting, we considered three sources of information, including two randomised controlled trials substituting nurse practitioners for doctors in the minors section of the ED.81,82 First, NHS ED clinical quality indicator data show that unplanned re-attendance at the same ED in England within 7 days for patients seen in any section and by any professional was 7.4%, with individual EDs ranging from 2.4% to 21.7% in December 2014.84 Second, Sakr et al.81 reported unplanned reconsultation rates, for the same condition within 28 days, of 8.6% (nurse practitioners) and 13.1% (doctors). Third, Cooper et al.82 reported rates of 18.3% and 21.5%, respectively. Clearly, the 7-day data at the same ED site will underestimate a 28-day outcome at any ED, urgent care or general practice site. Both studies were conducted in single ED sites and we did not know whether they were high or low on the spectrum seen in the clinical quality indicator data. Taking this into account, we chose Cooper et al.’s82 18.3% as an anticipated base rate towards the higher end of the observed range.

Minimum clinically important difference

We took the study by Sakr et al.81 as having the least risk of bias from the comparison and study design. Their sample size calculation was to detect a difference between groups of 2.5% versus 5%. Although this proved to be an underestimate, we were also interested in finding a relative difference of 50%, although in a non-inferiority hypothesis, so we compared 18.3% with 27.4%. It should be noted that as this is a pragmatic study investigating a wide range of measures of impact and cost, we considered that estimating with confidence intervals (CIs)85 and understanding the differences between professions would be more informative for clinicians, service managers and commissioners than a binary hypothesis test on one of the outcomes. We anticipated that 18.3% of patients would reconsult (unplanned) within 28 days for the same problem in any health service. We aimed to test a non-inferiority hypothesis on this primary outcome measure, which stated that PAs do not exceed 27.4% unplanned reconsultations, with 80% power at a 5% significance level. This required 284 patients in each group (calculation from Stata version 11.1 software). We included an extra 20 patients to allow for adjustment for case mix,86 requiring a total of 304 patients in each group (those seen by PAs vs. those seen by FY2 doctors).

Analysis

Data were entered into a patient-level Stata version 14.2 file. Analysis and presentation of these data involved summary measures as cross-tabulation (frequency/proportions) for categorical variables, and location (e.g. means/medians) and dispersion (standard deviations/percentiles) appropriate for continuous variables. The distribution of length of stay in the ED was inspected using a histogram plot. Descriptions of difference at the aggregate level were made between those patients seen by PAs and those seen by FY2 doctors for all patient outcomes. A likelihood ratio (LR) χ² test was used to assess whether or not the primary and secondary outcomes differed between PAs and FY2 doctors. A logistic regression86 was also carried out for the primary outcome (i.e. the patient making an unplanned reconsultation at the ED within 7 days), while adjusting for confounding factors, such as age, sex and Manchester Triage Score (MTS)87 the latter as a proxy measure of patient acuity. Although the advice of the lead clinicians in EDs and the advisory group was that patient acuity, as recorded through an early warning score,88 was the most appropriate method of classifying differences in patient caseload in the ED, one of the trusts did not supply this variable and missing data, therefore, precluded using that as the main adjustment variable. Because the difference between PAs and FY2 doctors may be different at different sites, an interaction term between profession and site was also tested. The odds ratios (ORs) estimated from the logistic regression were reported and their significance levels were assessed using Wald tests. All p-values are two-sided.

Economic analysis

An analysis of the comparative costs of using PAs and FY2 doctors in the ED was planned from a NHS perspective. The a priori analysis plan was for variables of interest to be extracted from the anonymous patient records, including consultation length (time in and time out in minutes); diagnostic tests ordered (blood, X-rays), treatments (e.g. prescriptions), referrals made and follow-on care recommendations; whether the PA or FY2 doctor sought advice from a senior colleague in the ED during the consultation; and unplanned reconsultations (primary outcome). The protocol stated that if statistically significant differences were observed between the two professional groups in any of these variables, after controlling for case complexity, costs would be attributed. The cost of consultation times would be based pro rata on salaries, with oncosts and overheads, obtained from professional bodies and national sources.89 The costs of tests and treatments would be obtained from NHS sources90 and local financial managers as needed. It was planned to use a cost–consequences framework91 to indicate differences in costs to the hospital of employing PAs and FY2 doctors in the ED, in relation to appropriateness outcomes, obtained from the record review. However, data limitations restricted the analysis to a comparison of a reduced number of variables and the employment costs of the professionals.

Ethics and governance

The research was undertaken using ethical principles derived from the Medical Research Council92 guidance. Professional and patient participants were volunteers and individual consent to participation was sought. Assurance was given to participating trusts and individuals that they would not be identified in any reports or papers that were published. Data storage and disposal complied with the Data Protection Act 1998.93

Workstream 3 was reviewed and approved (including amendments) by NHS London – Central Research Ethics Committee (reference 15/LO/1339). For the first case study sites, NHS research governance permission was obtained via the central single process application to the NHS trusts (Integrated Research Application System project ID 181193). The replacement of this system and subsequent delays to all research projects in the transfer are well documented. This study was no exception. Each study site had to then give formal written permission to proceed. The shortest time period from application for local research governance permission was 3 months and the longest was 10 months.

Workstream 4: synthesis of evidence

The evidence from workstreams 1, 2 and 3 was developed into an overall synthesis against the research questions by the research team. This was then discussed in an emerging-findings seminar (see Appendix 9). Participants included those who had advised the research team (patients, members of the public, health professionals and non-clinical health service managers) as well as some of those who had participated. The workshop was organised into presentations followed by round-table discussions in small groups. The presentations were then discussed as a whole group, led by the patient and public representative on the research team. A member of the research team took contemporaneous notes, which were then typed up and circulated to participants. The views, questions and issues raised in the seminar were then incorporated into the final report.