Parts of this chapter have been reproduced from Butler et al.15 © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2018. All rights reserved. No commercial use is permitted unless otherwise expressly granted. This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: https://creativecommons.org/licenses/by/4.0/. The text below includes minor additions and formatting changes to the original text.
Hospice at home is a complex intervention operating within the wider system of health and social care delivery. Hence, research methods were required that could capture the complexity of the intervention and the impact of the implementation of different HAH models on the organisation, delivery and experience of EOLC from the perspective of service users (patients) and their family carers, and service providers and commissioners.
The research design was informed by realist evaluation,17 a theory-driven methodology increasingly used to evaluate complex interventions,71 including services for EOLC.37 Realism provides the philosophical foundation for realistic evaluation. At the core of realism is the notion of ‘generative mechanisms’. A generative mechanism is a causal link, the ‘black box’ that leads from A to B and creates an ‘effect’.72 Realist evaluation attempts to theorise what the mechanisms are, even though they are not necessarily ‘measurable’ in an empirical sense, and it seeks to find evidence of their existence. The relationships between mechanisms, the contexts in which they are operating and the effects they produce are represented through propositions that take on a basic formula: context + mechanism = outcome. Thus, the aim of empirical research is to identify patterns to support an explanatory theory about what mechanisms are working (or not) in a given situation.73 A pluralist approach to data collection suits a realist evaluation.
Realist evaluation analysis in mixed-methods research
Realist evaluation analysis aims to understand both what is happening and how it is happening in an intervention. Understanding how contextual factors influence health interventions, such as HAH, is central to this methodology. It is acknowledged that an intervention and its outcome are dependent on contextual factors, and understanding how, why, for whom and when an intervention works16 is core to the approach. The context–mechanism–outcome (CMO) configuration is central to the analysis, which is intended to be pragmatic in that findings can be transferable across settings.74 The idea that it is people and their responses to interventions that create change is also important.
In realist evaluation, data analysis takes a ‘retroductive’ approach.75 This means that those factors that lie behind observed patterns are identified with the aim of understanding causation. Retroduction is the idea that we can explore the underlying social and psychological drivers that influence intervention outcomes by looking behind observable patterns to understand what produces them. Multiple data sources are typically required for realist evaluation.75 Both quantitative and qualitative data can be used to generate evidence to support and refine the CMO configurations.
Overall design and development of context–mechanism–outcome configurations
The OPEL HAH study employed a mixed-methods design, using realist evaluation methodology and incorporating an economic analysis. The design of the study comprised three phases, alongside which programme theories and CMO configurations were developed (Figure 2).
FIGURE 2 Study design, analysis and CMO configuration development. AHCR, Ambulatory and Home Care Record; AKPS, Australia-modified Karnofsky Performance Status; ECOG, Eastern Cooperative Oncology Group; IPOS, Integrated Palliative care Outcome Scale; MRT, middle-range theory; NPT, normalisation process theory; QODD, Quality of Dying and Death questionnaire; VOICES, Views of Informal Carers – Evaluation of Services. (Figure not reproduced here; see the original article.)
The first stage of a realist evaluation is to develop initial programme theories. These were elicited through a variety of sources.76 First, the NAHH standards21 were taken as indicative of an overall programme theory (how HAH should work). Second, an initial review of the literature (undertaken during proposal development and subsequently updated) was synthesised through a realist logic of analysis.18,77 Finally, as a theory-driven methodology, concepts from abstract theories were sought to inform the initial programme theories. Normalisation process theory (NPT)78 was identified as a suitable candidate theory. NPT is increasingly being used in combination with realist evaluation to understand what needs to be in place for the implementation of complex interventions,79 and it was anticipated that it would aid understanding of how HAH may become embedded as part of a local health-care economy. Data collection then occurred, based on these initial theories. CMO configurations emerging from the programme theories were identified and coded within the qualitative data.80 During analysis, CMO configurations were synthesised in an iterative process to refine and evolve the understanding. Quantitative data were analysed in tandem, with qualitative data reinterrogated to seek plausible explanations of quantitative findings.
Stakeholder involvement is integral to the whole process and is a key feature of realist studies; by engaging lay or content experts, evidence is built to support theories on the basis of coherence and plausibility.81 Stakeholder involvement was operationalised through PPI activities (see Chapter 4); 6-monthly meetings with the Project Oversight Group, which included lay and content experts; and two national consensus workshops.
Patterns within the data were used to refine and justify the emerging theory. The resulting CMO configurations describe common patterns (‘demi-regularities’)75 that can be applied to different settings and, in particular, the generative mechanisms at work.
Detailed design of each phase
Phase 1: national telephone survey
Hospice at home services serving adult palliative care patients in England were surveyed.
The survey aims were to (1) develop an understanding of the range of services and operations and (2) identify categories (types) of services from the survey information to use as a sampling framework for recruiting case study services in phase 2 of the study.
A total of 128 HAH services in England and the appropriate contact (e.g. service lead) were identified from the NAHH and Hospice UK directories of services and approached to take part in a telephone survey. Each service contact was posted an information letter, a survey and opt-out slip. An interview to collect the data over the telephone was proposed. Contacts were followed up 2 weeks later to arrange the interview if they had not already responded or opted out.
Telephone survey calls were conducted by a nurse with palliative care experience to facilitate understanding of the services. The survey was semistructured, comprising a selection of closed and open questions. Respondents were asked to provide details of the population characteristics in the catchment area; other relevant local services and access to palliative care beds; and HAH activity levels, staffing, facilities, equipment, processes, budget, and barriers to and facilitators of operating (see Report Supplementary Material 1 for the full questionnaire). HAH services were asked to provide supporting documents and to indicate willingness (or not) to consider becoming a case study site in phase 2.
Phase 2: case studies
Case study methods are a well-established approach to conducting research in ‘real-life’ health-care settings.82 The approach employs mixed methods to gain an in-depth understanding of the impact of service models, resource implications and the experiences of all stakeholders, including service users, providers and commissioners. Yin83 describes case study design as orientated towards a realist perspective The design allows methodological flexibility to generate theoretical insights from the findings,84 which is a key requirement for realist evaluative design.17 We adopted Yin’s83 approach to defining a case as an individual organisation. Although each case needed to be bounded, there was also some need to maintain flexibility, and each case was defined as what the site described as their HAH service.
Plan of investigation for case study sites
When HAH services had agreed to take part, full training on the study and the informed consent process was provided to site staff (research nurses, clinical staff, managers, etc.) by the members of the research team. The training was delivered in person at site initiation visits and follow-up training was also provided on site and remotely, as needed.
Recruitment and informed consent
Patient and informal carer dyads
Participants at each site were invited to take part in the research at the time they were admitted to the HAH services. A patient-directed flyer was made available at HAH sites to raise awareness about the study (see Report Supplementary Material 2).
The inclusion criteria were as follows:
- patient admitted to HAH service
- patient had a lay informal/family carer who also agreed to take part in the study (defined as someone who provided care and support at home on a daily basis)
- ability to obtain informed consent from patient and carer.
The exclusion criteria were as follows:
- patients without a suitable informal carer
- inability to obtain consent from the participants
- participants unable to complete questionnaires in English
- patients in a care home at the time of admission to the HAH service.
Hospice at home service staff introduced the study to the patient and their carer. Information sheets (see Report Supplementary Material 3) were given to the participants and they were allowed time to read the information and ask any questions; if needed, the information sheet was read out.
Patients were asked to consent to taking part in data collection at one time point (on admission to the study or as soon as possible thereafter). They were also asked to agree to the collection of information on their use of health and social care services from 2 weeks prior to joining the study until death. The carers were asked for their consent to be contacted to complete a post-bereavement questionnaire, and informed that the option of taking part in an in-depth interview post bereavement would also be offered.
Service staff took consent from the patient and carer, using the study consent forms (see Report Supplementary Material 4), and both were given a copy of their information sheets and consent forms. Copies of the consent forms were filed in the study site file; a copy of the patient consent form was also filed in the patient’s medical notes. The carer was asked to provide contact details and to indicate the best time of day for the research team to call to collect data.
Patients lacking capacity
Owing to the nature of the patient population, some of the potential participants lacked capacity and were unable to provide informed consent. For this reason, a variable consenting process, involving consultee assent, was used. The local HAH team made the decision to proceed using one of the following options:
- If the patient was deemed to have capacity by the local team, then consent was sought from the patient in the normal manner.
- If the patient was deemed not to have capacity, then a personal consultee (i.e. someone who has a role in caring for the person who lacks capacity or is interested in that person’s welfare but is not doing so for remuneration or acting in a professional capacity) was approached for advice regarding the patient entering the study. The personal consultee could be a relative or friend of the person, in practice often the informal carer.15
- If the main carer or personal consultee was not available, a nominated consultee was approached for advice regarding the patient entering the study. The nominated consultee was a clinically qualified member of the patient’s care team who was not involved in patient consent or in study procedures such as data collection.
When a personal or nominated consultee was used, they were given an information sheet (see Report Supplementary Material 3) about being a consultee and the patient information sheet. They were given time to read the information and the opportunity to ask questions about the study and asked if, in their opinion, the patient would object to taking part in the study. The local staff member then gained a declaration from the consultee, using the study consultee declaration form (see Report Supplementary Material 4), as to whether or not they agreed that the patient would be willing to participate in the study.
Service providers and commissioners
The managers of HAH services identified a range of staff from their organisations for interview, to give a detailed picture of each organisation and its operations. These included clinical staff, the HAH service manager, charity trustees, fundraising staff and volunteers. Relevant commissioners from the local area were identified by HAH service providers for interview . Potential participants were invited by e-mail or by telephone by the research team and an information sheet (see Report Supplementary Material 3) and consent form (see Report Supplementary Material 4) were sent by e-mail or post. If they agreed, interviews were arranged (either by telephone or in person) at a time and location convenient for the interviewee. Prior to the interview, the participant was asked to complete a consent form.
Data collection
Baseline patient data
After consent, a member of the participant’s clinical care team assessed the patient using the Integrated Palliative care Outcome Scale (IPOS) (staff version),85,86 the Phase of Illness87,88 and the Australian modified Karnofsky Performance Status.87,89 These instruments are recommended measures reflecting the key domains of palliative care, and have been validated for use in research. Patients were also asked if they had a PPOD (i.e. home, hospice, hospital).
Service use data
After consent, a member of the research team contacted the carer by telephone as soon as possible to collect health service use data retrospectively for the patient for the 2 weeks prior to recruitment. Data were collected using the Ambulatory and Home Care Record90 (AHCR), an instrument designed for capturing the use of health, social and voluntary services and informal caring for palliative care patients based at home. Items relate to services received both inside and outside the home [e.g. hospital appointments, accident and emergency (A&E) visits, inpatient stays]. The AHCR was customised for use in this study following piloting with a hospice population in England.91 Carers were sent an optional ‘home care diary’ to assist with tracking service use.
Completion of each AHCR included three additional questions: satisfaction with services (scaled as follows: 1, exceeded expectations; 2, just met expectations; 3, fell short of expectations), carer burden [Short Form Zarit Burden Interview: six items relating to stress, strain, relationships, health, control and time for self, each scored on a 5-point scale, leading to a total score ranging from 0 (best) to 24 (worst)]92 and the patient’s health status [Eastern Cooperative Oncology Group (ECOG) scale, range: 0 (fully active) to 4 (completely disabled)].93 The last satisfaction and carer burden responses prior to death were used as measures of the performance of services.
After the collection of retrospective service use data, contact was made with the carer by telephone every 2 weeks to request completion of another AHCR covering the intervening 2-week period. In this way, a continuous record of services used by the patient could be collected. Researchers were assigned to data collection in specific case study sites to enable continuity with carers. Each telephone call lasted approximately 15 minutes.
Post-bereavement data collection from carers
Post bereavement, a follow-up letter was sent to carers to remind them that the research team would be in touch to collect further data. This letter included information sheets about the questionnaire and about the optional in-depth interview. Participants were given a choice about how to complete the questionnaire: by telephone, using an online survey tool or by post. The original protocol stated that the invitation to complete the Quality of Dying and Death questionnaire (QODD) would be offered either when hospice bereavement services made contact with the carer (usually around 6 weeks post death) or at 4 months [replicating the design of the Views of Informal Carers – Evaluation of Services (VOICES) survey94]. The researchers, who were already in contact with the participants to collect AHCR data, found that some participants expressed that they would prefer to compete the QODD earlier; a study amendment was approved to enable this.
The questionnaire contained the primary outcome measure, namely the QODD (English, 7-day recall, version 1), a validated 30-item instrument95–97 (note that we removed a question on euthanasia not relevant for use in the UK). Two short questions about the overall care received were also included in this questionnaire. The first asked if the carer and family had received as much support as they needed when caring for the patient (five-point scale, from ‘as much as needed’ to ‘no help at all’); the second was a rating of the quality of care received (five-point scale, from ‘outstanding’ to ‘poor’). These questions were taken from the VOICES questionnaire, a national survey of bereaved people conducted by the Office for National Statistics and commissioned by NHS England, based on research by Addington-Hall and McCarthy.94
Three attempts were made to contact carers by telephone; if these were not successful, a paper copy of the QODD and VOICES questions was posted to the carer for completion (on one occasion only). This was accompanied by a cover letter to explain that the research team had been unable to contact them and if they would prefer to self-complete the questionnaire at home they could do so. A stamped addressed envelope was provided for return of the questionnaire.
Optional interview, bereaved carers
An in-depth interview was completed by a subset of participants; we aimed for up to 20 interviews per service model type, with a stopping criterion of three interviews if no new themes were coded, to achieve data saturation.98 If the QODD was completed by telephone, the researcher asked the participant if they would be willing to participate in an optional in-depth interview by telephone or in person to understand more about the HAH service received. If the QODD was completed in the postal or online formats, carers could indicate at the end of the questionnaire if they would be happy to take part in an optional interview. If the postal or online QODD was not completed within 1–2 months, a final follow-up letter was sent to invite carers to take part in the optional interview only. Interviews were semistructured, following a topic guide (see Report Supplementary Material 5), and explored the experience of the HAH service and the EOLC the patient received.
Interviews with service providers and commissioners
The research team conducted interviews with 5–10 managers, health-care staff and commissioners per case study site. Interview schedules were designed for both staff and commissioners (see Report Supplementary Material 5) and were semistructured; they included questions to explore the service history, logic, rationale, funding, processes and contextual features facilitating or inhibiting service delivery, as well as enablers of and barriers to providing HAH services.
Withdrawal criteria
Participants were free to withdraw from the study at any time. Patients and carers were made aware in the information sheet that withdrawal would not affect the care they would receive. If a participant withdrew from the study, they were asked, if possible, if the data collected to date may still be used in the final analysis. If they did not wish for their data to be used in this way, all data collected from the participant were destroyed. If it was not possible to consult the participant on this, data collected up to the point of withdrawal were utilised according to the original consent.
Distress
A distress protocol was designed and made available to all case study sites (see Report Supplementary Material 6).
Phase 3: stakeholder consensus
Two national consensus workshops, with up to 60 participants each, were held. To maximise potential attendance from stakeholders, one workshop was held in London and one in Leeds. Each event took place over 1 whole day at a conference venue, facilitated by the project research team, including PPI members. HAH services that had participated in the phase 1 survey were offered a £50 bursary to support attendance at a workshop. Additional invitees were identified through the NAHH, study co-applicants’ networks and the Project Steering Group. Other organisations also advertised the events: Clinical Research Networks, Applied Research Collaborations, Clinical Commissioning Groups (CCGs), Healthwatch and other national charities and groups (e.g. Marie Curie). Stakeholders included service providers, commissioners, researchers, members of the public and service users. The purposes of the workshops were to fine-tune the CMO configurations developed in phase 2 of the study and to provide a more nuanced understanding of the features of HAH models that work, for whom and under what circumstances.
Emerging findings and relationships between contexts, mechanisms and outcomes were presented to stakeholders using a mix of formats and approaches, including lecture-style, small-group work and poster presentations (see Report Supplementary Material 7). Consensus workshop methods were used99 to facilitate discussion. Consensus event delegates also contributed to planning the methods of communicating the study findings, in particular advising on the presentation of information relevant and accessible to the public, service providers and commissioners of HAH services. After the events, participants were sent a workshop report.
Analysis
Phase 1
A descriptive analysis of survey responses was undertaken using Statistical Product and Service Solutions (SPSS) software, version 15 (SPSS Inc., Chicago, IL, USA), to gain an understanding of the range of HAH services. The analysis also sought to enable the identification of types of HAH service models for phase 2. Findings were presented in tables. Categorical variables [e.g. urban/rural setting, presence of hospice building(s), yes/no] were cross-tabulated with each other to identify underlying associations. Associations were explored between all variables. These results were used to identify any natural groupings of service features that could be defined as service models or types. Based on prior survey work from the NAHH21/Hospice UK (Heather Richardson and Andrew Thomson, personal communication), it was projected that approximately four high-level types of the model would be distinguished.
Qualitative field notes collected during the survey data collection were typed up and analysed inductively to identify relevant factors for the development of the typology. These were combined with the output from quantitative analysis and discussed at a meeting of the full team, resulting in agreement on a typology that could provide a framework for the recruitment of services as case studies for phase 2 (Figure 3).
FIGURE 3 Service model typology. a, A 24-hour service provision is defined as two or all of the following: 24-hour hands-on care, 24-hour symptom assessment and management or fast response time (< 4 hours); b, provider size is based on having either more or fewer than 365 referrals per annum (one per day). (Figure not reproduced here; see the original article.)
Based on survey responses, hospices that had indicated willingness to consider becoming a case study site were then approached to represent the identified service types as case study sites. We purposively sought diversity in geographical spread, socioeconomic profile, staffing mix and funding sources. Services were approached initially by e-mail; further queries and negotiation were managed by e-mail and telephone follow-up.
Phase 2
Sample size
The total score for the primary outcome measure, the QODD, ranges from 0 to 100 (a higher score indicates better quality). Hales et al.100 identified scores of 30 and 70 as cut-off points for distinguishing terrible/poor (< 30), intermediate (30–70) and good/almost perfect (> 70) quality of death.15 Hence, on the basis of a difference of 10 representing a meaningful change, and using a standard deviation (SD) of 16.41,101 at least 44 participants in each model would be required for comparisons between any pair of models. To allow for a participant non-completion rate of 33%, a sample size of 66 patients per model type (up to four models) was proposed. The non-completion rate was based on the 55.4% response rate obtained when the 24-item intensive care unit QODD was mailed to carers 4–6 months post death;102 a higher response rate was expected in this study because telephone interviews were being used.
Based on estimated HAH service size and annual throughput of patients, it was anticipated that recruitment of 66 patients per model type was achievable, for medium and large units in particular. The national minimum data set 2013/14 by the National Council for Palliative Care103 grouped HAH services by size into roughly three equal groups: small, < 191 patients per annum; medium, 191–310 patients per annum; and large, > 310 patients per annum. If small sites were recruited, it was agreed by the steering group to recruit two case study sites of the same model type to reach an overall sample size of 66. In the final regression modelling process (outlined below), a dummy variable would be used to distinguish between providers if comparisons were necessary.
Statistical analysis
Availability of data from the various sources [background/baseline, AHCRs, date of death (DOD) from hospice services or carer, post-bereavement interview for QODD and VOICES] was examined and patient–carer dyads were broken down into four categories according to the data they provided (with categories 3 and 4 merged for analysis purposes because no post-bereavement interview was possible):
- date of patient death known and occurred before the end of the study period
- patient/carer withdrew from the study before the end of the study period; data were available for analysis up until withdrawal unless the participant requested otherwise
- patient still alive at the end of the study period
- patient died at unknown date after the end of the study period.
The baseline sociodemographic characteristics of patients and carers, and the clinical status of patients in the different service models, were summarised using relevant descriptive statistics (proportions, medians, ranges, means, SDs, 95% confidence intervals, etc.) before being compared on the basis of each patient sociodemographic, clinical and carer feature using the appropriate bivariate test [including one-way analysis of variance (ANOVA), chi-squared tests and Kruskal–Wallis tests, depending on the nature of the variable]. The time between recruitment to the study and death was calculated and compared across models.
A minimum of six responses [from the total of 30 items (20%)] were required for the calculation of the total QODD score (as 10 × mean of all non-missing responses) from each carer. Several questions were not always applicable (e.g. spend time with pets), so a response rate of 100% was unlikely in most instances. The frequency of responses and summary statistics for total QODD score, and for the dichotomised form of the total QODD score (≤ 70 vs. > 70, the latter indicating a good/almost perfect death), were calculated, as recommended by Hales et al.100
The secondary outcomes VOICES 1 (five-point scale relating to the sufficiency of the help and support from health and social care services that had been received) and VOICES 2 (a five-point quality rating of that help and support) were presented as frequency tables, given their ordinal nature. Achievement of PPOD was calculated for each patient for whom both the preferred and actual places of death were known; these values were presented by place of death as numbers and percentages. Means and SDs were calculated for the final secondary outcomes derived from the AHCRs: carer burden in last 28 days and service satisfaction in last 28 days. For each AHCR, the six-item carer burden (Zarit Burden Interview) total score92 was calculated [sum of six responses, 0 = never to 4 = nearly always, range 0 (best) to 24 (maximum worst burden)]. The mean of non-missing responses was assigned to missing items. For each AHCR, the service satisfaction was coded as ‘exceeded expectations’ = 1 (best outcome), ‘just met expectations’ = 2 and ‘fell short of expectations’ = 3 (worst outcome). A ‘last 28 days’ carer burden or service satisfaction score was included only if the final AHCR was conducted no earlier than 28 days before the patient died; any patient alive at the end of the study period, or with no known DOD, was therefore counted as missing.
All outcomes were compared between models using appropriate statistical tests. Because three of the four models included multiple HAH services, summary statistics were also generated at the HAH service level. Bivariate associations were explored between the QODD primary outcome score (total QODD score) and a set of covariates that were agreed as important by the research team. These included sociodemographic and clinical characteristics of the patient at baseline; place of death; how long before death the patient was aware that they were dying (from the QODD); number of days the carer had seen the patient in the last 7 days before death (from the QODD); how long the patient was in the study, and hence receiving hospice services (between recruitment and death); and time between death and when the carer completed the QODD. Appropriate statistical tests were used, depending on the nature of the variables.
Linear regression was used to model total QODD score, VOICES 1 and 2, carer burden in the last 28 days and service satisfaction in the last 28 days; logistic regression modelling was used for the dichotomised total QODD score and achieved PPOD. All outcomes were modelled (using forward stepwise selection) against predictors agreed on as important by the research team, with results including 95% confidence intervals for fitted parameters and goodness-of-fit statistics for the overall model. Service model was always included as a predictor, such that the fitted parameters in the final models indicate if service type is associated with differences in QODD scores. The characteristics of service types that result in better QODD outcomes were identified from descriptive data collected at each site as part of the realist evaluation.
Economic analysis
The economic analysis was planned at two levels. First, a descriptive analysis of the resources and costs of running each case study HAH service, covering staff; service facilities, inpatient beds, equipment, overheads; transport for home care; and other sundry items associated with care delivery. These data were requested during the interviews with service managers, together with information on activity rates and financing, so that costs per patient receiving the HAH service could be calculated and compared between case studies.
Second, a patient-level analysis was undertaken. Owing to the nature of this study, patients recruited were likely to have short and variable life expectancy, leading to an inconsistent time horizon for the individual patient-level data captured. This lack of a normalised time-integrated measure of health outcome (such as a quality-adjusted life-year) or cost, makes a traditional comparative cost-effectiveness analysis problematic. Hence, the economic analysis was limited to a descriptive analysis of service use and cost for the different HAH models. Whole-system resource use (provided by the hospice; local NHS primary, community and hospital services; and the voluntary sector) in EOLC was captured prospectively from the point of recruitment to the study for each patient. A customised version of the AHCR,91 which included informal care, was used for this purpose, as described in Service use data. At their first interview, participants were asked to report retrospectively, via recall, service use for the 2 weeks prior to recruitment.
Service use data, once captured, were grouped into time periods of approximately equal sample sizes, delimited by survival time following the start of the service use data collection. The cut-off points were determined by the distribution of the data. The methodology explaining how service use data were allocated to time periods, and how missing AHCR data were dealt with, is given in Appendix 2. Resource use was converted to costs [in 2019 Great British pounds (GBP)] using national tariffs.104 Informal care was valued using replacement cost methods (see Appendix 3).
For each of the models of HAH service provision, an average cost per day of treatment was estimated for each time period. This provided descriptive cost data, independent of expected survival time, that can be compared between HAH models. Costs are presented as means and medians, given the typical skew in the distribution of costs. Comparisons of costs between each pair of HAH models were performed using the Mann–Whitney U-test and are presented as box plots showing medians and interquartile ranges (IQRs). Analysis was also broken down by individual HAH services within models to illustrate variability. Costs were considered in relation to outcomes from different models in a cost–consequences framework.
Qualitative data analysis of phases 2 and 3
A four-stage framework approach98,105 was undertaken, using retroductive analysis of the data. Retroduction demands counterfactual thinking based on knowledge and experience, analysing why expected phenomena anticipated in initial programme theories (such as volunteering) may or may not be present, and identifying what conditions are needed for them to be in place.106 Consequentially, qualitative data analysis throughout the project was characterised by monthly team meetings to discuss what the data were suggesting could be happening, regular sounding-out with lay and content expert stakeholders and testing out these hypotheses in subsequent batches of data.
Stage 1, familiarisation, involved the research team reading the detailed written field notes taken during the telephone survey in phase 1, alongside phase 1 quantitative results suggestive of enablers of HAH services. In addition to monthly research team meetings, two discussion sessions with project stakeholders were conducted; from these, eight initial CMO configuration tables were developed (see Report Supplementary Material 8). These CMO configuration tables and the four core constructs of NPT78 (i.e. coherence, cognitive participation, collective action and reflexive monitoring) were used in stage 2, constructing an initial thematic framework. This framework was uploaded to qualitative data software (NVivo 12; QSR International, Warrington, UK). Interviews were transcribed verbatim and coded in NVivo. During monthly meetings, the research team reviewed a set of six to eight transcripts that were independently coded and then compared, discussed emerging themes and ensured a shared understanding of the coding framework. Stage 3, reviewing data extracts, was conducted during these regular meetings. Over 18 months, interviews from all participants and sites were reviewed by the research team. This involved organising data into more coherent groupings.98 CMO configuration tables were refined down to six after further stakeholder discussions and it was noted that coding to CMO configuration tables and to NPT core constructs was not adding to interpretation at this level, but that NPT was useful as a lens for the next stage. Stage 4, mapping and interpretation, involved further refinement of CMO configuration tables before presenting them at phase 3 consensus workshops. Feedback from small-group exercises and group discussions was recorded by facilitators (research team members, including lay co-applicants) and compiled into a detailed report (see Appendix 4, including Table 26, Boxes 9–17 and Figures 18–22). The report was mapped against the CMO configuration tables and further refinements were made. NPT was used as a higher level of interpretation and to check that implementation of HAH could be explained through the generative mechanisms identified. Quantitative questions were derived from qualitative data and used for quantitative analysis (see Appendix 5). One example of this was that carers’ qualitatively described the perception of staff having time to provide extensive hands-on care and develop a relationship with the family. Quantitative data were then tested for staff grade (it being hypothesised that lower bands of staff were more likely to be providing hands-on care), number of visits per day, duration of visits and the relationship with the QODD scores. Once quantitative data analysis was completed, suggested statistical associations were used as a guide to further interrogate the qualitative data for plausible generative mechanisms. For example, quantitative data suggested that smaller service models had more positive outcomes, and the qualitative data were analysed for possible explanations.
In total, 25 iterations of CMO configuration tables were developed. These were based on the qualitative data findings (what was working and why), literature (what, in theory, should be working but is not present, and why) and outcome patterns suggested by quantitative data.
Ethics
Patients approaching the end of their lives are entitled to evidence-based care as much as any other NHS service user, and research is required to develop high-quality palliative care. There are, however, important considerations and vulnerabilities to be taken into account when planning research in this context and safeguards need to be put in place. The family/informal carers of dying patients are also experiencing a key life event and their vulnerability must be taken into account in study design.107
Key areas of concern were as follows.
- Emotional distress exacerbated by the demands of the research. A number of strategies were put in place to deal with this concern:
- A leaflet was made available in all case study sites to advertise the research, so that potential participants might already be prepared to be approached when referred to HAH.
- The study required written consent from both patient (or consultee) and carer, supported by full information about the study procedures. There was also a strength in the patient–carer dyad recruitment – because study subjects (patients and their carers) were recruited together as a pair, they were mutually supportive of the research and processes.
- The majority of the data were collected by telephone, with support from researchers [there was a distress protocol for researchers to follow if a carer became distressed during data collection (see Report Supplementary Material 6)].
- Participants were made aware of their right to withdraw from the study at any time, without providing a reason or jeopardising their clinical care.
- Patients lacking capacity at recruitment or losing capacity as the study proceeded. For this reason, we had a variable consent process (see Patients lacking capacity) and we did not collect data directly from patients; instead, we used staff or carer proxy data collection methods.
- Gate-keeping by staff as a result of patient vulnerability. We offered training and support to all staff involved in recruitment to address this concern and additional training and support resources were deployed when staff voiced concerns about this issue or when recruitment did not proceed as expected.
Summary
This chapter summarises the methods used in this study, which used mixed methods with an overarching realist evaluation approach.108 The study was in three phases. Phase 1 was a survey of HAH services in England, to understand the range of contexts and operations and to develop a broad typology of services for further investigation. Literature reviewing, stakeholder insight and phase 1 of the study all contributed to the development of initial programme theories and candidate CMO configurations. Phase 2 involved quantitative, qualitative and health economics data collection from case study sites across England, to assess the impact of each model on patient and carer outcomes; to investigate the resource implications and costs of patient care in each model; to explore the experiences of patients, family carers, providers and commissioners of the different HAH models; and to identify the enablers of and barriers to embedding HAH models as part of service delivery. As the qualitative data were collected, the CMO configurations were iteratively tested and refined. Reciprocal reviews of CMO configurations and qualitative interview data were undertaken with quantitative and health economics data to synthesise the findings. In phase 3, through stakeholder consensus workshops, the data and explanatory CMO configurations were presented, further refined and validated.