Overview
This study was a mixed-methods evaluation of COVID-19 remote home monitoring services in England.
Phase 1 of the evaluation (rapid systematic review, empirical mixed-methods implementation study) aimed to analyse implementation and impact of COVID-19 remote home monitoring services, develop a conceptual map of these services, explore implementation and staff experiences and understand data use, staffing and resource allocation (see Chapter 1 and Vindrola-Padros et al.32 and Vindrola-Padros et al.36). We also shared findings in a timely way so these could inform the national roll-out of COVID-19 remote home monitoring services implemented throughout England.
Phase 2 aimed to explore effectiveness, cost, implementation, and staff and patient experiences of COVID-19 remote home monitoring services (see Chapter 1). It comprised four workstreams. Workstream 1 explored effectiveness of COVID-19 remote home monitoring services (see Chapters 5 and 6). Workstream 2 included an analysis of the composition of the workforce involved in providing COVID-19 remote home monitoring services and a cost analysis of these services (see Chapter 7). Workstreams 3 and 4 included a national survey and in-depth case studies of implementation, staff and patient experience, including a focus on health disparities and mode of data submission offered by services, whether technology-enabled and analogue or analogue-only (see Chapters 4 and 8–11).
Within this chapter, we provide a summary of key findings with a focus on our research questions (see Chapter 1). We then discuss the implications and lessons learned, the strengths and limitations of our evaluation, the potential impact of our findings, future research areas and conclusions.
Summary of key findings
Below, we describe key findings from phases 1 and 2 of our evaluation (Figure 26 for a summary).
FIGURE 26 Summary of key findings. (Figure not reproduced here; see the original article.)
Key findings from phase 1 (see Chapter 3)
Systematic review of COVID-19 remote home monitoring services32
- Remote home monitoring services have been implemented internationally for COVID-19, an acute condition, during the pandemic.
- Models of remote home monitoring for COVID-19 varied (online platforms, paper-based systems with telephone calls or wearable sensors), but models using phone calls were considered more inclusive.
- This review was unable to reach substantive conclusions regarding patient safety and cost, due to study design, missing data and the lack of standardised reporting.
- Gaps in evidence for effectiveness, cost-effectiveness and patient experience of COVID-19 remote monitoring services.
Implementation of services during wave 136
- COVID-19 remote home monitoring services implemented throughout England varied in relation to the health-care settings and mechanisms used for patient triage, monitoring and escalation.
- Many factors facilitated implementation, including good communication within clinical teams, culturally appropriate information for patients and carers, and the combination of multiple approaches for patient monitoring (app and paper-based).
- The mean cost per monitored patient varied from £400 to £553, depending on the service.
Key findings from phase 2
Development and implementation of services during wave 2 (see Chapter 4)
- The rapid development of national remote home monitoring services took place in three phases: local development, national development and roll-out and local implementation.
- Development and implementation were facilitated by learning from wave 1, a range of national and local stakeholders and organisations, and a range of initiatives (community of practice, national learning network, AHSN patient safety collaborations).225
- Despite national roll-out, patient enrolment to COVID-19 remote home monitoring services was lower than expected.
- There was large variability in the models of remote home monitoring services that were implemented.
- Variation was influenced by patient, workforce, organisational and resource factors.
Outcomes (see Chapters 5 and 6)
- We found no association between rates of patient enrolment to CO@h services and COVID-19-associated mortality or admission to hospital. Additionally, for hospital admissions, no relationships were found between enrolment rates and in-hospital mortality or LOS (see Chapter 5).
- We found no evidence that the roll-out of CVW services reduced LOS in hospital or rates of readmission for patients hospitalised with COVID-19 (see Chapter 6).
- Low rates of patient enrolment and incomplete data may have affected the chances of detecting any possible impact.
Cost (see Chapter 7)
- We were unable to conduct a cost utility analysis due to the absence of impact of the services on mortality or hospitalisation.
- For CO@h services, the mean cost per patient triaged was £351.80 and per patient monitored was £527.50. For CVW services, the mean cost per patient triaged was £581.80 and per patient monitored was £599.10.
- The mean cost per patient monitored was lower for tech-enabled and analogue sites than analogue-only sites (CO@h and CVW services).
- Most staff involved in running COVID-19 remote home monitoring services were clinical staff. Examples of clinical roles included consultants, emergency department staff, GPs, nurses, ANPs and medical students.
Workforce experience and delivery (see Chapter 8)
- Staff generally reported positive experiences of delivery and felt that services were easy to deliver.
- Staff valued support provided nationally and locally. But some staff would have benefited from further training.
- Support for delivering the service, NHS resources, capacity and staff workload, multidisciplinary team dynamics, and patient (dis)engagement influenced delivery.
Patient experience and engagement (see Chapter 9)
- Many patients and carers reported positive experiences and felt that services and human contact received reassured them and were mostly easy to engage with.
- Engagement was conditional on a range of factors including patient factors (e.g. knowledge and health), support and resources, and service characteristics.
- Findings indicate that burden of treatment may be experienced by patients and families when implementing remote monitoring for COVID-19.
Disparities (see Chapter 10)
- Many sites designed their service in an attempt to be inclusive of the needs of local populations and to ensure a broad reach.
- Despite local adaptations to services, disparities were reported across patient groups (e.g. with age, health status, ethnicity and level of education) in their experience of, and engagement with services in terms of accessibility, achievability of tasks, and problems experienced.
- Certain groups may therefore require additional support to engage with COVID-19 remote home monitoring services.
Modality of service (see Chapter 11)
- Most services offered both analogue and tech-enabled submission options.
- The patient cohort using tech-enabled modes of relaying symptom data differed from patients using analogue methods (i.e. by age, level of education and ethnicity).
- The choice of mode used by patients was primarily made by staff, but most patients reported being satisfied with the mode allocated.
- Staff considered the tech-enabled models better equipped to manage large patient numbers.
- Many improvements were suggested to improve functionality of technology systems to better fit staff and patient needs.
- Tech-enabled engagement was not a substitute for human contact.
How findings relate to previous research
In this section we present findings in relation to previous research.
Implementation
The use of remote home monitoring
In recent years, remote home monitoring services have become a priority for policy-makers and the NHS.18,158 Previous research demonstrated that remote home monitoring models have been widely implemented for a range of chronic conditions.19–21 This evaluation extends the evidence base by showing that remote home monitoring models have also been widely used internationally for patients with COVID-19 (see Chapters 3 and 4). We provide evidence for the range of services implemented for COVID-19 and the types and frequencies of remote home monitoring used. Additionally, our review and implementation study provide insights into the components and processes involved in these services.
Major system change
Our evaluation provides nuanced understanding of major system change in a rapid emergency context (i.e. a global pandemic). There are different ways of implementing major system change (top-down and bottom-up approaches), but a combination of approaches may be beneficial.104 Findings from COVID-19 remote home monitoring services indicate that rapid implementation processes may follow a bottom-up, top-down then bottom-up process (as described in Chapter 4). Therefore, rapid implementation may vary from major system change in non-rapid contexts such as demonstrated by Fulop et al.103
Previous research outlined five conditions for major system change: involving stakeholders from all levels, establishing feedback loops, attending to history, engaging health-care providers, and engaging patients and families.102,104 Our findings offer support for the involvement of professionals and patients and families in service change as staff and patient engagement was integral in implementing COVID-19 remote home monitoring services. While engagement from staff and patients/carers is critical when implementing services rapidly, it is also potentially more challenging due to time constraints. Additionally, we highlight the importance of national and local clinical leaders and their pivotal roles in developing and shaping services, and the facilitating impact of collaboration between organisations.
Factors influencing implementation and variation
Prior to this evaluation, there was limited evidence regarding how COVID-19 remote home monitoring services have been implemented. This evaluation (see Chapters 3 and 4) found that most remote home monitoring services studied had low patient enrolment and that there was considerable variation in which model was implemented and how according to local context. These findings support previous research demonstrating likelihood of variation when implementing national services generally81–83,89–92 and in response to the COVID-19 pandemic.106–108 Previous research has highlighted differences in the models implemented and the ways in which services are implemented must be considered when interpreting effectiveness and cost-effectiveness of services.57,81,83,84,88
The context of rapidly developing COVID-19 remote home monitoring services during a global pandemic may explain some of the variation identified between national guidance and local implementation. Findings demonstrated that service models were initially developed locally, then further developed and rolled out nationally, before being implemented locally (see Chapter 4). It is likely that the rapid process of development of services may have culminated in variation (between national guidance and local services, and between different local services), as sites were often running prior to national guidance. Given the low enrolment rates and variation demonstrated in our evaluation, further consideration is needed to identify the ‘right’ balance between adhering to national SOPs (e.g. NHS)38,39 and flexibility needed within local services. This balance may be challenging to achieve for services like COVID-19 remote home monitoring, which have been rapidly developed and implemented during a pandemic, when time is limited. This limits the ability for national and local stakeholders to give, receive and implement feedback and also to understand how different approaches to implementation may relate to effectiveness.
Additionally, our evaluation offers support for previous research which has highlighted the role that local factors have on implementation.81–87 We demonstrate that local factors (patient, staff, organisational and resource factors) are of importance when implementing national services for COVID-19 (see Chapter 4). Our Phase 1 study also demonstrated facilitators (e.g. clear communication channels, positive engagement from patients, dedicated clinical leaders and support from senior management) and barriers (e.g. unclear referral processes, challenges monitoring patients remotely, lack of administrative support and resources, workforce capacity and lack of data systems) (see Chapter 3 and Vindrola-Padros et al.).36
Evaluating effectiveness and cost of services
Effectiveness of COVID-19 remote home monitoring services
Prior to this evaluation, there was little robust evidence for the effectiveness and cost of COVID-19 remote home monitoring services. Our findings on the national CO@h programme are consistent with other evaluations of the same programme, which found no impact on mortality or health service utilisation.43 However, our findings (for both CO@h and CVW) contrast with potentially promising effects of remote home monitoring services for patients with COVID-19 found by others,115–118,124–126 and individuals with other health and care needs.226,227 Our findings differ from previous COVID-19 research findings possibly due to differences in study design and limitations associated with these (including with respect to the intervention cohorts, outcomes, and comparator groups). In addition, these other studies typically reported on single-site or single-area implementation (see Chapters 5 and 6 for discussion of these studies). The low rates of patient enrolment and lack of complete, or linkable data may have affected the ability of our studies to detect any true impact that may have existed.
Despite some evidence that remote monitoring of patients with COVID-19 can be locally effective we have not been able to replicate such findings at a wider level. Because of the influence of missing data, lower enrolment to the CO@h service than expected and the lack of linkable data for the virtual wards, the more immediate concerns are for better data collection processes and enrolment rather than any judgement as to whether the services are worthwhile. Indeed, the effectiveness of some local programmes could have been lost among the analysis of national data.
Cost of COVID-19 remote home monitoring services
Prior to this evaluation, there was limited evidence of cost analyses for remote home monitoring services (see Chapter 3 and Vindrola-Padros et al.)32 Previous studies have stated that similar interventions may be cost saving by reducing downstream costs (in terms of avoiding hospital admissions and bed-days).31 Findings from our cost analysis of COVID-19 remote home monitoring services were consistent with a recent Australian study where staffing costs were the main contributors to the total costs of monitoring.148
Staff and patient experiences
Staff-patient relationships
Previous research has highlighted concerns over the staff-patient relationship being jeopardised by remote delivery of health-care services.164,170 However, learning from our staff and patient studies indicated that this was not the case. For example, our analysis of staff experience delivering tech-enabled and analogue modes of remote home monitoring indicated that staff valued remote home monitoring services as they were able to support patients and appropriately identify the right place of care for them, allowing a therapeutic relationship to develop (see Chapter 11). Additionally, the patient study indicated that patients felt reassured by the contact provided by staff remotely and that the care provided at a distance was appropriate (see Chapter 9). Remote home monitoring services do not necessarily jeopardise staff-patient relationships usually built during face-to-face appointments. That said, our findings indicate that some form of human contact is imperative within these services, supporting previous research.221,222
Staff experiences delivering COVID-19 remote home monitoring services
Previous research indicated that there were several potential benefits and challenges of remote home monitoring services for staff.153,155,157,158 This evaluation extends previous research by demonstrating that staff generally had positive experiences of delivering remote home monitoring services in this context, but that there are specific barriers that need to be overcome, for example relating to training and provision of oversight by senior clinicians. However, it must be acknowledged that some staff felt that there was additional burden and workload associated with delivering remote home monitoring services and, more generally, it is clear that delivery relies heavily on staff.
Patient engagement
Previous research has explored patient involvement in health-care delivery (e.g. self-management, engagement and treatment burden) for many chronic conditions,50,51,179,186,187 but little research had been conducted into patient experience and involvement in remote home monitoring services (using tech-enabled and analogue modes) for acute conditions such as COVID-19. Our evaluation offers considerable evidence to indicate that patients were able to engage with remote home monitoring services when managing acute conditions such as COVID-19 (see Chapters 9 and 11), but that this engagement was conditional on a range of patient, staff, organisational and resource factors and heavily relied on support from family and friends (see Chapter 9).
Staff and patient qualitative and survey findings indicated that there are groups of individuals for whom remote home monitoring services may be less appropriate due to enhanced difficulties engaging with services (see Chapters 8–11). This finding is consistent with previous research.197–200,208,215,228,229 Additionally, we highlight further challenges that must be considered for COVID-19, including infection transmission for family members providing support, and uncertainty and fear around seeking help in hospital settings.
Co-production
This evaluation extends previous research by outlining the importance of co-production of services with staff involved in delivering these services and patients receiving these services. Previous research has highlighted that it is necessary to involve patients and staff in development and implementation to ensure that services meet their needs230 and that service components, such as digital solutions, are fit for purpose.164,216,218,223 Our results indicated that efficiency, utility and value functionality of tech-enabled solutions could be improved by involving staff in co-design and offering them opportunities to observe and test different platforms. While findings indicated services did try to develop and implement strategies to meet the needs of their population (see Chapter 10), patient involvement was not necessarily consistently incorporated throughout service development and implementation (see Chapter 4). The rapid nature of development and set-up of services appeared to limit scope for involving staff and patients in decision-making. This meant that staff and patient experience was at times hindered by decisions made without involvement (see Chapter 11). There remain gaps on if and how conventional methods of engagement can work within rapid timescales; it is possible that engagement approaches for coproduction may need to be reconsidered within rapid contexts.
Implications
In this section we outline overarching implications arising from these findings which relate to implementation, evaluating remote home monitoring services, and staff and patient experience (see Table 23 for implications). These implications apply to COVID-19 but may also apply to remote home monitoring services for patients with other conditions.
| Topic | Implication |
|---|---|
| Implementation | |
| The use of remote home monitoring | 1. Details on how remote home monitoring services have been used, and the components of these services, can be used to inform the rapid future development, implementation and local adaptation of remote home monitoring services for COVID-19 but potentially also other acute conditions. |
| Implementation of COVID-19 remote home monitoring services | 2. Our understanding of how services have been implemented can be used to interpret effectiveness and cost of COVID-19 remote home monitoring services. |
| Factors influencing implementation and variation | 3. When developing and rapidly implementing national roll-out of remote home monitoring services, considering the context and priorities of local organisations may facilitate implementation. For example, service planners need to acknowledge that the eligible population and local population needs may vary between sites and over time, and that clinician discretion may play an important role in enrolment decisions. |
| 4. Services and national programmes need to be developed and evaluated in ways that encourage local variation and flexibility. Key considerations identified in the study of 28 services included:providing strategies to proactively identify cases and onboard eligible patientsproviding an indication to local services of the level of clinical oversight and staffing neededsupporting local services to work collaboratively across settingsproviding guidance on the resources needed to implement remote monitoring services. | |
| 5. Learning networks and communities of practice may be helpful in supporting local services to share and receive information on how services have adapted the national model, what is working or not working, and to learn from one another. This sharing may facilitate informal feedback loops into the development and implementation process. | |
| Evaluating remote home monitoring services | |
| Effectiveness | 6. There is a need for better data collection processes that operate alongside future services in order to evaluate impact and cost, not least to help understand which modes of implementation are most effective. This includes routine linkable patient-level data on onboarding and use of remote monitoring in the community and on the use of virtual wards after hospital discharge. |
| Service reach | 7. Demographic data for those accessing services should be collected, so that service planners/providers can understand reach and gaps in reach, and develop strategies to address them. |
| Cost | 8. Information on staff involved (by band, hours worked), the monitoring mode use (tech-enabled vs. analogue) and the other resources used should be consistently and carefully collected. |
| 9. Owing to the ad hoc nature of the COVID-19 remote home monitoring wards, information on planned and unused capacities will help to understand actual costs and support future planning. | |
| 10. Linkage of health outcomes with the economic costs would add significant knowledge for a possible cost-effectiveness analysis of remote home monitoring services. | |
| Staff and patient experiences | |
| Staff–patient relationships | 11. Service planners may be able to consider the use of COVID-19 remote home monitoring services (which include human contact) in lieu of establishing relationships face-to-face. Some patients had a wish for human contact (for health-related and emotional support), especially in the context of safety-netting. Therefore, services that do not offer human contact with staff may not always be appropriate or meet the full range of patient needs. The balance between remote and face-to-face modes needs to be considered for patients with COVID-19 and other conditions as appropriate. However, learnings must be viewed within the unique pandemic context (in which face-to-face contact was minimised and not necessarily desirable). It is not yet known whether public and staff views on the acceptability and appropriateness of remotely delivered care will endure beyond the pandemic. |
| Staff experiences | 12. Service planners and local services must ensure that processes are in place to support staff to deliver services efficiently (e.g. making sure there is sufficient staff capacity to manage demand for the service, availability of senior clinicians to provide oversight and greater training and guidance available to staff). |
| Patient engagement | 13. When designing services, service planners need to take into account and address the impact of treatment burden and the barriers faced by groups of patients who found engagement harder (e.g. older patients, those from ethnic minorities and patients with existing health problems). For example, different patients may require different levels of support, therefore services could develop/adapt their services to ensure they are providing sufficient frequency of contact with patients at different level of risk. |
| 14. As many patients required support to engage with these services, consideration needs to be given to infection transmission, the caring burden and those living alone | |
| Co-production | 15. Service planners (nationally and locally) should ensure that intended service users (patients and carers) and staff are consulted at the design stage of a service, to ensure that services are appropriate for use. It is not yet known how best to involve patients and staff in co-production when developing services rapidly. |
Strengths and limitations
Strengths
Our evaluation was a large-scale mixed methods evaluation which included analysis of national data sets, and primary data collection from 28 sites that were representative of a range of regions, levels of deprivation, ethnicity, size of population, and a range of urban versus rural sites. However, it is possible that our study may not be representative of all sites. Using mixed methods (consisting of a systematic review, qualitative data collection and analysis, quantitative data collection and analysis and economic data collection and analysis), enabled us to explore a range of topics and triangulate data from multiple perspectives and sources. For example, we were able to explore possible reasons for low patient enrolment demonstrated by the quantitative national data sets within the qualitative case-study findings.
This evaluation was a rapid evaluation completed over two time periods. Our phase 1 study was completed within one months, and our phase 2 mixed-methods study was completed within one year, with data collection taking less than six months. Our rapid approach to this evaluation meant that learnings from the evaluation were able to be used in real time to inform future service development and evaluation (see Project website documentation – Dissemination). Additionally, conducting our evaluation over two time periods (coinciding with waves 1 and 2 of the pandemic) enabled us to capture learnings and constants/changes over time.
Throughout our evaluation, a number of actions were taken to ensure robustness of qualitative findings (see Chapter 2).
Our evaluation team (a multidisciplinary research team) worked collaboratively with, yet independently from two other evaluation teams and key stakeholders from NHSEI and other organisations. This meant that evaluation teams were able to share findings to evaluate services from multiple perspectives, thus providing findings with more credibility. Additionally, our evaluation team worked closely with our PPI group and the 70@70 cohort throughout the study.
Limitations
Data completeness was a limitation of the quantitative and cost elements of this study. The evaluation of effectiveness for CO@h could only be carried out for CCGs where the onboarding data was believed to be complete, which was only one quarter of all CCGs. For CVWs, we were able to undertake a national analysis, but we had no knowledge about which patients were referred to a virtual ward, or even how many. Additionally, cost and resource use data provided by some sites were incomplete.
Our CO@h effectiveness study (see Chapter 5) and the cost study (see Chapter 7) used aggregated data rather than individual patient-level data for key analyses. This meant that patient-level variation in outcomes and programme costs could not be examined. However, patient-level data on outcomes were examined by the other evaluation teams.42–45
It was not possible to conduct all aspects of the evaluation as planned in the protocol. For example, a cost–utility analysis was not undertaken, given there was no evidence found of an effect of COVID-19 remote home monitoring services on mortality or downstream hospital utilisation. Additionally, for the qualitative study, we were unable to fully embed ‘think aloud’ interviews within our tech-enabled four case study sites as this was not feasible in practice.
Compared with patient onboarding data,114 the sample in the patient experience study was under-representative of some groups (e.g. older patients, ethnic minorities and most deprived) and over-representative of other groups. Additionally, the response rate for the patient and carer survey was fairly low (17.5%). For patient experience surveys and interviews, participants who had declined services could not be recruited. Additionally, our study would not have included the views of staff or patients who were unable or did not want to take part in surveys and interviews. Therefore, our study may not be representative of all patient and staff groups and experiences.
These services were rapidly developed to respond to the global pandemic. Additionally, our evaluation was rapidly conducted within a pandemic. Therefore, our conclusions may lack generalisability to other acute conditions, as staff and patient expectations and attitudes may be different during a pandemic (e.g. perhaps staff and patients may be more appreciative of the service, given the wider context).
Future research
While this evaluation has contributed to growing evidence on COVID-19 remote home monitoring services, there are still some further avenues of future research which would benefit from being explored.
First, further research, including longitudinal evidence, is needed to explore further the effectiveness and cost-effectiveness of COVID-19 remote home monitoring services. However, for this to happen, high quality complete data need to be collected for the uptake and use of these services that can be linked to routine data sets.
Second, further research is needed on the most appropriate type of model for remote home monitoring for different groups of patients. For example, further research could compare the effectiveness, cost-effectiveness, implementation and staff/patient experiences of remote home monitoring services which are supported by staff, versus those which are purely self-monitoring models (patients monitor and escalate their own care). Additionally, research could explore whether there is an optimum frequency of contact for patients with varying needs and risks; which models of remote home monitoring services are most appropriate for which groups of patients; how ‘standard’ models can be adapted to be more accessible and appropriate for diverse groups; and whether different models impact on number of patients onboarded to these services.
Finally, further research is needed to explore other experiences which may not have been captured within this evaluation. For example, sites who were less research active, groups of staff whose views may not have been included or were limited in this evaluation (i.e. ambulance staff and volunteers) and patients who we were unable to reach in this evaluation (e.g. those who declined or disengaged from the service). Additionally, future research needs to explore carers’ perspectives in more detail and any treatment burden associated with informal care provided as part of these services.
Equality, diversity and inclusion
Participant representation
Within this evaluation, we aimed to ensure that participants were representative of different locations within England and different backgrounds.
Site characteristics
We recruited 28 sites to our national staff and patient surveys and 17 sites for staff and patient interviews. The 28 sites included services from all regions across England, represented CCGs and trusts, varied in urban/rural characteristics, deprivation scores, ethnicity and size (see Chapter 4 and Report Supplementary Material 2).
Strategies to improve representation
Within the study we undertook a range of strategies to try and ensure that our study was inclusive. For example, we offered a choice of paper or electronic surveys for patients, and attempted to make it easy for participants to return surveys by providing free post envelopes. We also provided patient and carer study materials (information sheets, surveys) in English, Polish, Bengali, Urdu, Punjabi, French and Portuguese and offered translation services for interviews if needed. While sites offered potential participants translated materials, uptake of these was low and the research team received no completed translated surveys. We also aimed to purposively sample using a range of criteria (e.g. age, gender, ethnicity, onboarding mechanism, type of monitoring and outcome), to ensure that our sample was as inclusive and representative as possible. We offered different modes of interview (telephone, MS Teams or Zoom) and different times depending on participants’ preferences and to ensure accessibility for as many people as possible. We offered the option for a family member to answer on behalf of patients for both the survey and interview if appropriate.
Further efforts are needed to ensure that studies are as representative as they can be in future (e.g. raising awareness of translated materials and opportunities for interpretation services). There were limited opportunities in our study to reach out further due to services being gatekeepers to potential participants and the rapid pragmatic timescales of our study.
Participant characteristics
Our patient and carer surveys and interviews included a range of participants of different ages, living circumstances, educational qualifications, work situations, disability, and deprivation score. However, our patient and carer survey and interview participants were mostly white, straight and had English as a first language (see Chapter 4 and Report Supplementary Material 2). We also compared our participant demographics with service patient onboarding data,114 and found that our survey and interview sample was under-representative of some groups (e.g. older patients and ethnic minorities) and over-representative of other groups (e.g. participants from white ethnic groups and patients in the least deprived deciles of the IMD). Additionally, patient and carer response rates for the survey was only 18%. We were unable to interview or survey anyone who had declined the service or disengaged.
Staff included in our surveys and interviews varied in their role, whether they were redeployed, length of time in position, and the location they were based (see Chapter 4 and Report Supplementary Material 2). Additionally, the staff response rate for the survey was only 39%. We did not collect data relating to staff personal demographics (e.g. ethnicity) as we focused on professional characteristics.
Research topics relating to equality, diversity and inclusion
Our study had a clear focus on inequalities of COVID-19 remote home monitoring services, including an analysis of disparities relating to service reach and patient engagement with these services (see Chapter 10) and an analysis of patient and staff experiences of different modes of submission (see Chapter 11).
Reflections on research team and wider involvement
Research team
The research team consisted of researchers from two rapid evaluation teams: RSET and BRACE. Overall, our team included researchers from academic university settings and think tanks. The research team was a large, mixed-methods multidisciplinary team from many different disciplines (statistics, sociology, applied health research, health psychology, health economics and project management) and team members differed in seniority from professors to postdoctoral researchers and research fellows. The research team comprised a mix of backgrounds in relation to gender and ethnicity. Team members who took maternity/paternity leave during the course of the study were supported to remain engaged with the study and to ensure their contributions were recognised.
The team held weekly project meetings and weekly data collection and analysis meetings throughout the project to ensure that all members of the team were well supported. All members of the team have been involved in all aspects of the project, from conception, to data collection and analysis and dissemination of findings – both written (papers for publication and chapters) and oral (see Contributions of Authors). All researchers were supported by the principal investigator and senior members of the team.
Wider involvement
In addition to the research team, our evaluation also included input from our Clinical Advisory Group and RSET and BRACE PPI groups, who were actively involved throughout the project (including project planning, analysis and dissemination). Our PPI panel, involved throughout the project, included six members (representing a range of ages, gender and ethnicity). All our PPI panel members have experience with engaging with PPI activity across research and evaluation prior to their involvement in this study. Our PPI panel members have experience with a range of long-term conditions and all have experience of navigating and/or working within the health-care system. Through online meetings, workshops and e-mail, PPI members were involved throughout all stages of the research, including shaping the research questions, focus and methods, developing data collection tools for staff and patients, interpreting findings, and dissemination (e.g. all PPI members had the opportunity to comment on all chapters/papers if they wanted to and based on their interests, prior to their submission).
We attempted to work with services to seek specific PPI input from patients who had received COVID-19 remote home monitoring services, in relation to survey development and topic guide development; though this proved difficult to achieve due to rapid timescales. We also could have developed a specific study PPI group, but again this would have been difficult within the urgent rapid time frame.
Conclusions
To conclude, our evaluation was unable to provide evidence regarding the effectiveness of COVID-19 remote home monitoring services, but this may have been influenced by incomplete data and lower patient enrolment to services than expected. However, this evaluation offers insight into the cost, implementation, staff and patient experience of these services, and lessons learned. There was large variability in the models implemented in relation to the design and intensity of monitoring, workforce, uptake levels and enrolment criteria. Implementation was influenced locally by a range of patient, staff, organisational and resource factors. Tech-enabled monitoring was widely used and all sites were offered an analogue monitoring option. Services proactively used a range of strategies to ensure that their services were inclusive. Staff and patients generally viewed COVID-19 remote home monitoring services positively and felt that they provided reassurance for patients. However, these services were not without limitations, with staff and patients identifying some challenges to delivery and engagement. Findings indicate that for some patients and carers, remote home monitoring services may be burdensome, particularly for some groups. Future remote home monitoring services for COVID-19 and other conditions should operate alongside effective data collection processes, ensure that staff are well supported and have capacity to deliver these services and that patients have appropriate support, ability, and understanding to engage with them.