Overview
This chapter draws on a manuscript by Sherlaw-Johnson et al.109 published in Lancet eClinicalMedicine (2022): https://doi.org/10.1016/j.eclinm.2022.101318. This is an open access article under the term of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits use, distribution and reproduction in any medium, provided the original work is properly cited (see https://creativecommons.org/licenses/by/4.0/).
What was already known
- Previous quantitative studies have mostly assessed remote oximetry monitoring services for patients with COVID-19 at individual sites and focused on their safety.
- There is some existing evidence that remote monitoring of patients with COVID-19 can be locally effective, particularly among higher-risk patient groups. However, effectiveness of national programmes has been little studied.
What this chapter adds
- We have found no evidence that national implementation of pre-admission home monitoring of older people diagnosed with COVID-19 impacts on mortality, hospital admission rates, in-hospital mortality or length of hospital stay.
- Missing data and lower than expected enrolment of patients to services may have influenced our results, and the effectiveness of some local programmes could have been lost among the analysis of national data.
- Future implementation requires better data collection strategies and effective learning from areas that have achieved higher patient enrolment.
Background
Among the aims of remote monitoring of the CO@h service was the reduction in mortality through earlier identification of deterioration. Furthermore, it was hypothesised that if fewer patients with COVID-19 were admitted to hospital with advanced disease and critically low oxygen levels, there may be a reduction in the use of critical care facilities, fewer deaths within hospital and shorter lengths of stay. The anticipated impact on numbers of hospital admissions was less certain since the aim of the programme was not to reduce admissions, but to make sure people who needed to be in hospital were admitted sooner. However, any consequence on the number and mix of patients admitted with COVID-19 would be useful to understand as remote monitoring may have different impacts on different types of individuals.
Earlier studies of the use of oximetry for remote monitoring within England during the country’s first wave focused on aspects of safety and implementation, but were unable to establish reliable comparators for measuring impact.32,110
This chapter presents findings from the clinical effectiveness workstream of the study addressing the specific research questions:
- What was the impact of CO@h on mortality?
- What was the impact of CO@h on the incidence of hospital admission for COVID-19 or suspected COVID-19 and on the characteristics of those admitted?
- For these admissions, what was the impact on in-hospital mortality and LOS?
Our evaluation was requested by NHSEI alongside two other evaluations of clinical effectiveness which were intended to complement each other.43,45,55 Our approach used combinations of unlinked, aggregated population-level data and hospital administrative data. We were able to undertake a rapid analysis that worked alongside the other evaluations and provided valuable insight in the future evaluation of similar programmes implemented at scale.
Methods
Our methods are described in Chapter 2 and Report Supplementary Material 1.
Results
Data completeness and patient enrolment
As we mentioned in Chapter 2, because of the design of our study we were only able to evaluate services within 37 CCGs where we judged there was complete data on numbers of people enrolled on to services (onboarded) between 2 November 2020 and 21 February 2021. The study population included anyone aged 65 years or over with a laboratory-confirmed positive test for COVID-19 and any hospital admission within that age group for COVID-19 or suspected COVID-19.
The included CCGs had no notable differences in mean age, proportions of non-white population or proportions resident in most deprived areas when compared with the remaining 98 that were not included, although included CCGs had a lower incidence of positive test results (Table 2). There were also regional differences: no CCGs from the East NHS region were included, and only one from the North East and Yorkshire (see Figure 8). The South West, North West and Midlands were the best represented regions.
| Included CCGs (n = 37) | Excluded CCGs (n = 98) | p-value for difference between groupsa | |||
|---|---|---|---|---|---|
| Mean | (Standard error) | Mean | (Standard error) | ||
| Mean proportion aged 65 years or more | 18.1% | (0.7%) | 17.9% | (0.5%) | 0.77 |
| Mean proportion aged 80 years or more | 4.8% | (0.2%) | 4.8% | (0.1%) | 0.91 |
| Mean proportion in non-white ethnic groups | 17.3% | (2.7%) | 17.7% | (1.8%) | 0.97 |
| Mean CCG-level IMD | 22.7 | (1.3) | 22.6 | (0.8) | 0.96 |
| Incidence of laboratory-confirmed COVID-19 per 1000 people | |||||
| All ages | 41.9 | (1.7) | 46.8 | (1.4) | 0.03 |
| Age 65 years or over | 30.4 | (1.3) | 35.0 | (1.2) | 0.01 |
FIGURE 8 Number of CCGs with complete data by NHS region (% complete shown in each bar). (Figure not reproduced here; see the original article.)
Fortnightly patient enrolment to CO@h among people aged 65 years or over within the 37 CCGs is shown in Figure 9. Rates were particularly low earlier in the period because many sites had not commenced implementation. The numbers of CCGs where there were sites onboarding patients within each fortnight is shown along the horizontal axis. Sites within nine (24%) CCGs were operating services in the first fortnight, which had risen to 33 (89%) within the fortnight beginning 28 December 2020. Services were operating within all CCGs during the fortnight beginning 11 January 2021. The median enrolment rate only exceeded 10% in the final fortnight, although, from the end of November 2020, the maximum was consistently above 30%, with one or two CCGs each fortnight achieving much higher rates than the rest. The overall enrolment rate over the period across all 37 CCGs was 5.9%. If we exclude fortnights during which services were either not operating or operating for only part of the fortnight, the overall enrolment rate was 8.7% with only one CCG averaging a rate of more than 30%.
FIGURE 9 Variation in enrolment rates of CO@h among people aged 65 years or over with laboratory-confirmed COVID-19 across the 37 CCGs included in the analysis. (Figure not reproduced here; see the original article.)
Summary data and outcomes
Summary data and outcomes for the period from 2 November 2020 to 21 February 2021 for the 37 CCGs by age band are shown in Table 3. Although there were many more positive tests recorded among the under 65s, there were approximately half as many deaths and equivalent numbers of hospital admissions. Enrolment rates after implementation was low across both age groups and highest among those aged 65–79 years.
| Age band | Both age groups | ||
|---|---|---|---|
| 65–79 | 80+ | ||
| Positive tests and onboarding over the whole period: | |||
| New positive diagnoses (n) | 41,437 | 27,944 | 69,381 |
| Patients onboarded to CO@h (n)a | 2734 | 1382 | 4116 |
| Enrolment rate (%) | 6.60 | 4.95 | 5.93 |
| Positive tests and onboarding since implementationb | |||
| New positive diagnoses (n) | 26,507 | 18,636 | 45,143 |
| Patients onboarded to CO@h (n)a | 2610 | 1320 | 3930 |
| Enrolment rate (%) | 9.85 | 7.08 | 8.71 |
| Mortality: | |||
| Deaths (n) | 4269 | 8699 | 12,968 |
| New diagnoses over the period (%)c | 10.30 | 31.13 | 18.69 |
| Hospital admissions: | |||
| Admissions (n) | 12,351 | 12,128 | 24,479 |
| New diagnoses over the period (%)c | 29.81 | 43.40 | 35.28 |
| Hospital admissions (excluding readmissions within 28 days) (n) | 10,895 | 10,683 | 21,578 |
| New diagnoses over the period (%)c | 26.29 | 38.23 | 31.10 |
| Within-hospital outcomes: | |||
| In-hospital deaths (n) | 2748 | 3948 | 6696 |
| In-hospital deaths per admission (%) | 22.25 | 32.55 | 27.35 |
| Median LOS in days (IQR) | 8 (3–15) | 9 (4–17) | 8 (4–16) |
| Patients staying 2 weeks or more (n) | 3489 | 4134 | 7623 |
| Proportion staying 2 weeks or more (%) | 28.25 | 34.09 | 31.14 |
Time series plots of numbers of newly diagnosed cases and numbers onboarded to the programme within the selected 37 CCGs over the study period (2 November 2020 and 21 February 2021) are shown in Figures 10 and 11. New diagnoses peaked in the fortnight straddling the new year, while onboarding peaked in the following fortnight. Profiles of all COVID-19 deaths and admissions for confirmed or suspected COVID-19 are shown in Figures 12 and 13.
FIGURE 10 New diagnoses of COVID-19 by fortnight; data for the 37 CCGs included within the analysis. (Figure not reproduced here; see the original article.)
FIGURE 11 People onboarded to the CO@h programme by fortnight; data for the 37 CCGs included within the analysis. (Figure not reproduced here; see the original article.)
FIGURE 12 All deaths from COVID-19 by fortnight within the 37 CCGs included in the analysis. (Figure not reproduced here; see the original article.)
FIGURE 13 Hospital admissions for confirmed or suspected COVID-19 by fortnight within the 37 CCGs included in the analysis. (Figure not reproduced here; see the original article.)
Rates of in-hospital mortality and median lengths of stay for patients admitted to hospital with confirmed or suspected COVID-19 are shown in Figures 14 and 15. Mortality rates peak at approximately 30% over the new year and decline thereafter to around 21%. Median lengths of stay decline from the end of November 2020 while admission numbers increase.
FIGURE 14 In-hospital mortality by fortnight for patients resident within the 37 CCGs and admitted with confirmed or suspected COVID-19. (Figure not reproduced here; see the original article.)
FIGURE 15 Median LOS by fortnight of admission for patients resident within the 37 CCGs and admitted with confirmed or suspected COVID-19. (Figure not reproduced here; see the original article.)
The characteristics of patients admitted with confirmed or suspected COVID-19 over the study period are shown in Table 4.
| Explanatory variable | Value | Patients | |
|---|---|---|---|
| (n) | (%) | ||
| Ethnicity | White | 19,811 | 88.99 |
| Non-white | 2450 | 11.01 | |
| Charlson Index | ≤5 | 19,442 | 79.42 |
| >5 | 5037 | 20.58 | |
| Sex | Male | 12,807 | 52.32 |
| Female | 11,672 | 47.68 | |
| Deprivation | Most deprived quintile | 5295 | 21.75 |
| Excluding most deprived quintile | 19,053 | 78.25 | |
| Age (Years) | 65–79 | 12,351 | 50.46 |
| 80+ | 12,128 | 49.54 | |
| Virtual ward active on admission | Yes | 9537 | 27.98 |
| No | 13,562 | 28.94 | |
| COVID-19 bed occupancy on admission | <15% | 5932 | 24.23 |
| 15–22% | 6131 | 25.05 | |
| 22–29% | 5944 | 24.28 | |
| >29% | 6472 | 26.44 | |
| Weighted enrolment | <5% | 15,914 | 65.01 |
| 5–10% | 4182 | 17.08 | |
| > 10% | 4383 | 17.91 | |
Mortality and hospital admissions
Results from our models for mortality and hospital admissions are shown in Table 5, Figures 16 and 17. For every 10% increase in enrolment, mortality fell by 2% [relative risk = 0.98, 95% confidence interval (CI) 0.96 to 1.01] and admissions increased by 3% (relative risk = 1.03, 95% CI 0.99 to 1.07), but neither result is statistically significant. There is, however, a significantly higher risk of mortality and admission among the older age group (p < 0.001) and higher risk of mortality in the months following November 2020 (p < 0.001 for each month in comparison with November).
| Variable | Relative risk of death (95% CI) | Relative risk of hospital admission (95% CI) | |||
|---|---|---|---|---|---|
| Enrolment | Every 10% increase in enrolment | 0.98 | (0.96 to 1.01) | 1.03 | (0.99 to 1.07) |
| Age (years) | 80+ vs. 65 to 79 | 3.06 | (2.94 to 3.17) | 1.55 | (1.48 to 1.63) |
| Month | December 2020 vs. November 2020 | 1.25 | (1.17 to 1.34) | 1.06 | (1.01 to 1.11) |
| January 2021 vs. November 2020 | 1.24 | (1.14 to 1.34) | 1.03 | (0.97 to 1.11) | |
| February 2021 vs. November 2020 | 1.40 | (1.27 to 1.55) | 0.97 | (0.88 to 1.07) | |
FIGURE 16 Impact of each variable used in the multivariate model on the risk of death (mortality rate ratios). Error bars denote 95% CIs. (Figure not reproduced here; see the original article.)
FIGURE 17 Impact of each variable used in the multivariate model on the risk of hospital admission (admission rate ratios). Error bars denote 95% CIs. (Figure not reproduced here; see the original article.)
We also found no significant relationship between rates of enrolment to CO@h and the ethnicity, sex, deprivation or health status of patients admitted to hospital with COVID-19 or suspected COVID-19 (Table 6). The impact on age is of borderline statistical significance with each 10% increase in enrolment associated with a reduction of nearly four months (95% CI −7.9 months to 0.2 months, p = 0.07).
| Characteristic | Percentage change with each 10% increase in enrolment (95% CI) | |
|---|---|---|
| Proportion non-white ethnicity | 0.43 | (−8.99 to 10.82) |
| Proportion female | 0.07 | (−1.9 to 2.11) |
| Proportion resident in most deprived areas (by quintile) | 0.30 | (−7.19 to 8.41) |
| Proportion with Charlson scores > 5 | −2.75 | (−6.37 to 1.00) |
| Change in mean age with each 10% increase in enrolment (95% CI) | ||
| Mean age | −3.9 months | (−7.9 to 0.2) |
In-hospital outcomes
The results of our analysis of in-hospital outcomes are shown in Table 7, Figure 18 and Figure 19. For every 10% increase in patient enrolment, in-hospital mortality fell by 3% (relative risk = 0.97, 95% CI 0.92 to 1.03) and LOS increased by 1.8% (95% CI −1.2% to 4.9%). Again, neither result is statistically significant. Non-white ethnicity and existing COVID-19 bed occupancy on admission were both associated with shorter lengths of stay.
| Variable | OR associated with in-hospital mortality (95% CI) | Relative change in LOS (95% CI) | |||
|---|---|---|---|---|---|
| Enrolment | Every 10% increase in enrolment | 0.97 | (0.92 to 1.03) | 1.8 | (−1.2 to 4.9) |
| Age (years) | 80 + vs. 65 to 79 | 1.73 | (1.63 to 1.84) | 6.5 | (3.1 to 9.7) |
| Month | December 2020 vs. November 2020 | 1.27 | (1.14 to 1.42) | −4.2 | (−9.5 to 1.5) |
| January 2021 vs. November 2020 | 1.12 | (1.00 to 1.26) | −11.5 | (−14.9 to −8.1) | |
| February 2021 vs. November 2020 | 0.88 | (0.76 to 1.03) | −19.7 | (−27.4 to −11.1) | |
| Sex | Female | 0.71 | (0.66 to 0.76) | 2.5 | (0.0 to 5.0) |
| Charlson score | >5 | 2.14 | (1.96 to 2.33) | 1.1 | (−2.2 to 4.6) |
| Ethnicity | Non-white ethnicity | 1.24 | (1.07 to 1.42) | −12.0 | (−16.2 to −7.6) |
| Deprivation | Lowest IMD quintile | 0.98 | (0.89 to 1.07) | 0.0 | (−3.0 to 3.1) |
| Has a virtual ward | 0.96 | (0.88 to 1.04) | 1.0 | (−6.5 to 9.1) | |
| COVID-19 bed occupancy in the trust on admission | Every 10% increase in occupancy | 1.04 | (0.97 to 1.11) | −5.2 | (−7.6 to −2.7) |
FIGURE 18 Impact of each variable used in the multivariate model on the risk of in-hospital mortality for people admitted with confirmed or suspected COVID-19 (OR). Error bars denote 95% CIs. (Figure not reproduced here; see the original article.)
FIGURE 19 Impact of each variable used in the multivariate model on LOS for people admitted with confirmed or suspected COVID-19 (% change). Error bars denote 95% CIs. (Figure not reproduced here; see the original article.)
Sensitivity analysis
Results from sensitivity analyses are shown in Report Supplementary Material 1. None of the changes we made in our assumptions affected our findings with respect to the association between coverage of remote oximetry and outcomes.
Discussion
Summary of findings
In this study, we have found no significant association between rates of patient enrolment to the CO@h programme, and COVID-19-associated mortality or COVID-19-associated admission to hospital. For such hospital admissions, we found no significant relationship to in-hospital mortality or LOS. Also, for COVID-19-associated admissions, there appears to have been no associated change in the patient characteristics we have tested.
Interpretation
There are several possible explanations for these results, and it would be premature to conclude that the CO@h programme has had no impact. Firstly, limitations in data completeness meant that we were only able to analyse onboarding data for one-quarter of CCGs across England. This, combined with lower than expected patient enrolment within these CCGs, meant that our ability to detect any possible impact was smaller than anticipated. This also meant that some areas of the country were under-represented. Secondly, individual-level associations may not have been seen at the aggregate level. Thirdly, qualitative findings from our wider study revealed that CO@h was implemented in a variety of ways by different services within CCGs, some of which may have had more impact on outcomes than others (see Chapter 4). The possible impact of the intervention on hospital admissions was always uncertain, since the aim of the programme was to ensure people who needed hospital treatment were admitted at the right time. Also, once in hospital, the determinants of LOS are complex and multifactorial, and may have varied during the time course of the second wave of COVID-19 infection in England.
Strengths and limitations
The anticipated value of our approach with aggregated data was that it would complement the other two simultaneous quantitative evaluations of CO@h,43,45 could be carried out more rapidly and, if COVID-19 continues to stretch national health services, it could be more readily repeated as new data become available, provided the right information is routinely collected at source.
We were able to handle small number suppression and the rounding of aggregate data by multiple random sampling throughout the range of possible values and it was encouraging to discover that this uncertainty did not have a large impact on results.
Using aggregated data has some limitations because it does not enable us to trace direct links between the onboarding of an individual and their outcomes. There may also be an ecological fallacy, where individual-level effects are not observable in the aggregated data.111 However, obtaining linked individual-level data is a complex and potentially long process that may not always be feasible when there is a need to provide rapid feedback to a developing programme and where resources are stretched. Unfortunately, however, our ability to provide rapid feedback was compromised by delays in obtaining onboarding data which proved an understandable challenge for local services in the midst of a pandemic.
The wider mixed-methods study has added insight into some of our findings and provided locally collected bespoke data against which we could verify information we received centrally about patient enrolment and data completeness. However, these checks could only be made against the 28 study sites and we were not able to verify the data from the other CCGs in the same way. Findings from surveys and interviews have helped interpret what we have found and, conversely, our data analyses have helped provide a context against which to understand the relative importance of the qualitative findings (see Chapters 4 and 8–11).
The qualitative study has also shed light on the different ways that the services had been implemented. However, with just 12 of the 28 study sites being among the 37 included in our analysis, in combination with low enrolment, it was not possible to undertake a quantitative study of the relative performance of the different types of services implemented.
We anticipated that finding a suitable comparator group during the national implementation of a programme was likely to be problematic, and we therefore avoided this problem by treating the relationship between rates of enrolment and outcome as a dose–response. However, the power to detect any impact in such an analysis depends on the level of enrolment which, in practice, was lower than we hoped. This approach also assumes any selection bias with those being enrolled is negligible.
During the period of our analysis the vaccination programme was starting and by the end of our study period 88% of people aged 65 years or over had had at least one dose.112 Although this study investigates outcomes of people after being diagnosed with COVID-19, there is evidence that vaccination changes the subsequent risks of mortality and hospital admission113 which could have had a confounding effect on our analysis.
In the context of fortnightly data, we assumed a minimal lag between the diagnosis of COVID-19 and onboarding to CO@h. However, there was evidence from sites that they sometimes encountered delays in identifying positive cases,114 although the overall impact on this assumption is uncertain.
Comparison with other studies
Prior to this study, very little was known about the quantitative impact of the use of pulse oximeters for COVID-19 remote home monitoring. One of the other evaluations of the CO@h programme in England also found no significant impact on mortality or health service utilisation.43 However, the study did find reductions in mortality and increases in hospital attendance (yet with lower use of critical care) among people enrolled on to the programme after attending the emergency department.42 A study of 4384 high-risk patients receiving home monitoring of vital signs, including pulse oximetry, in one region of Galicia, Spain, found lower admissions, lengths of hospital stays and in-hospital mortality when compared with other local regions.115 A retrospective cohort study from South Africa evaluated the use of pulse oximetry for home monitoring of people diagnosed with COVID-19 and, as with the Spanish study, deemed to be of high clinical risk.116 They found a significant improvement in mortality but no impact on admissions to hospital; the reduction in mortality being explained by earlier admission. A recent study of CO@h carried out at one site in England demonstrated reductions in 30-day mortality and lengths of stay among people admitted to hospital over the second wave.117 The control group in this study were patients admitted with COVID-19 who were not on the CO@h programme and it is not clear whether that would be an inherently higher risk group. In another study implemented in the UK during the first wave, patients with suspected COVID-19 attending the emergency department were discharged home with an oximeter. They observed a reattendance rate of 4.7% compared with 22.7% among a retrospective control group.118 However, this was a younger cohort (median age 41 years) and the absolute numbers of reattendance were small (nine in all). Other studies have reported on the safety of similar programmes, but have lacked comparators.28,43,110,119–121
Implications and further research
At the start of this study, we anticipated that services would have higher rates of patient enrolment and that complete data would be available from more CCGs. Although the use of aggregated population-level data can enable more rapid evaluation of a new service, these two elements had an influence on the power of the analysis to detect an impact. The resulting shortfall in expected data reflects the challenges of trying to centrally manage a bespoke data collection while services are already stretched. However, sufficient quantities of data are vital to determining whether a service is effective, so it is important to understand how this can be improved, for example, by concentrating data collection in a few sites and using routinely collected data wherever possible.
Furthermore, low patient enrolment raises questions about capacity of both staff and resources amid high infection rates and how it is possible to secure the best value from such a service under the circumstances. The fact that at least one CCG managed to achieve reasonably good rates of enrolment indicates the possibility for learning from others.
Other studies, as described above, have provided evidence of positive impacts on high-risk cohorts, and similar findings may follow from focusing evaluation of the CO@h programme in the same way.
Conclusions
This study provides an evaluation of the national implementation of remote home monitoring of pulse oximetry for people diagnosed with COVID-19 across the English NHS. Although we detected no significant impact on outcomes, there are potential explanations for this finding that are unrelated to the effectiveness of the programme. Taking due account of populations that may respond less well to oximetry, there is no evidence that future implementation of similar programmes would be unsafe. However, the challenges of providing sufficient data so that effectiveness can be adequately measured need to be overcome.