COVID Oximetry At Home - (CO@H): Improvement Analytics Unit

The purpose of this work is derived from the need to evaluate national roll out of the Covid Oximetry @Home (CO@H) programme.

The COVID Oximetry @home (CO@H) programme involves the remote monitoring of patients with coronavirus symptoms. Patients use a pulse oximeter, a small monitor clipped to their finger, to measure their oxygen saturation levels three times a day.

They record their results using a smartphone app, web portal or paper diary. The paper-based option is available at all sites for patients who are uncomfortable with or unable to use a digital solution to record their readings. Patients are supported by clinical staff locally, so that if they need further treatment they can be admitted to hospital at the right time. Currently, services are delivered by a range of provider organisations, including Clinical Commissioning Groups, Primary Care Networks and acute hospital trusts.

Processing this data is in the public interest as it will provide evidence as to the health and safety outcomes of home oximetry as a clinical pathway for patients with Covid-19. This evidence will support the health service to more effectively treat and manage Covid-19, and therefore is of substantial public interest. The Health Research Authority decision tool for defining research has been completed and the proposed work is not considered research and therefore does not need ethical approval.

The data is required for service evaluation purposes - meeting the conditions outlined as per Article 9 (2)(h) of the GDPR. “processing is necessary for the purposes of preventive or occupational medicine, for the assessment of the working capacity of the employee, medical diagnosis, the provision of health or social care or treatment or the management of health or social care systems and services on the basis of Union or Member State law or pursuant to contract with a health professional and subject to the conditions and safeguards referred to in paragraph 3”. The Health Foundation are carrying out service evaluation work (under the instruction of NHS England) as described in this agreement to investigate how the quality of care can be improved.

As such, this work is of significant public health concern undertaken by a service evaluation team conforming to General Data Protection Regulation (GDPR) Article 6(1)(e) and Article 9(2)(h).

Patients with serious coronavirus symptoms often do not go to hospital early enough, and this can negatively impact patient outcomes. Some patient groups more likely to present late include healthcare workers and BAME patients. Observational data suggests that for every day of delay in hospital admission the chance of death increases by 1%. (Source NHS Digital https://digital.nhs.uk/coronavirus/covid-oximetry-at-home-digital-and-data-services).

Delayed presentation can also lead to invasive treatment in Intensive Care Units being required and longer hospital stays.

The national Covid Oximetry @Home programme builds on initial pilot work that focused on the use of pulse oximetry and the remote monitoring of patients with coronavirus at home, then referred to as ‘virtual wards’. The pilot work focused on three locations (Tees Valley, Slough and North West London). The pilot work aimed to address the clinical problem that patients with severe coronavirus sometimes present to hospital relatively late – mainly due to the fact that in coronavirus oxygen levels can fall to dangerously low levels with very few symptoms.

Clinical Commissioning Groups (CCGs) were advised in November to set up ‘COVID Oximetry @home’ services as rapidly as possible. The default assumption is that the model will be primarily implemented in general practice as one of seven priority goals for the additional £150m General Practice COVID Capacity Expansion Fund. The model relies upon timely referral of patients that may meet the entry criteria from all relevant providers operating within the area. The supply of pulse oximeters available to CCGs is based on national modelling assumptions of case demand using agreed entry criteria.

The eligible population for the CO@H programme is people who are clinically suspected of having or tested positive for COVID-19 and are aged 65 or over or aged 18 and over and considered clinically extremely vulnerable or have a learning disability.

The aim of this work is to quantitatively assess the cost effectiveness and health and safety outcomes of the CO@H intervention as well as variation in access and outcomes. The data requested is uniquely capable of achieving this aim as it contains necessary information about patients who are enrolled onto the CO@H programme as well as their outcomes. Furthermore, the data requested will support critical analysis regarding inequalities to accessing the pathway, patient-level outcomes after being onboarded into the pathway and CCG-level performance of the CO@H programme. Only this data, inclusive of the requested variables, will be able to support these analyses and therefore achieve the aim identified.

The proposed evaluation work is commissioned by NHS England, who have also commissioned Imperial College London to evaluate the CO@H programme. Although both Imperial and the Health Foundation will lead on their own evaluation work stream (and as such apply for these data separately via the DARS Data Sharing Agreement DARS-NIC-421524-R0Y3P), there is commitment to share learning between the two pieces of work to ensure robust findings, a common definition of key metrics, and improved efficiency in tackling any issues arising from the data.

This evaluation will use routinely collected data from patients who have tested positive for COVID or are clinically suspected of having COVID and have been enrolled into the CO@H programme. Eligibility criteria for the work will NOT include patients under the age of 18. These data will be used comparatively, by comparing against patients testing positive for COVID, but not in receipt of the CO@h programme. The eligibility criteria for the work are people who are clinically suspected of having, or test positive for COVID-19 and are:

• aged 65 or over, or

• aged 18 and over and considered clinically extremely vulnerable, or

• aged 18 and over and diagnosed with a learning disability.

Data is required for COVID POSITIVE Patients (18+) across all CCGs, (Approx 3.7 million records) PLUS those on the CO@H Programme.

In terms of the datasets required, the following justifications apply:

• General Practice Extraction Service (GPES) Data for Pandemic Planning and Research (GDPPR) will be used to determine the demographic features (eg sex and LSOA) and clinical comorbidities (eg asthma, diabetes or COPD) of patients enrolled into the CO@H programme. Additionally, these data will be obtained for individuals not enrolled into CO@H, both before and after the start of the CO@H intervention (from March 2020 onwards - CV19 positive patients). This will enable construction of appropriate control populations to address key evaluation questions regarding equity of access to the service and the safety of the service.

This dataset comprises general practice data on patients registered in 97.5% of all practices in England. This will be the source of data on baseline characteristics for the patients included in the analysis.

The age of patients (in whole weeks relative to a reference date) will be calculated by NHS Digital using this dataset. From this, The Health Foundation will derive the age at first COVID-19-positive test (e.g. 65 years and 3 weeks). The Health Foundation will request data for patients born between 1st October 1940 (i.e. turning 80 on 1st October 2020, the earliest potential start date) and 1st May 1971 (i.e. turning 50 on 1st May 2021).

• Covid-19 testing data [COVID-19 Second Generation Surveillance System (SGSS)]

This system holds data on COVID-positive tests, including the full date of first COVID-positive test. SGSS is the national laboratory reporting system used in England to capture routine laboratory data (mainly on infectious diseases and antimicrobial resistance).

Only the first COVID-positive test is captured in this database. The Health Foundation will also be provided testing data from the COVID-19 UK Non-hospital Testing Results (‘pillar 2’). These sources will be used to identify the case cohort.

[COVID-19 UK Non-hospital Antigen Testing Results] will be used to identify those patients enrolled into CO@H following a positive test for Covid-19, and the time between a positive test and enrollment. Additionally, Covid-19 testing data will be used to identify those patients who had a positive Covid-19 test prior to the start of the CO@H intervention as a means of establishing a matched preintervention control population. Patient with a positive Covid-19 test after the start of the CO@H intervention will be identified in order to evaluate the equity of allocation of the intervention across all those with a positive Covid-19 test.

• Bespoke Covid Oximetry @Home data (collected as part of the program) will be used to identify the time and routes of enrolment of patients (primary care or secondary care), in addition to the clinical acuity of patients at onboarding. These data are required to determine the duration between enrolment and presentation to hospital or mortality, and to evaluate variation in clinical acuity and routes of enrolment between sites. These data will also be used to determine the time spent on the home oximetry programme across patient groups and between sites. It will also be used to determine whether patients used a standard or ‘tech enabled’ intervention during their CO@H programme.

o This dataset lists the date of patient onboarding (i.e. enrolment onto the CO@h intervention).

o These data will allow The Health Foundation to check the assumptions of a discontinuity in onboarding at age 65.

o In addition, by triangulating with hospital data, we will be able to check other assumptions – e.g. that onboarding is primarily occurring before hospital attendance.

o Onboarding data were due to be collected across all CCGs from 1st December 2020, with the option of retrospectively collected data from as early as 1st October 2020 from sites that started rolling-out CO@h earlier.

o However, due to COVID-19 pressures, as of mid-January 2021, this patient-level data is not currently being reliably recorded nationally. Thus, the quality of this dataset is a key risk to the execution and validity of the evaluation and its ability to identify a treatment effect.

• HES Admitted Patient Care (HES APC) and HES Critical Care (HES CC) data will be used to identify presentations to critical care departments and hospital admissions for patients enrolled into the CO@H programme, and also those within relevant control populations. Evaluating the rates of secondary care interaction are a critical aspect of the evaluation. Data from March 2017 is required in order to identify any previous diagnoses of comorbidities contained in the hospital records as part of the process of matching intervention patients to control patients. Data are requested from April 2017 to the present date in order to identify clinical comorbidities and procedures not captured in GDPPR that may influence COVID-19 prognosis.

• Shielded Patient List

This dataset identifies patients who are regarded as clinically extremely vulnerable (CEV) and at high risk of serious illness from COVID. Only the CEV flag will be disseminated.

• Emergency Care Data Set (ECDS) data

o This will be used to provide information on A&E attendances; one of the evaluation outcome measures.

o Data are requested from April 2019 to the present date in order to identify clinical comorbidities not captured in GDPPR that may influence COVID-19 prognosis.

• Civil Registration Mortality Data are required for all individuals in order to identify mortality rates in the CO@H intervention population, and also in control populations. Date of death is required to determine the time from enrolment and Covid-19 testing to death in order to conform to Public Health England definitions of Covid-19 mortality. Cause of death will allow non-Covid-19 mortality to be distinguished from deaths attributed to Covid-19.

PDS Data - Care Home Flag from the UPRN Asset

• Data used to flag care home residents

o Based on patient registration information derived from pseudonymised National Health Application and Infrastructure Services (NHAIS) data.

o This will facilitate focussed evaluation of the impact of the CO@h programme in care homes, as requested by NHS England

All data will be required in pseudonymised format to enable patient-level evaluation across datasets.

All datasets will cover the period from March 2020 till the present day. This date range has been selected as it marks the starting point of the COVID pandemic in England, and will therefore be relevant to the programme. The bespoke CO@H data set is only being collected as part of the CO@h programme, and will not be available before October 2020. The date ranges for the SUS APC and ECDS data go back in time further. This historic data will be used to help characterise the patient population eligible for the CO@h programme by looking at previous hospital utilisation.

Data will be required for all of England in order to evaluate performance of the national implementation of CO@H.

Following discussion between the evaluation partners, NHS England and in consultation with NHS Digital, who are assisting with data flows, it has been determined that there is no alternative, less intrusive way (using less information) of achieving the purpose of the evaluation.

In order to minimise the data requested, the Health Foundation have only included variables needed to carry out the analysis that has been agreed by NHS England. As part of a consultation with NHS Digital and clinical and academic experts, the Health Foundation has narrowed the dataset to focus only on information that can demonstrate the health and safety outcomes of the CO@H programme.

Given the role of NHS England in determining the scope, and purpose of the evaluation they are the Data Controller for this data, and will also carry out data processing alongside the Health Foundation.

For this dissemination - the legal basis to disseminate the data is the Control of Patient Information Notice (COPI) Regulations. NHS Digital has chosen to pseudonymise the confidential information in accordance with the COPI regulations.

All organisations party to this agreement must comply with the Data Sharing Framework Contract requirements, including those regarding the use (and purposes of that use) by “Personnel” (as defined within the Data Sharing Framework Contract i.e: employees, agents and contractors of the Data Recipient who may have access to that data).

The Health Foundation will use two distinct statistical methods to evaluate the CO@h programme. Each of these approaches is described in a Statistical Analysis Plan, or protocol, but also described below.

(1) Regression discontinuity design (RDD):

This evaluation will take advantage of the 65-year threshold for eligibility for CO@h for patients with a confirmed COVID-19 diagnosis. The Health Foundation will compare hospital outcomes between patients who have tested positive to COVID-19 and are aged 65 or over, with those of patients who have tested positive to COVID-19 just under 65. The analysis will produce a ‘local treatment effect’ at age 65.

The Health Foundation expects that the risk of severe complications from COVID-19 is highest in the first 28 days – consistent with Public Health England’s definition of COVID death as being within 28 days of the first positive test. Furthermore, the standard operating procedure for CO@h stipulates that if there are no signs of deterioration within 14 days of onset of symptoms, patients can be discharged from CO@h. Therefore, in this evaluation, each patient above and below the running variable’s threshold will have a follow-up period of up to 28 days from their first COVID-positive test. This analysis will focus primarily on the pre-hospital use of oximeters.

Regression discontinuity design is appropriate in situations where eligibility for an intervention changes sharply at a predefined threshold (in this instance at different age thresholds. This design largely avoids problems of observed or unobserved confounding, as patients just below and above the threshold will in general be expected to be similar in baseline characteristics. Under these circumstances the Health Foundation can therefore attribute any change in outcomes at the threshold to the intervention.

(2) Generalised Synthetic Control (GSC):

The use of pulse oximeters (as described in section 5a above) for remote monitoring of high-risk individuals with a COVID-19 diagnosis is taking place in a growing number of GP practices and community teams in England. A clinical pathway using pulse oximetry technology for remote monitoring is being delivered by a growing number of CCGs in England not only in primary and community health-care settings (primary care remote home monitoring model), but also to patients being discharged from A&E departments (secondary care model) or from hospitals (step-down model). Individuals eligible to enter a CO@H pathway (the target population) are COVID-19 confirmed patients who are either of 65 years of age or over, or identified as clinically extremely vulnerable irrespective of their age.

National roll-out of pulse oximeters for remote monitoring of high-risk COVID-19 patients since December 2020. Known delays in a number of CCGs ordering and accessing the technology has led in practice to a staggered roll-out across CCGs over time. A programme evaluation could therefore proceed by comparing on a range of outcomes CCGs offering to their eligible patient population a CO@h pathway (treated or intervention units) over a prefixed period of time (the analysis or follow-up period) with those CCGs not concurrently providing the technology (untreated or control units). The start and length of the follow-up period would be chosen to ensure that by its end there remains a sufficiently sized pool of control CCGs in England to carry out a meaningful outcome comparison from. The exact analysis length will be based on the data and will depend upon uptake and spread. Outcomes chosen to inform this comparison would comprise selected secondary care and mortality indicators, which will be retrospectively collected from administrative data.

The effect on a given outcome of the CO@h intervention implemented in a treated CCG at a given time following its introduction (post-intervention or follow-up period) will be quantified by the change (if any) detected in the CCG at that time between the outcome rate that was actually recorded, and what would have been hypothetically observed had the CCG not implemented a CO@h pathway (the counterfactual). As is invariably the case in programme evaluation assessments relying on retrospectively collected data, the counterfactual cannot be observed in practice; as such it needs estimating. The evaluation will rely on the Generalised Synthetic Control (GSynth ) method. The synthetic control method is used to evaluate the effect of an intervention in comparative case studies. It involves the construction of a weighted combination of groups used as controls, to which the treatment group (those who receive the CO@H intervention) is compared. This comparison is used to estimate what would have happened to the treatment group if it had not received the intervention. Unlike difference in differences approaches, this method can account for the effects of confounders changing over time, by weighting the control group to better match the treatment group before the intervention.

Once a counterfactual is derived for each CCG implementing a CO@h pathway during the evaluation follow-up period, an estimate of the impact at a given time of the CO@h initiative across treated CCGs (the Average Treatment effect among the Treated, or ATT) will be obtained by averaging across treated CCGs the effect estimates obtained for the corresponding period. Lastly, an overall estimate of the impact of CO@h in CCGs where it is implemented can be obtained by averaging previously derived impact estimates over the evaluation follow-up period.

The Health Foundation will not link any additional information to the data requested on patient level for either analysis design listed above, however, the Health Foundation will add contextual information derived from public data sources, and will include information at LSOA level (e.g. English indices of deprivation 2019, www.gov.uk), at GP level (e.g. characterisation of GP practice registered population using Quality Outcome Framework achievement and prevalence scores, NHS Digital), and at CCG level (e.g. number of registered GPs, NHS Digital).

Both analysis designs listed above will only require the use of pseudonymised data. These data will be processed on the Health Foundation Secure Data Environment (SDE), a dedicated analytics environment designed to maintain the confidentiality of the data, and protect against any unwarranted disclosure of information. The SDE is a secure environment hosted in an off-site secure data processing facility owned by UK Cloud (listed as storage location).

All processing will be carried out by Approved Users of the SDE. Approved Users are substantive employees of the Health Foundation (including associate analysts that maybe payed through a contract, rather than pay-roll), or substantive employees of NHS England with access to the Health Foundation’s environment.

NHS England employees who are part of the Improvement Analytics Unit (IAU), a partnership between the Health Foundation and NHS England, will have access to and process the data on the Health Foundation’s Secure Data Environment.

All Approved Users need to complete mandatory information governance training specific to the SDE, to help them understand their responsibilities, and make sure they have the appropriate skills to process information safely. No information can be removed from the SDE without going through a thorough Statistical Disclosure Control (SDC) process, to make sure that analysis findings cannot be used to identify individuals, and are in line with the latest guidance on SDC (which includes the suppression of any small numbers). This process requires at least two pairs of eyes to review analysis findings prior to release from the system.

Statistical data analysis will be carried out using on the Health Foundation’s cloud-based Secure Date Environment (SDE) hosted by UK Cloud. Devices must use a specific Windows virtual desktop application, with security configuration in place at the application level to block screen sharing and recording capabilities. Approved Users can only access the SDE from either the user’s nominated UK residence or from within the designated SDE area at the Health Foundation’s offices. To remotely access the SDE, the devices require a secure 2-factor authenticator and users are then able to securely access the secure server hosted by UK Cloud using an encrypted connection. All data analysis will be conducted within the confines of the SDE using appropriate statistical packages within the SDE and cannot be downloaded to remote devices for storage or processing.

UK Cloud do not access data held under this agreement as they only supply the building. Therefore, any access to the data held under this agreement would be considered a breach of the agreement. This includes granting of access to the database[s] containing the data.

This extension (v1.2):

The study team has requested that their agreement be extended to compensate for the delay in data dissemination. The data required to carry out the evaluation as described in this application were not received by the Health Foundation till 23 July 2021 due to delays in data collection from the COVID Oximetry @ home implementation sites. The project has been extended from an initial endpoint of May 2021 to September 2021. The timeline of the project have been adjusted accordingly (see below) with a view to share results with the programme team early September. In order to maximise the benefit from this work the study team intend to submit the findings for peer-review, as per the original application. Data access for an additional 6 months up to the end of March 2022 is required to support write-up and peer review of their analysis.

An initial assessment of the data has shown that our Regression Discontinuity Design is not feasible, and we intend to not carry out this work going forward. That said, we will go ahead with our Generalised Synthetic Control analysis as planned. The data required for both study designs is the same, and no changes to the data received is required to support this work.

------------------------------------------------------------------------------

The Health Foundation expects to produce the following outputs, which are described in detail below:

• Monthly reports with interim findings prepared for the CO@h evaluation workstream

• Final internal briefing for key stakeholders in NHS England, NHS Digital and NHSX.

• Open access paper in peer review journal on the RDD evaluation

• Open access paper in peer review journal on the GSC evaluation

• Publicly available policy briefing published through the Health Foundation website

• Conference presentation on CO@h evaluation (e.g. the annual HSR UK Conference organised by HSR UK for organisations and individuals involved in health services research in policy and practice)

As a result of the data processed described above, the Health Foundation will share interim findings with the CO@h evaluation workstream on a monthly basis (on top of regular reporting in weekly meetings).

The Health Foundation will prepare a briefing on the findings for circulation to key stakeholders in NHS England, NHS Digital and NHSX. These outputs will only contain aggregated data with small number suppression applied.

The Health Foundation will submit two academic papers to peer-reviewed journals. Upon submission, the Health Foundation will also publish a pre-print version of the paper, for immediate dissemination. Provided the Health Foundation is successful in the publication of our paper, the Health Foundation will pay for open access where required.

The Health Foundation will synthesise findings from both studies, as well as the wider literature in a policy briefing that will be made publicly available through the Health Foundation website.

The Health Foundation will present findings from the evaluation at HSR UK (described above).

Where possible the Health Foundation will collaborate with evaluation partners in the CO@h programme, to synthesise evidence across multiple studies, and maximise impact form the work.

The overall timeline of this work will be dependent on the development of the COVID-19 pandemic in England, and the duration of the CO@h programme as a result of this. The suggested timeline below may be subject to change.

Timeline:

• Monthly reports to CO@h programme from February 2021 to March 2021 - extended to September 2021.

• Briefing to key stakeholders – 7 September 2021.

• Submission to peer review journal on GSC – October 2021

• Dissemination of paper through pre-print server – October 2021

• Publication of both papers will depend on peer review process – March 2022

• Policy briefing – December 2021 - TBC

• Conference presentation at HSR UK – July 2021, as described above - feasibility findings presented.

• Conference presentation - TBC. We will explore possibilities to share findings at a further conference.

Sharing this data enables the evaluation of a major pathway of care for patients with Covid-19. Understanding the cost effectiveness and health and safety outcomes of the CO@H programme is critical to building safe remote monitoring pathways for Covid-19 patients in the community. Delivering effective, safe Covid-19 care is vitally important to patients and their care, as well as to NHS staff and the future of how the pandemic is managed.

In terms of measurable benefits, this work aims to quantifiably demonstrate the health and safety outcomes of remote oximetry for Covid-19 at a patient and CCG-level. These outcomes will drive how the NHS cares for and monitors Covid-19 patients in the community. They will also reveal any changes needed to make remote oximetry services more equitable and more effective.

The pipeline from outputs to national policy is explicit in this piece of work: the results from this evaluation will be shared regularly with NHS England, specifically with those involved in the policies and standards surrounding Covid Oximetry @ Home services. Therefore, the results from this evaluation are aimed to directly inform national policy makers about how best to roll out, modify and improve home oximetry for Covid-19 patients. This pipeline from evaluation to decision making to community benefit has already been demonstrated in the pilot phase of this work, whereby an evaluation of safety of the ‘Covid Virtual Ward’ programme led to the national roll out of the CO@H programme.

The quantitative evaluation of CO@H requires understanding the clinical outcomes of patients onboarded into the CO@H programme as well as key features of their medical history. These data points are included in the data collection specification and directly enable an evaluation of the CO@H programme which would not be possible with any other data. Given the unprecedented need to understand how best to manage Covid-19 in the community, the data required for this evaluation aims to enable a statistically robust analysis which can inform and direct national policy making.

The results from this work aim to be made available to key stakeholders, namely NHS England in regular reports. Furthermore, the final results will be made public with targeted publications for NHS England, but also within the public academic literature, as a scientific contribution.

The evaluation aims to guide the ongoing implementation of the CO@H programme at national and local level. It will provide evidence as to the equity of access to the CO@H programme and the safety of the CO@H programme. These findings will be used during the implementation of the programme to support emerging evaluation concerns raised by local sites and the national delivery partners. Actions arising from the evaluation will be implemented centrally by NHS England or locally by Clinical Commissioning Groups and other care providers as required.

Findings from this analysis hopes to benefit all patient eligible for the CO@h programme which include:

• All registered patients in England aged 65 and over

• All registered patients in England aged 18 and over who are deemed clinically extremely vulnerable in relation to COVID-19, in line with Government guidelines

Findings from this evaluation aim to serve the following purposes:

• To improve health service delivery (by evaluating policies and reporting feedback to the NHS and policy makers)

• To make health policymaking more effective (the work packages look at specific policy implementation and how this has been effective and feedback will be given to policy makers)

The outputs are intended to directly support the provision of the Covid Oximetry at Home programme of care by NHS England and will therefore meet the stated objective.

A key benefit may be the continued roll out of this programme potentially reducing morbidity and mortality from COVID-19.

As final datasets were received on 23rd July 2021, there have been no yielded benefits from the final data. Interim analyses have been shared internally with partners in NHSX, NHS England and NHS Digital to guide their understanding of CO@H program implementation to date.