Pre-Hospital Early Warning Scores for Sepsis

The School of Health and Related Research (ScHARR) at the University of Sheffield are running a research study to address the National Institute for Health and Care Excellence (NICE) research recommendation “Can early warning scores be used to improve the detection of sepsis and facilitate prompt and appropriate clinical response in prehospital settings and in emergency departments?” (NICE 2016). The specific research question to be addressed is “What are the accuracy, impact and cost-effectiveness of prehospital early warning scores for adults with suspected sepsis?” This study is not part of a wider project or collaboration.

Sepsis is a life-threatening organ dysfunction due to a dysregulated host response to infection. Early recognition and treatment of sepsis are essential to reducing mortality. The Surviving Sepsis Campaign Bundle 2018 update recommends delivery of intravenous fluids and broad-spectrum antibiotics within one hour of presentation. This can only be achieved if sepsis is recognised and prioritised in the emergency care system.

Sepsis can be recognised by identifying clinical features such as altered mental state, low blood pressure or rapid respiratory rate. Early warning scores use simple measurements to calculate a score, with a higher score indicating a higher risk of serious illness and adverse outcomes. They can be used by prehospital providers, such as ambulance paramedics, to identify people with suspected sepsis who need to be prioritised for treatment.

An effective early warning score needs to ensure that people with the potential to benefit from urgent treatment are not missed. This can lead to delayed treatment and avoidable mortality and morbidity; whilst also ensuring that people without sepsis are not inappropriately prioritised.

Prehospital early warning scores can be used to prioritise treatment for people at high risk of sepsis but the accuracy of existing scores and the impact of their use is uncertain.

ScHARR at the University of Sheffield have the specific study objectives to:

1. Estimate the accuracy of early warning scores for predicting potential to benefit from time-critical treatment for sepsis

2. Estimate the operational consequences and cost-effectiveness of using early warning scores to guide prehospital decisions for adults with suspected sepsis

The data provided by NHS digital will allow the study team to identify a manageable number of patients for focused instigation by hospitals, rather than looking at tens of thousands of patient records. The NHS digital data will allow the study team to describe the study cohort, the treatments they have received, and their diagnoses. The NHS digital data will also allow the study team to estimate the health and financial benefit of implementing any early warning scores. These actions would not be possible in any reasonable or accurate way within the constraints of the study budget or time without the support of data from NHS digital.

The study is a retrospective diagnostic cohort study using routine data sources across four acute hospitals with associated ambulance services and decision-analytic modelling. The target population is adults transported to hospital by emergency ambulance with possible sepsis. Patient report form data will be used to identify eligible cases and collect pre-hospital measures used to calculate early warning scores. Cases will be linked to routine hospital data sources. The index tests will be prehospital early warning scores identified through a systematic literature search and selected by an expert group, who will also consider if there is a need to create any additional scores.

The passing of NHS Digital data to ScHARR University of Sheffield has three purposes 1) to allow ScHARR University of Sheffield to identify those patients who may have had sepsis and warrant investigation for inclusion to the study reference standard. The reference standard is how ScHARR University of Sheffield define if someone has sepsis and therefore warrant further investigation. 2) To securely send NHS numbers to participating hospitals to allow them to identify patients and the relevant episode for inclusion to the reference standard. 3) To allow the University of Sheffield to effectively describe the patient cohort, the treatments they received, and their outcome, e.g. mortality.

The data subjects in this research study are patients conveyed from two participating ambulance trusts to the four participating hospital trusts. The study will include all adult patients transported to hospital by emergency ambulance unless they would clearly not be managed as suspected sepsis. A group of approximately 200 patients with confirmed sepsis will be identified within this cohort. This sub-group will act as the reference standard positive for early warning score testing. This number of patients will allow ScHARR University of Sheffield to estimate how accurate existing scores are at detecting sepsis.

ScHARR University of Sheffield will create a decision-analytic model to estimate the impact of using early warning scores to: (i) Alert the receiving hospital so that the patient is seen immediately on arrival; (ii) Provide prehospital treatment for sepsis. The model will simulate the prehospital management of a hypothetical cohort of patients with suspected sepsis and then model their flow through an emergency department (ED), alongside all other patients attending the ED. The model will estimate the operational consequences of using different early warning scores, in terms of the number of attendances appropriately and inappropriately prioritised and/or given prehospital treatment. They will adopt a health service perspective to estimate costs and will value outcomes as Quality Adjusted Life Years (QALYs) to estimate the incremental cost per QALY of using different strategies and will undertake a full incremental analysis.

ScHARR University of Sheffield are requesting NHS Digital data to help estimate the number of patients transported to hospital that may require a pre alert of their condition to an ED, allowing the ED to provide more timely care.

The data being requested is individual-level health data, identifiable by NHS number, and includes:

Civil Registration Deaths (Secondary Care Cut) - This will be used to identify patient outcomes including mortality and recorded cause of death.

HES Admitted Patient Care (APC) and HES A&E – These will be used to describe the patient cohort (including characterising patients who would be identified by an early warning score but who do not have sepsis – false positives; and characterising patients who have a low early warning score but meet the study’s reference standard of sepsis – false negatives); identify cases with the reference standard of sepsis in the cohort for assessment against early warning scores; and contribute to the care impact and economic modelling of early warning scores.

HES Critical Care (CC) – This will assist with identification of sepsis in the cohort, and contribute to the care impact and economic modelling of early warning scores.

Identifying false positives and false negatives, alongside the outcomes (e.g. treatments and diagnoses) of patients in the reference standard will help estimate the operational consequences and economic modelling of early warning scores. The NHS Digital data will also allow the study to identify secondary benefits to early warning scores, e.g. identification of other conditions.

Data fields being requested provide information on investigation activity conducted for patients, treatments provided to patients, patient diagnosis, comorbidities, discharge information, and basic demographics.

Individual-level data is required to be able to achieve effectively the study aim by linking individual predictor data to NHS Digital data and the reference standard. Individual-level patient data is required for us as a study team to link together records from Ambulance services, NHS digital, and hospitals. Without individual-level data the study team could not assess early warning scores for which is most effective at detecting sepsis and sepsis related adverse in a pre-hospital setting. No less intrusive ways have been identified to achieve the study purposes to the same level of academic rigour. The level of data requested for ScHARR University of Sheffield is considered the minimum required to complete the study effectively.

The period of time requested has been chosen as the most recent year where complete data is likely to be available without interference from COVID-19, this being 2018/19 & 2019/20. The aim is to access data from the whole of 2019. One whole year of data is being requested as there is a need to fully capture different seasonal episodes of sepsis.

ScHARR University of Sheffield are only requesting Ambulance conveyance to EDs between Yorkshire Ambulance Service or West Midlands Ambulance Service and the four participating hospitals. This geographical spread, i.e. the number of Ambulance trusts and hospitals, has been selected to allow for the required number of patients with Sepsis to be identified for inclusion in the reference to effectively test early warning scores.

Combined, the Ambulance services of West Midlands and Yorkshire expect to identify an estimated 92,000 cases across one year transported to the four participating hospitals. The Pre-Hospital Early Warning Score (PHEWS) research team anticipate that around 200 will be reference standard positive. The estimate of the incidence of reference standard positive cases is based on data from the Intensive Care National Audit and Research Centre (ICNARC, 2012) and National Confidential Enquiry into Patient Outcome and Death (NCEPOD, 2015) investigations of critical care cases with sepsis.

Data minimisation

. The following steps have been taken to minimise the data being requested in the application:

• Only data relevant to the cohort of patients provided to NHS Digital by the two participating Ambulance Trusts is being requested.

• Only requesting 12 months worth of data.

• Ambulance services are removing patients who meet the study exclusion criteria, i.e. under 18s, patients with injury, mental health problems, cardiac arrest, and direct transfers to specialist services e.g. maternity, cardiac or stroke services.

• Only the data fields that will assist in achieving the stated study aim and objectives are being requested.

ScHARR University of Sheffield will not be excluding patients with symptoms such as delirium. The diagnostic classifications used by the two ambulance services code presentations such as delirium in ways that we have classified as non-specific presentations, such as confused/distressed/upset, and have therefore included in our cohort. We only exclude those coded with clear mental health classifications, i.e. Schizophrenia, Depression, Bipolar Disorder, Anxiety Disorder, OCD, Eating Disorder, Personality Disorder, Dementia, Brain Injury, Alzheimer's, Schizoaffective Disorder.

ScHARR University of Sheffield specifically stated in our funding application that we would exclude children on the basis that “The presentation and management of sepsis differs markedly between adults and children so the use of early warning scores in these patients therefore needs to be studied separately”. ScHARR University of Sheffield are therefore not funded to undertake an evaluation involving children. The National Institute for Health Research, which funded our study, has funded other research into an assessment tool for unwell children, including those with suspected sepsis. Specifically, this project - https://www.fundingawards.nihr.ac.uk/award/PB-PG-0815-20034.

ScHARR University of Sheffield do not specifically exclude pregnant women, but will exclude women presenting with obstetric diagnostic classifications (i.e. Eclampsia, Ectopic, Hyperemesis, Labour, Miscarriage, Pregnancy, PV Bleed, Child Birth, Obstetric (other)) that would result in their being taken to a specialist maternity service rather than a general emergency department. Paramedics would not use an early warning score for sepsis to prioritise women who were being taken to a specialist maternity service with an obstetric complaint.

The processing of NHS Digital data under this data sharing agreement is considered to be in the public interest in the area of public health due to the potential to improve the standard of care provided to patients with sepsis with early identification of their condition in a pre-hospital setting. The General Data Protection Regulation legal bases are Article 6(1)(e) - Processing is necessary for the performance of a task carried out in the public interest or in the exercise of official authority vested in the controller and Article 9(2)(j) - Processing is necessary for archiving purposes in the public interest, scientific or historical research purposes or statistical purposes. This research will be working towards ensuring high standards of quality and safety of health care provided to the public by prehospital and emergency department services in the early detection and treatment of Sepsis.

The University of Sheffield is the data controller who process data in this agreement. The four participating hospital trusts are also acting as data processors of data provided by NHS digital to the University of Sheffield. The four hospitals are University Hospitals Coventry & Warwickshire, Doncaster and Bassetlaw Teaching Hospitals NHS Foundation Trust, The Rotherham NHS Foundation Trust, and Sheffield Teaching Hospitals NHS Foundation Trust. Additionally, Yorkshire Ambulance Service NHS Foundation Trust and West Midlands Ambulance Service University NHS Foundation Trust will provide NHS Digital with data.

This is an NIHR funded study that commenced on July 1st 2019 and is scheduled to finish on the December 31st 2022.

Yorkshire and West Midlands Ambulance Services will identify patients transferred to any of the four participating hospital trusts between 1st January and 31st December 2019 who meet study inclusion criteria. Patients to whom any of the study exclusion criteria apply, i.e. under 18s, patients with injury, mental health problems, cardiac arrest, and direct transfers to specialist services e.g. maternity, cardiac or stroke services will not be included the cohort.

The Ambulance Services will send NHS Digital the direct patient identifiers (NHS number, date of birth, and sex) along with unique study identifiers for each member of the cohort via NHS Digital’s Secure Electronic File Transfer system. They will also send ambulance patient data affiliated with the unique study identifier to ScHARR University of Sheffield, but no direct identifiers. Data from the Ambulance Services will be transferred to an authenticated account on ScHARR University of Sheffield servers, provided by the University for this purpose, using an encrypted connection. Two separate passwords (1. to access the server account and 2. To unencrypt the encrypted datasets) will only be provided by telephone.

NHS Digital will retrieve the patient data that ScHARR University of Sheffield has been approved to receive. NHS Digital will screen the patient list for patients who have opted out of research and remove them. NHS Digital will then provide the HES and mortality data with the affiliated unique study identifiers and patient NHS numbers and securely send it to ScHARR University of Sheffield. ScHARR University of Sheffield will screen the patient data received from NHS Digital to identify patients who may have ultimately received the reference standard diagnosis of sepsis after being transported to one of the participating hospitals.

All data received by the University of Sheffield will be stored on a virtual machine with access controlled by the Information Asset Owner. Access to the virtual machine will be restricted to a minimum number of people. All users granted access to the study data will be compliant with the ScHARR Information Governance policy and processes.

ScHARR University of Sheffield will provide participating hospitals with data on patients who attended their hospital only. The four participating hospitals will have read-only remote access to NHS numbers, unique study identifiers, and date of linked attendance/ admission only for relevant episodes that the ScHARR study team believe warrant investigation for inclusion to the study reference standard. The four participating hospitals will only have this data provided on patients that meet the study inclusion criteria, including arrival in the relevant hospital’s emergency department in 2019. This data will be held on a RedCap database hosted on University of Sheffield servers. Only members of the study team at the hospital will have access to these identifiers; in order to identify the patient and the relevant episode of care. The Information Asset Owner at ScHARR University of Sheffield will control access. The four hospitals will use the data from NHS Digital along with patient notes to determine who did or did not have sepsis.

Participating hospitals will inform ScHARR University of Sheffield on whether a patient had sepsis and some observations made on the patient in the hospital’s emergency department. This includes: admittance to critical care, if no critical care admission why not, details of sepsis 3 criteria (internationally recommended criteria for diagnosing sepsis), was treatment withdrawn on the basis of a lack of response, the potential value of early treatment, patient’s clinical frailty score (a measure to assess a patient’s frailty), was there a Do Not Attempt Resuscitation order or upper level of care decision (commonly known as a ceiling of care) present for the patient, patient's Charlson Co-morbidity Index (the number of pre-existing illnesses that the patient has), and patient survival details. Further details found about the reference standard are in the study protocol. Data from participating hospitals is being returned to ScHARR University of Sheffield through a Red Cap database hosted on University of Sheffield servers. The data provided by participating hospitals is vital to assess the severity of sepsis and the treatment provided. This is data that cannot be obtained from other sources and compliments data provided by NHS digital.

ScHARR University of Sheffield will link patient data provided by Ambulance Services, NHS Digital, and reference standard details provided by participating hospitals. These linkages will be achieved by the unique study identifier being present at each stage of data collection/identification and sent to ScHARR University of Sheffield. This study will not be seeking to re-identify participants for research purposes outlined in this application.

Data processing is being conducted by participating hospitals and ScHARR University of Sheffield. ScHARR University of Sheffield will process NHS provided data to 1) link to data provided by Ambulance services, 2) to identify patients that warrant further investigation for the study reference standard 3) describe the cohort of patients, 4) provide direction to participating hospitals with NHS numbers, study identifiers and date of attendance/admission for relevant episodes for patients to be investigated for inclusion in the reference standard. The description of the cohort is integral for the study to assess the impact of early warning scores, e.g. their economic impact on services or secondary benefits. Additionally, identifying patients to be investigated for inclusion in the reference standard is integral to the study/hospitals being able to identify a full range of patients with sepsis and therefore allowing for a more accurate assessment for early warning scores.

A statistical method called receiving-operator characteristic (ROC) curves will be used to graphically represent and compare how accurately different approaches or early warning scores identify sepsis. The ROC curve allows us to unite data from patients who had a correctly identified case of sepsis (true positive rate) against those who had sepsis but were not treated correctly (false positive rate). A range of ROC curves will be created to test the scores’ different ranges. True positive rate, true negative rate, and the area under the ROC curve (a measure of diagnostic accuracy) will be reported.

This study will also explore whether a clinically credible new score can be derived to improve upon existing scores. This would entail the best predictor variables being selected from a set of potential predictors, with the aim of reducing complexity and enhancing prediction accuracy.

The proposed approach to data processing in this study has been carefully considered by members of the studies patient and public involvement (PPI) group. This group includes members with personal experience of sepsis and care in emergency care settings. Two of these members are co-applicants to the study and have been involved in the design and ongoing management of the study as members of the study’s project management group.

This project is being undertaken to address a NICE research recommendation: “Can early warning scores be used to improve the detection of sepsis and facilitate prompt and appropriate clinical response in prehospital settings and in emergency departments?” (NICE 2016). It is expected to communicate their findings to the NICE Guideline Development Group in the form of a study report due January 2023. The University of Sheffield is expected to also send their findings to other key organisations responsible for producing guidelines for the management of sepsis, including the UK Sepsis Trust and the Joint Royal Colleges Ambulance Liaison Committee.

The study's dissemination is expected to be built around informing health care professionals, policy makers and the wider public on the effectiveness of sepsis early warning scores in pre-hospital/ambulance settings. With this in mind it is hoped that dissemination channels will be selected to maximise the impact and reach of results, depending on their applicability to that audience. Dissemination channels is expected to include but not be limited to: open access peer reviewed publications, online report, website, social media, presentations and posters. All publications are hoped to be open access, funds have been secured within the project grant for this purpose. The study outputs will be drafted in conjunction with the studies patient and public involvement (PPI) group. Additionally, two members of the PPI group are study co-applicants, meaning they are involved in all project management meetings and will be co-authors on the study’s final report.

It is expected that The University of Sheffield will prepare plain language and professional summaries of their findings, downloadable presentations and materials that can be used to calculate early warning scores and present their operational consequences in a tailored format (for example, the consequences for an ambulance service or for an Emergency Department (ED)). These plain language summaries will be created and approved in partnership with members of the study’s PPI group.

The University hope to publish their findings in high-impact, open access, peer-reviewed journals and aim to present at relevant professional meetings, such as the 999 Emergency Medical Services Research Forum and the Royal College of Emergency Medicine Annual Scientific Meeting. They will send a summary of their findings to each NHS ambulance service, with links to dissemination materials.

To reach the public and interest groups in the sepsis community organisations such as the sepsis trusts will be contact to pass on if appropriate disseminate results from the study. This aspect of the study dissemination plan will be developed alongside the study's patient representative group, which has members from, Sheffield Emergency care forum, Yorkshire Ambulance Trust, the UK Sepsis Trust.

The data in publications will be aggregated data with small numbers suppressed in line with the HES analysis guide.

Expectations from the funder will be a minimum of an end of study report. The study has already secured funding from the funder for two potential publications and a conference.

The dissemination of results from this study will hopefully contribute to the debate on the improvement in “the provision of health care” within the health care system. This will aim to be achieved by providing evidence on what ways pre-hospital early warning scores for sepsis can detect sepsis and if they can be cost effective.

The ultimate aim of this project is to identify an optimal prehospital early warning score for the NHS, in terms of maximising prioritisation of treatment for people with sepsis without over-burdening the emergency care system. If this is achieved, then the early warning score would be recommended by the study team in relevant national guidance and leading to potential implementation across the NHS. This may lead to better treatment for people with sepsis and/or more targeted prioritisation of treatment across the emergency care system. In turn this could contribute to a reduction in mortality and morbidity.

The study outputs should help to inform policy makers, professionals, and the public on the accuracy of pre-hospital early warning scores for predicting potential to benefit from time-critical treatment for sepsis. Additionally, outputs should further inform policy makers, professionals, and the public on the operational consequences and cost-effectiveness of using early warning scores to guide prehospital decisions for adults with suspected sepsis.

ScHARR University of Sheffield believe that the publications on the accuracy, impact and cost effectiveness of pre hospital early warning scores for sepsis is in the public interest. No individual would be identifiable from the information that ScHARR propose to be published from this study. However, details about patient groups, e.g. those who have had sepsis, would be published. ScHARR University of Sheffield are consulting with their patient representatives group on their views of the study being in the public interest, including those from an interest perspective of emergency care, Ambulance, and sepsis.

The benefits from this study, processing of data, and subsequent publication outputs is to reduce mortality and morbidity, and can contribute to practice and/or debate on the pre-hospitalisation early detection of sepsis, appropriate targeting of resources for sepsis, and the cost effectiveness of using early warning scores for sepsis. Depending upon the results the study may lead to a change in the pre-hospital early warning score approach used in the NHS to improve patient care and/or cost benefit.

The study team has not yet received the data from NHS digital. The study has made good progress with data provided from participating ambulance services, to calculate a raft of early warning scores and contribute to the economic modelling.