Digital triage: investigating patient service use and health outcomes following triage in Urgent Care settings

Project summary:

Care services that provide out of hours care (care outside of general practice opening) often use digitally supported ‘telephone triage’. This telephone triage involves a health care service staff member using a ‘digital triage tool’, to help signpost the patient to the most appropriate service to receive health care; in part this helps health care services manage high demand. The digital triage tool is used by the clinician to input the patient’s symptoms in order to automatically generate the signposting or care advice. This research investigates how patients use these types of services. It also aims to understand how telephone triage affects patients’ use of other health care services, such as Accident and Emergency (A&E), and what happens to patients following telephone triage, in terms of their health. This study will analyse pseudonymised, pre-existing data from NHS providers in order to better understand how these telephone triage services are used and how they affect patient health. Ultimately the aim is to better understand how telephone triage is used and to identify areas for improvement that will benefit patients.

Research aims:

The two research aims below will be addressed before and after the start of the COVID-19 Pandemic;

1. How do patients use urgent care services that are delivered though telephone based digital triage?

2. How do patients use health care services following telephone based digital triage?

Objectives

The following objectives will be explored in urgent care settings both before and after the start of the Covid 19 pandemic;

1. To describe the characteristics and symptoms of patients accessing urgent care in urgent and emergency care settings.

2. To investigate associations between patient characteristics and triage disposition (triage advice generated through Odyssey).

3. To describe triage dispositions in primary and secondary triage in patients who are referred from NHS 111 for clinician triage.

4. To explore patients’ health outcomes and service use following telephone based digital triage, which will serve as indicators of appropriateness of triage and safety.

5. To draw conclusions and make recommendations of how telephone based digital triage is utilised and how it can be improved, in periods of ‘normal’ and high service use.

Care setting:

This study focuses on the use of Odyssey, a digital triage tool that has been developed by Advanced Health and Care Ltd, an NHS technology supplier (https://www.oneadvanced.com/). Clinician led triage using Odyssey in UK based urgent care settings will be explored in this study.

In this setting, a clinician uses Odyssey within an out of hours (OOH) or urgent care centre. In England, the patient will usually access the urgent care / OOH centre via NHS 111, in some cases patients may call the out-of-hours or urgent care centre directly. In either case, patients will be triaged by a clinician (usually a nurse) over the phone using Odyssey, which will result in prioritisation and advice. Advice generated by the tool may include: a face to face appointment at the centre, home visit, a routine GP appointment or self-care advice. Urgent care providers that utilise Odyssey in this way include: Mastercall (http://www.mastercall.org.uk/out-of-hours) and Practice Plus Group (PPG) https://practiceplusgroup.com/our-services/integrated-urgent-care/out-of-hours-services/.

This study ultimately aims to understand how clinician led digital triage is used in different settings and how this may be influenced by previous care advice through NHS 111 and the associated patient outcomes following triage.

The routine dataset being requested will enable several associations to be explored including the characteristics of repeat callers and access of services by particular subgroups. The study also enables the analysis of the impact of the COVID-19 pandemic on patient service use, which may provide insight on how services function in times of high service use and could feed into future planning and service delivery.

Use of data in the project:

Data is being requested for use in a PhD research project that investigates patient health and safety outcomes following the use of telephone based urgent care services. The project is led by the University of Warwick and is partly funded by a company called Advanced Health and Care Ltd (https://www.oneadvanced.com/). Advanced provide digital triage software to NHS telephone based care services; the triage software is used by clinicians to generate algorithm based care advice based on a patients symptoms. Advanced will be providing the patient cohort, which is made up of patients who have been digitally triaged by a clinician.

Patient outcomes data (relating to A&E attendance and hospital admissions) linked to a cohort are being requested in this agreement. The research project will analyse two non-identifiable (pseudonymised) datasets: 1) patient outcomes (A&E attendance/hospital admissions) together with 2) data from telephone triage services. The project aims to understand patient safety and health outcomes, following the use of telephone based urgent care services, in order to make recommendations in how these services can be best delivered and improved.

The agreement is for Hospital Episodes Statistics (HES) Admitted Patient Care (APC) and Emergency Care (ECDS) datasets for a given cohort; the cohort is made up of users of telephone triage services in England. NHS England linked the patient identifiers for the cohort (provided by Advanced Health and Care Ltd) to patient outcomes in HES APC and ECDS datasets. NHS England hen removed the identifiable fields (NHS Numbers / date of births) before securely sending to University of Warwick as pseudonymised datasets.

The legal bases are;

UK GDPR Article 6 (1)(f) as the data processing is necessary for a legitimate interest legal basis due to the commercial connection, this project investigates how digital triage and urgent care delivery can be improved to benefit patient care. A legitimate interests assessment has been completed by the University of Warwick.

UK GDPR Article 9(2)(j): as data processing is necessary for scientific research purpose.

There are minimal risks of harm; limited identifiable data will be handled by Advanced for data linkage purposes. Two identifiable data fields will be sent by Advanced to NHS England (DARS) for the data linkage: (NHS number and dates of births).

University of Warwick are the controller who also process data. University of Warwick are the controller because they determine the purposes and means of processing the data under this Agreement. Advanced Health and Care Ltd will not have access to this data. The University of Warwick has all the prerequisite security and technical requirements and is experienced in research.

How will the data requested achieve the aim identified?

The pseudonymised HES APC and ECDS data will be linked to a separate pseudonymised triage dataset before analysis is conducted. This separate ‘triage dataset’ will be sent by Advanced to University of Warwick. The dataset is pseudonymised: all patient identifiers have been removed, leaving only a study ID corresponding to the study ID that is provided by Advanced to NHS England in the study cohort. The HES APC/ ECDS data will be linked to the triage dataset using this study ID. All other fields in this ‘triage dataset’ are non-identifiable, fields include: time of call to out of hours service, presenting symptoms e.g. ‘chest pain’, outcome of digital triage, which is the advice the patient is given over the telephone (e.g. urgent face to face appointment within 6 hours / routine GP appointment / self care)

The research aims to understand the rates of A&E attendances and hospital admissions in different groups of patients who have been digitally triaged by telephone based urgent care providers. ‘Groups of patients’ will be based on: presenting symptom (e.g. patients presenting with chest pain), age group (e.g. older adults, aged over 65), and sex. These groups will be identified from the triage dataset.

Linking the two datasets will allow comparison of the digital triage advice that patients are given over the phone (for example, to self care or attend A&E immediately), with their actual service use (e.g. attendance at an Emergency department or hospitalisation). For example, it will be possible to identify if patients who are recommended to “self care” when triaged over the phone, go on to attend A&E or are hospitalised; and whether this differs in groups with different presenting symptoms. This could help to identify areas for improvement in the digital triage tool (specifically the clinical content), which could improve patient safety.

Length of research:

The research will be conducted by as part of a PhD project, which commenced in October 2019 and will finish in March 2024. The PhD project is funded by University of Warwick and Advanced.

Collaborations/wider setting:

The PhD project has been designed independently by the research team (PhD student and supervisory team) at the University of Warwick.

Advanced’s input is supporting with the provision of data for analysis in the research project. It is intended that the research findings will help to identify areas for improvement in the triage software, service delivery and to the international research community for the overall improvement in urgent care that is provided to patients.

The PhD project consists of a systematic review, to identify the areas of research focus; this is complete and has fed into the design of two parts of the project 1) The analysis of non-identifiable triage data and health outcomes, using data from 2019 – 2020 (as provided by Advanced) 2) An interview sub-study with service users, (which was planned for 2021). These two studies are independent.

The PHD student has become an employee of the University of Warwick in 2023.

Description of the data participants, including e.g. control and cohort groups:

The cohort is made up of patients/carers who have called and been triaged by a telephone based urgent care service, usually accessed via NHS 111, but in some cases patients may have called via the Covid-19 helpline (119) or have called the urgent care provider directly. This includes patients who were triaged over an 18 month period spanning 2019 – 2020. The patients have been triaged over the phone by a clinician using Odyssey which is a “digital triage tool” (a software based decision support tool, similar to ‘NHS Pathways’ used by NHS111).

Purpose of the request:

The research study investigates the use of telephone based digital triage and its impact on subsequent health care service use by patients and their health outcomes. This project is line with the Care Quality Commission’s(CQC) objectives for technology suppliers to work with researchers to better understand and improve patient outcomes relating to digital triage. There is no involvement of the CQC in this project.

Justification for the number of years requested:

Data spanning two years has been requested to allow for the analysis of data prior to and during the Covid-19 pandemic. The project has been designed with the input of a statistician and data over this time period was deemed to be sufficient for this research.

Justification for the geographical spread of the data requested:

The cohort that will be provided to NHS England is based on the callers to urgent care providers that use the Odyssey digital triage tool. These urgent care providers are based in England; this is not based on any particular geographical spread.

As this study seeks to analyse a large amount of non-identifiable “historical” data (cohort size 231,419) it is not possible to collect A&E attendance / admissions data in an alternative way.

Details of any efforts taken to minimise the data required

The minimum amount of data for the research purpose has been requested. Two years of data have been requested; only the specific fields that will be used in the analysis have been requested. The requested fields have been reviewed by the supervisory team, which includes a statistician. No identifiable data has been requested in this agreement. The data requested relate to patients use of A&E and information about hospital admission (if any). Data requested from the ECDS relates to the patients arrival time at A&E, their chief complaint or injury, co-morbidities, whether they were admitted, investigations and treatments, and discharge.The data fields requested for admissions (APC) relate to when and how they were admitted, the reason they were admitted (diagnosis codes), procedures that were done when admitted, and the length of admission.

The cohort consists of 231,419 patients who attended A&E following telephone triage between 01 April 2019 and 30 September 2020. The cohort cannot be minimised as the study team are keen to discover evidence of clinical risk in the digital triage tool, which may be relatively small in size. The team's initial analysis of the digital triage cohort data (which includes the referral advice that patients are given) indicates that fewer patients than anticipated are being advised to go to the ED following the digital assessment, which could mean that the system is working well or could mean that patients are being delayed access to ED. Therefore, the opportunity to include a larger cohort of patients is of value as it will lead to greater understanding of the safety of the digital triage process.

The exact cohort size was not known at the beginning of the application process to NHS England and this was reflected in the Confidentiality Advisory Group (CAG) application, in which the University of Warwick stated: “A minimum of 100,000 triage cases will be analysed. This has been deemed to provide an appropriate number of A&E attendances based on previous studies. The actual number may be higher than 100,000 depending on data availability from the services however it has been established that a minimum of 100,000 cases will be available and appropriate for analysis.”

The sample size of 100,000 was a very conservative estimate to demonstrate that the study had enough power to conduct the analyses and the study team always expected more. In addition the sample size will increase the precision of their estimates. The increased sample size will also allow the study team to understand how triage works for certain patient groups, e.g. ethnic minorities.

The project has been designed and will be conducted independently by University of Warwick. Advanced have not been involved in the design and are not determining the purpose and the means of the data processing. They will not have any access to the linked data generated by NHS England. The only role that Advanced has is to provide the research cohort and supporting the research by providing non-conditional funding through the Warwick Collaborative Postgraduate Research Scholarship (WCPRS) scheme.

Advanced are a partial funder of the project; University of Warwick are the other funder.

Findings/Outputs will be presented in a PhD thesis.

Additionally, recommendations from this project will be presented in an oral presentation and a short written report that can feed into the design of digital triage tools, service delivery and healthcare policy. This will be in plain English (non-scientific format), which will be shared with all project stakeholders, key audiences and on social media.

Findings of the study will be disseminated through the following:

1. Oral presentations to Advanced who develop and provide the Odyssey digital triage tool. Audiences within Advanced will include leadership, clinical and product management teams. There will be an emphasis on areas for improvement in terms of service delivery and the clinical content within the digital triage tool.

2. Findings will be shared with the urgent care providers whose data is being analysed in the study: A report and presentation of findings will be offered to participating urgent care providers, including suggestions for service delivery improvements. Examples of the care providers include: Mastercall (https://mastercall.org.uk/our-services/), Bardoc (http://bardoc.co.uk/)

It is expected that audiences will include those responsible for managing service delivery, clinical staff (digital triage end users), and those responsible for training and quality assurance of digital triage tools. Findings from the research will be summarised in a PowerPoint presentation, which will be presented to audiences including policymakers and commissioners responsible for urgent care services, including members of NHS England and through special interest groups, for example the Society for Academic Primary Care (SAPC) special interest group: Digital Technologies in Primary Care.

Findings from the research will be summarised (in lay English) in a PowerPoint presentation. A written summary of findings report will also be created. The oral presentation and study reports will be shared with collaborating patient and public groups (for example: Healthwatch or GP practice patient groups) and service users who participated in the study who have indicated their interest in receiving a study report.

Findings will be written into academic research papers, which will be disseminated in peer reviewed journals (such as BJGP, BMC journals and BMJ open) and will be presented at conferences focussed on digital health and primary care. There are three anticipated papers: the systematic review, quantitative findings and mixed methods findings.

All outputs will contain only data that is aggregated with small numbers suppressed in line with the HES Analysis Guide.

There will be no suppression of results or outputs by the funders. This means that if the results show anything negative towards Advanced and the Odyssey digital triage tool, that these finding will still be published.

Target date: November 2024

Analysis has started, but this is not yet applicable, as the team received the data in February 2023 which was later than expected.

Project summary:

Care services that provide out of hours care (care outside of general practice opening) often use digitally supported ‘telephone triage’. This telephone triage involves a health care service staff member using a ‘digital triage tool’, to help signpost the patient to the most appropriate service to receive health care; in part this helps health care services manage high demand. The digital triage tool is used by the clinician to input the patient’s symptoms in order to automatically generate the signposting or care advice. This research investigates how patients use these types of services. It also aims to understand how telephone triage affects patients’ use of other health care services, such as Accident and Emergency (A&E), and what happens to patients following telephone triage, in terms of their health. This study will analyse pseudonymised, pre-existing data from NHS providers in order to better understand how these telephone triage services are used and how they affect patient health. Ultimately the aim is to better understand how telephone triage is used and to identify areas for improvement that will benefit patients.

Research aims:

The two research aims below will be addressed before and after the start of the COVID-19 Pandemic;

1. How do patients use urgent care services that are delivered though telephone based digital triage?

2. How do patients use health care services following telephone based digital triage?

Objectives

The following objectives will be explored in urgent care settings both before and after the start of the Covid 19 pandemic;

1. To describe the characteristics and symptoms of patients accessing urgent care in urgent and emergency care settings.

2. To investigate associations between patient characteristics and triage disposition (triage advice generated through Odyssey).

3. To describe triage dispositions in primary and secondary triage in patients who are referred from NHS 111 for clinician triage.

4. To explore patients’ health outcomes and service use following telephone based digital triage, which will serve as indicators of appropriateness of triage and safety.

5. To draw conclusions and make recommendations of how telephone based digital triage is utilised and how it can be improved, in periods of ‘normal’ and high service use.

Care setting:

This study focuses on the use of Odyssey, a digital triage tool that has been developed by Advanced Health and Care Ltd, an NHS technology supplier (https://www.oneadvanced.com/). Clinician led triage using Odyssey in UK based urgent care settings will be explored in this study.

In this setting, a clinician uses Odyssey within an out of hours (OOH) or urgent care centre. In England, the patient will usually access the urgent care / OOH centre via NHS 111, in some cases patients may call the out-of-hours or urgent care centre directly. In either case, patients will be triaged by a clinician (usually a nurse) over the phone using Odyssey, which will result in prioritisation and advice. Advice generated by the tool may include: a face to face appointment at the centre, home visit, a routine GP appointment or self-care advice. Urgent care providers that utilise Odyssey in this way include: Mastercall (http://www.mastercall.org.uk/out-of-hours) and Practice Plus Group (PPG) https://practiceplusgroup.com/our-services/integrated-urgent-care/out-of-hours-services/.

This study ultimately aims to understand how clinician led digital triage is used in different settings and how this may be influenced by previous care advice through NHS 111 and the associated patient outcomes following triage.

The routine dataset being requested will enable several associations to be explored including the characteristics of repeat callers and access of services by particular subgroups. The study also enables the analysis of the impact of the COVID-19 pandemic on patient service use, which may provide insight on how services function in times of high service use and could feed into future planning and service delivery.

Use of data in the project:

Data is being requested for use in a PhD research project that investigates patient health and safety outcomes following the use of telephone based urgent care services. The project is led by the University of Warwick and is partly funded by a company called Advanced Health and Care ltd (https://www.oneadvanced.com/). Advanced provide digital triage software to NHS telephone based care services; the triage software is used by clinicians to generate algorithm based care advice based on a patients symptoms. Advanced will be providing the patient cohort, which is made up of patients who have been digitally triaged by a clinician.

Patient outcomes data (relating to A&E attendance and hospital admissions) linked to a cohort are being requested in this agreement. The research project will analyse two non-identifiable (pseudonymised) datasets: 1) patient outcomes (A&E attendance/hospital admissions) together with 2) data from telephone triage services. The project aims to understand patient safety and health outcomes, following the use of telephone based urgent care services, in order to make recommendations in how these services can be best delivered and improved.

The application is for Hospital Episodes Statistics (HES) Admitted Patient Care (APC) and Emergency Care (ECDS) datasets for a given cohort; the cohort is made up of recent users of telephone triage services in England. NHS England will link the patient identifiers for the cohort (provided by Advanced Health and Care) to patient outcomes in HES APC and ECDS datasets. NHS England will then remove the identifiable fields (NHS Numbers / date of births) before securely sending to University of Warwick as pseudonymised datasets.

The legal bases are;

UK GDPR Article 6 (1)(f) as the data processing is necessary for a legitimate interest legal basis due to the commercial connection, this project investigates how digital triage and urgent care delivery can be improved to benefit patient care.

UK GDPR Article 9(2)(j): as data processing is necessary for scientific research purpose.

There are minimal risks of harm; limited identifiable data will be handled by Advanced for data linkage purposes. Two identifiable data fields will be sent by Advanced to NHS Digital (DARS) for the data linkage: (NHS number and dates of births).

University of Warwick are the data controller who also process data, and are the storage location. University of Warwick are the data controller because they are determining how the data is analysed and have designed the research. Advanced Health and Care Ltd will not have access to the disseminated data. The University of Warwick has all the prerequisite security and technical requirements and is experienced in research.

How will the data requested achieve the aim identified?

The pseudonymised HES APC and ECDS data will be linked to a separate pseudonymised triage dataset before analysis is conducted. This separate ‘triage dataset’ will be sent by Advanced to University of Warwick. The dataset is pseudonymised: all patient identifiers have been removed, leaving only a study ID corresponding to the study ID that is provided by Advanced to NHS Digital in the study cohort. The HES APC/ ECDS data will be linked to the triage dataset using this study ID. All other fields in this ‘triage dataset’ are non-identifiable, fields include: time of call to out of hours service, presenting symptoms e.g. ‘chest pain’, outcome of digital triage, which is the advice the patient is given over the telephone (e.g. urgent face to face appointment within 6 hours / routine GP appointment / self care)

The research aims to understand the rates of A&E attendances and hospital admissions in different groups of patients who have been digitally triaged by telephone based urgent care providers. ‘Groups of patients’ will be based on: presenting symptom (e.g. patients presenting with chest pain), age group (e.g. older adults, aged over 65), and sex. These groups will be identified from the triage dataset.

Linking the two datasets will allow comparison of the digital triage advice that patients are given over the phone (for example, to self care or attend A&E immediately), with their actual service use (e.g. attendance at an Emergency department or hospitalisation). For example, it will be possible to identify if patients who are recommended to “self care” when triaged over the phone, go on to attend A&E or are hospitalised; and whether this differs in groups with different presenting symptoms. This could help to identify areas for improvement in the digital triage tool (specifically the clinical content), which could improve patient safety.

Length of research:

The research will be conducted by as part of a PhD project, which commenced in October 2019 and will finish in March 2023. The PhD project is funded by University of Warwick and Advanced.

Collaborations/wider setting:

The PhD project has been designed independently by the research team (PhD student and supervisory team) at the University of Warwick. Advanced’s input is supporting with the provision of data for analysis in the research project. It is intended that the research findings will help to identify areas for improvement in the triage software, service delivery and to the international research community for the overall improvement in urgent care that is provided to patients.

The PhD project consists of a systematic review, to identify the areas of research focus; this is complete and has fed into the design of two parts of the project 1) The analysis of non-identifiable triage data and health outcomes, using data from 2019 – 2020 (as provided by Advanced) 2) An interview sub-study with service users, which are currently being planned and will take place in 2021. These two studies are independent.

Description of the data participants, including e.g. control and cohort groups:

The cohort is made up of patients/carers who have called and been triaged by a telephone based urgent care service, usually accessed via NHS 111, but in some cases patients may have called via the Covid-19 helpline (119) or have called the urgent care provider directly. This includes patients who were triaged over an 18 month period spanning 2019 – 2020. The patients have been triaged over the phone by a clinician using Odyssey which is a “digital triage tool” (a software based decision support tool, similar to ‘NHS Pathways’ used by NHS111).

Purpose of the request:

The research study investigates the use of telephone based digital triage and its impact on subsequent health care service use by patients and their health outcomes. This project is line with the Care Quality Commission’s(CQC) objectives for technology suppliers to work with researchers to better understand and improve patient outcomes relating to digital triage. There is no involvement of the CQC in this project.

Justification for the number of years requested:

Data spanning two years has been requested to allow for the analysis of data prior to and during the Covid-19 pandemic. The project has been designed with the input of a statistician and data over this time period data was deemed to be sufficient for this research.

Justification for the geographical spread of the data requested:

The cohort that will be provided to NHS Digital is based on the callers to urgent care providers that use the Odyssey digital triage tool. These urgent care providers are based in England; this is not based on any particular geographical spread.

As this study seeks to analyse a large amount of non-identifiable “historical” data (cohort size 231,419) it is not possible to collect A&E attendance / admissions data in an alternative way.

Details of any efforts taken to minimise the data required

The minimum amount of data for the research purpose has been requested. Two years of data have been requested; only the specific fields that will be used in the analysis have been requested. The requested fields have been reviewed by the supervisory team, which includes a statistician. No identifiable data has been requested in this agreement. The data requested relate to patients use of A&E and information about hospital admission (if any). Data requested from the ECDS relates to the patients arrival time at A&E, their chief complaint or injury, co-morbidities, whether they were admitted, investigations and treatments, and discharge.The data fields requested for admissions (APC) relate to when and how they were admitted, the reason they were admitted (diagnosis codes), procedures that were done when admitted, and the length of admission.

The cohort consists of patrients who attended A&E following telephone triage between 01 April 2019 and 30 September 2021. The cohort cannot be minimised further as the research team aim to discover evidence of clinical risk in the digital triage tool, which may be relatively small in size. Their initial analysis of the digital triage cohort data (which includes the referral advice that patients are given) indicates that fewer patients than anticipated are being advised to go to the ED following the digital assessment, which could mean that the system is working well or could mean that patients are being delayed access to ED. Therefore, the opportunity to include a larger cohort of patients is of value as it will lead to greater understanding of the safety of the digital triage process.

The exact cohort size was not known at the beginning of the agreement, and this was reflected in the CAG application, in which the University of Warwick stated: “A minimum of 100,000 triage cases will be analysed. This has been deemed to provide an appropriate number of A&E attendances based on previous studies. The actual number may be higher than 100,000 depending on data availability from the services however it has been established that a minimum of 100,000 cases will be available and appropriate for analysis.”

The sample size of 100,000 was a very conservative estimate to demonstrate that the study had enough power to conduct the analyses and the study team always expected more. In addition the sample size will increase the precision of their estimates. The increased sample size will also allow the team to understand how triage works for certain patient groups, e.g. ethnic minorities.

Organisations involved

University of Warwick is the sole data controller who also process the data. University of Warwick will conduct the research, using non-identifiable data only.

The project has been designed and will be conducted independently by University of Warwick. Advanced have not been involved in the design and are not determining the purpose and the means of the data processing. They will not have any access to the linked data generated by NHS Digital. The only role that Advanced has is to provide the research cohort and supporting the research by providing non-conditional funding through the Warwick Collaborative Postgraduate Research Scholarship (WCPRS) scheme.

Advanced are a partial funder of the project; University of Warwick are the other funder.

Findings/Outputs will be presented in a PhD thesis.

Additionally, recommendations from this project will be presented in an oral presentation and a short written report that can feed into the design of digital triage tools, service delivery and healthcare policy. This will be in plain English (non-scientific format), which will be shared with all project stakeholders, key audiences and on social media.

Findings of the study will be disseminated through the following:

1. Oral presentations to Advanced who develop and provide the Odyssey digital triage tool. Audiences within Advanced will include leadership, clinical and product management teams. There will be an emphasis on areas for improvement in terms of service delivery and the clinical content within the digital triage tool.

2. Findings will be shared with the urgent care providers whose data is being analysed in the study: A report and presentation of findings will be offered to participating urgent care providers, including suggestions for service delivery improvements. Examples of the care providers include: Mastercall (http://www.mastercall.org.uk/services/out-of-hours-service), Bardoc (http://bardoc.co.uk/)

It is expected that audiences will include those responsible for managing service delivery, clinical staff (digital triage end users), and those responsible for training and quality assurance of digital triage tools. Findings from the research will be summarised in a PowerPoint presentation, which will be presented to audiences including policymakers and commissioners responsible for urgent care services, including members of NHS England and through special interest groups, for example the Society for Academic Primary Care (SAPC) special interest group: Digital Technologies in Primary Care.

Findings from the research will be summarised (in lay English) in a PowerPoint presentation. A written summary of findings report will also be created. The oral presentation and study reports will be shared with collaborating patient and public groups (for example: Healthwatch or GP practice patient groups) and service users who participated in the study who have indicated their interest in receiving a study report.

Findings will be written into academic research papers, which will be disseminated in peer reviewed journals (such as BJGP, BMC journals and BMJ open) and will be presented at conferences focussed on digital health and primary care. There are three anticipated papers: the systematic review, quantitative findings and mixed methods findings.

All outputs will contain only data that is aggregated with small numbers suppressed in line with the HES Analysis Guide.

There will be no suppression of results or outputs by the funders. This means that if the results show anything negative towards Advanced and the Odyssey digital triage tool, that these finding will still be published.

Outputs are expected to be complete by March 2023.

Analysis has started, but this is not yet applicable, as the team received the data in February 2023 which was later than expected.

Project summary:

Care services that provide out of hours care (care outside of general practice opening) often use digitally supported ‘telephone triage’. This telephone triage involves a health care service staff member using a ‘digital triage tool’, to help signpost the patient to the most appropriate service to receive health care; in part this helps health care services manage high demand. The digital triage tool is used by the clinician to input the patient’s symptoms in order to automatically generate the signposting or care advice. This research investigates how patients use these types of services. It also aims to understand how telephone triage affects patients’ use of other health care services, such as Accident and Emergency (A&E), and what happens to patients following telephone triage, in terms of their health. This study will analyse pseudonymised, pre-existing data from NHS providers in order to better understand how these telephone triage services are used and how they affect patient health. Ultimately the aim is to better understand how telephone triage is used and to identify areas for improvement that will benefit patients.

Research aims:

The two research aims below will be addressed before and after the start of the COVID-19 Pandemic;

1. How do patients use urgent care services that are delivered though telephone based digital triage?

2. How do patients use health care services following telephone based digital triage?

Objectives

The following objectives will be explored in urgent care settings both before and after the start of the Covid 19 pandemic;

1. To describe the characteristics and symptoms of patients accessing urgent care in urgent and emergency care settings.

2. To investigate associations between patient characteristics and triage disposition (triage advice generated through Odyssey).

3. To describe triage dispositions in primary and secondary triage in patients who are referred from NHS 111 for clinician triage.

4. To explore patients’ health outcomes and service use following telephone based digital triage, which will serve as indicators of appropriateness of triage and safety.

5. To draw conclusions and make recommendations of how telephone based digital triage is utilised and how it can be improved, in periods of ‘normal’ and high service use.

Care setting:

This study focuses on the use of Odyssey, a digital triage tool that has been developed by Advanced Health and Care Ltd, an NHS technology supplier (https://www.oneadvanced.com/). Clinician led triage using Odyssey in UK based urgent care settings will be explored in this study.

In this setting, a clinician uses Odyssey within an out of hours (OOH) or urgent care centre. In England, the patient will usually access the urgent care / OOH centre via NHS 111, in some cases patients may call the out-of-hours or urgent care centre directly. In either case, patients will be triaged by a clinician (usually a nurse) over the phone using Odyssey, which will result in prioritisation and advice. Advice generated by the tool may include: a face to face appointment at the centre, home visit, a routine GP appointment or self-care advice. Urgent care providers that utilise Odyssey in this way include: Mastercall (http://www.mastercall.org.uk/out-of-hours) and Practice Plus Group (PPG) https://practiceplusgroup.com/our-services/integrated-urgent-care/out-of-hours-services/.

This study ultimately aims to understand how clinician led digital triage is used in different settings and how this may be influenced by previous care advice through NHS 111 and the associated patient outcomes following triage.

The routine dataset being requested will enable several associations to be explored including the characteristics of repeat callers and access of services by particular subgroups. The study also enables the analysis of the impact of the COVID-19 pandemic on patient service use, which may provide insight on how services function in times of high service use and could feed into future planning and service delivery.

Use of data in the project:

Data is being requested for use in a PhD research project that investigates patient health and safety outcomes following the use of telephone based urgent care services. The project is led by the University of Warwick and is partly funded by a company called Advanced Health and Care ltd (https://www.oneadvanced.com/). Advanced provide digital triage software to NHS telephone based care services; the triage software is used by clinicians to generate algorithm based care advice based on a patients symptoms. Advanced will be providing the patient cohort, which is made up of patients who have been digitally triaged by a clinician.

Patient outcomes data (relating to A&E attendance and hospital admissions) linked to a cohort are being requested in this application. The research project will analyse two non-identifiable (pseudonymised) datasets: 1) patient outcomes (A&E attendance/hospital admissions) together with 2) data from telephone triage services. The project aims to understand patient safety and health outcomes, following the use of telephone based urgent care services, in order to make recommendations in how these services can be best delivered and improved.

The application is for Hospital Episodes Statistics (HES) Admitted Patient Care (APC) and Emergency Care (ECDS) datasets for a given cohort; the cohort is made up of recent users of telephone triage services in England. NHS Digital will link the patient identifiers for the cohort (provided by Advanced Health and Care) to patient outcomes in HES APC and ECDS datasets. NHS Digital will then remove the identifiable fields (NHS Numbers / data of births) before securely sending to University of Warwick as pseudonymised datasets.

The legal bases are;

GDPR article 6 (1)(f) as the data processing is necessary for a legitimate interest legal basis due to the commercial connection, this project investigates how digital triage and urgent care delivery can be improved to benefit patient care.

GDPR article 9(2)(j): as data processing is necessary for scientific research purpose.

There are minimal risks of harm; limited identifiable data will be handled by Advanced for data linkage purposes. Two identifiable data fields will be sent by Advanced to NHS Digital (DARS) for the data linkage: (NHS number and dates of births).

University of Warwick are the data controller who also process data, and are the storage location. University of Warwick are the data controller because they are conducting the research study by analysing the pseudonymised HES APC and ECDS data provided by NHS Digital. Advanced Health and Care Ltd will not have access to this data. The University of Warwick has all the prerequisite security and technical requirements and is experienced in research.

How will the data requested achieve the aim identified?

The pseudonymised HES APC and ECDS data will be linked to a separate pseudonymised triage dataset before analysis is conducted. This separate ‘triage dataset’ will be sent by Advanced to University of Warwick. The dataset is pseudonymised: all patient identifiers have been removed, leaving only a study ID corresponding to the study ID that is provided by Advanced to NHS Digital in the study cohort. The HES APC/ ECDS data will be linked to the triage dataset using this study ID. All other fields in this ‘triage dataset’ are non-identifiable, fields include: time of call to out of hours service, presenting symptoms e.g. ‘chest pain’, outcome of digital triage, which is the advice the patient is given over the telephone (e.g. urgent face to face appointment within 6 hours / routine GP appointment / self care)

The research aims to understand the rates of A&E attendances and hospital admissions in different groups of patients who have been digitally triaged by telephone based urgent care providers. ‘Groups of patients’ will be based on: presenting symptom (e.g. patients presenting with chest pain), age group (e.g. older adults, aged over 65), and sex. These groups will be identified from the triage dataset.

Linking the two datasets will allow comparison of the digital triage advice that patients are given over the phone (for example, to self care or attend A&E immediately), with their actual service use (e.g. attendance at an Emergency department or hospitalisation). For example, it will be possible to identify if patients who are recommended to “self care” when triaged over the phone, go on to attend A&E or are hospitalised; and whether this differs in groups with different presenting symptoms. This could help to identify areas for improvement in the digital triage tool (specifically the clinical content), which could improve patient safety.

Length of research:

The research will be conducted by as part of a PhD project, which commenced in October 2019 and will finish in March 2023. The PhD project is funded by University of Warwick and Advanced.

Collaborations/wider setting:

The PhD project has been designed independently by the research team (PhD student and supervisory team) at the University of Warwick. Advanced’s input is supporting with the provision of data for analysis in the research project. It is intended that the research findings will help to identify areas for improvement in the triage software, service delivery and to the international research community for the overall improvement in urgent care that is provided to patients.

The PhD project consists of a systematic review, to identify the areas of research focus; this is complete and has fed into the design of two parts of the project 1) The analysis of non-identifiable triage data and health outcomes, using data from 2019 – 2020 (as provided by Advanced) 2) An interview sub-study with service users, which are currently being planned and will take place in 2021. These two studies are independent.

Description of the data participants, including e.g. control and cohort groups:

The cohort is made up of patients/carers who have called and been triaged by a telephone based urgent care service, usually accessed via NHS 111, but in some cases patients may have called via the Covid-19 helpline (119) or have called the urgent care provider directly. This includes patients who were triaged over an 18 month period spanning 2019 – 2020. The patients have been triaged over the phone by a clinician using Odyssey which is a “digital triage tool” (a software based decision support tool, similar to ‘NHS Pathways’ used by NHS111).

Purpose of the request:

The research study investigates the use of telephone based digital triage and its impact on subsequent health care service use by patients and their health outcomes. This project is line with the Care Quality Commission’s(CQC) objectives for technology suppliers to work with researchers to better understand and improve patient outcomes relating to digital triage. There is no involvement of the CQC in this project.

Data that is required:

Data showing patients' use of A&E and hospital admissions is required for the project. The A&E / admissions data will be analysed in pseudonymised form together with a pseudonymised triage dataset, in order to investigate the relationships between triage outcomes (e.g. triage advice given to a patient) and patients compliance with advice, use of services and health outcomes.

Two datasets are being requested through DARS:

1. Admitted patient care (APC)

2. Emergency Care Dataset (ECDS)

Advanced will provide a cohort (including: pseudonymisation ID, NHS number and date of birth) for linking with the APC and ECDS data. The requested data will be linked and sent by DARS to University of Warwick with the NHS Numbers and date of births removed. The University of Warwick will keep this data for the duration of the DSA with NHS Digital and may request an extension if necessary.

Justification for the number of years requested:

Data spanning two years has been requested to allow for the analysis of data prior to and during the Covid-19 pandemic. The project has been designed with the input of a statistician and data over this time period data was deemed to be sufficient for this research.

Justification for the geographical spread of the data requested:

The cohort that will be provided to NHS Digital is based on the callers to urgent care providers that use the Odyssey digital triage tool. These urgent care providers are based in England; this is not based on any particular geographical spread.

As this study seeks to analyse a large amount of non-identifiable “historical” data (cohort size 231,419) it is not possible to collect A&E attendance / admissions data in an alternative way.

Details of any efforts taken to minimise the data required

The minimum amount of data for the research purpose has been requested. Two years of data have been requested; only the specific fields that will be used in the analysis have been requested. The requested fields have been reviewed by the supervisory team, which includes a statistician. No identifiable data has been requested in this application. The data requested relate to patients use of A&E and information about hospital admission (if any). Data requested from the ECDS relates to the patients arrival time at A&E, their chief complaint or injury, co-morbidities, whether they were admitted, investigations and treatments, and discharge.The data fields requested for admissions (APC) relate to when and how they were admitted, the reason they were admitted (diagnosis codes), procedures that were done when admitted, and the length of admission.

The cohort cannot be minimised as they are keen to discover evidence of clinical risk in the digital triage tool, which may be relatively small in size. Their initial analysis of the digital triage cohort data (which includes the referral advice that patients are given) indicates that fewer patients than anticipated are being advised to go to the ED following the digital assessment, which could mean that the system is working well or could mean that patients are being delayed access to ED. Therefore, the opportunity to include a larger cohort of patients is of value as it will lead to greater understanding of the safety of the digital triage process.

The exact cohort size was not known at the beginning of the application, and this was reflected in the CAG application, in which we stated: “A minimum of 100,000 triage cases will be analysed. This has been deemed to provide an appropriate number of A&E attendances based on previous studies. The actual number may be higher than 100,000 depending on data availability from the services however it has been established that a minimum of 100,000 cases will be available and appropriate for analysis.”

The sample size of 100,000 was a very conservative estimate to demonstrate that they did have enough power to conduct their analyses and they always expected more. In addition the sample size will increase the precision of their estimates. The increased sample size will also allow them to understand how triage works for certain patient groups, e.g. ethnic minorities.

Organisations involved

University of Warwick is the sole data controller who also process the data. University of Warwick will conduct the research, using non-identifiable data only.

The project has been designed and will be conducted independently by University of Warwick. Advanced have not been involved in the design and are not determining the purpose and the means of the data processing. They will not have any access to the linked data generated by NHS Digital. The only role that Advanced has is to provide the research cohort and supporting the research by providing non-conditional funding through the Warwick Collaborative Postgraduate Research Scholarship (WCPRS) scheme.

Advanced are a partial funder of the project; University of Warwick are the other funder.

Findings/Outputs will be presented in a PhD thesis.

Additionally, recommendations from this project will be presented in an oral presentation and a short written report that can feed into the design of digital triage tools, service delivery and healthcare policy. This will be in plain English (non-scientific format), which will be shared with all project stakeholders, key audiences and on social media.

Findings of the study will be disseminated through the following:

1. Oral presentations to Advanced who develop and provide the Odyssey digital triage tool. Audiences within Advanced will include leadership, clinical and product management teams. There will be an emphasis on areas for improvement in terms of service delivery and the clinical content within the digital triage tool.

2. Findings will be shared with the urgent care providers whose data is being analysed in the study: A report and presentation of findings will be offered to participating urgent care providers, including suggestions for service delivery improvements. Examples of the care providers include: Mastercall (http://www.mastercall.org.uk/services/out-of-hours-service), Bardoc (http://bardoc.co.uk/)

It is expected that audiences will include those responsible for managing service delivery, clinical staff (digital triage end users), and those responsible for training and quality assurance of digital triage tools. Findings from the research will be summarised in a PowerPoint presentation, which will be presented to audiences including policymakers and commissioners responsible for urgent care services, including members of NHS England and through special interest groups, for example the Society for Academic Primary Care (SAPC) special interest group: Digital Technologies in Primary Care.

Findings from the research will be summarised (in lay English) in a PowerPoint presentation. A written summary of findings report will also be created. The oral presentation and study reports will be shared with collaborating patient and public groups (for example: Healthwatch or GP practice patient groups) and service users who participated in the study who have indicated their interest in receiving a study report.

Findings will be written into academic research papers, which will be disseminated in peer reviewed journals (such as BJGP, BMC journals and BMJ open) and will be presented at conferences focussed on digital health and primary care. There are three anticipated papers: the systematic review, quantitative findings and mixed methods findings.

All outputs will contain only data that is aggregated with small numbers suppressed in line with the HES Analysis Guide.

There will be no suppression of results or outputs by the funders. This means that if the results show anything negative towards Advanced and the Odyssey digital triage tool, that these finding will still be published.

Outputs are expected to be complete by March 2023.

Project summary:

Care services that provide out of hours care (care outside of general practice opening) often use digitally supported ‘telephone triage’. This telephone triage involves a health care service staff member using a ‘digital triage tool’, to help signpost the patient to the most appropriate service to receive health care; in part this helps health care services manage high demand. The digital triage tool is used by the clinician to input the patient’s symptoms in order to automatically generate the signposting or care advice. This research investigates how patients use these types of services. It also aims to understand how telephone triage affects patients’ use of other health care services, such as Accident and Emergency (A&E), and what happens to patients following telephone triage, in terms of their health. This study will analyse pseudonymised, pre-existing data from NHS providers in order to better understand how these telephone triage services are used and how they affect patient health. Ultimately the aim is to better understand how telephone triage is used and to identify areas for improvement that will benefit patients.

Research aims:

The two research aims below will be addressed before and after the start of the COVID-19 Pandemic;

1. How do patients use urgent care services that are delivered though telephone based digital triage?

2. How do patients use health care services following telephone based digital triage?

Objectives

The following objectives will be explored in urgent care settings both before and after the start of the Covid 19 pandemic;

1. To describe the characteristics and symptoms of patients accessing urgent care in urgent and emergency care settings.

2. To investigate associations between patient characteristics and triage disposition (triage advice generated through Odyssey).

3. To describe triage dispositions in primary and secondary triage in patients who are referred from NHS 111 for clinician triage.

4. To explore patients’ health outcomes and service use following telephone based digital triage, which will serve as indicators of appropriateness of triage and safety.

5. To draw conclusions and make recommendations of how telephone based digital triage is utilised and how it can be improved, in periods of ‘normal’ and high service use.

Care setting:

This study focuses on the use of Odyssey, a digital triage tool that has been developed by Advanced Health and Care Ltd, an NHS technology supplier (https://www.oneadvanced.com/). Clinician led triage using Odyssey in UK based urgent care settings will be explored in this study.

In this setting, a clinician uses Odyssey within an out of hours (OOH) or urgent care centre. In England, the patient will usually access the urgent care / OOH centre via NHS 111, in some cases patients may call the out-of-hours or urgent care centre directly. In either case, patients will be triaged by a clinician (usually a nurse) over the phone using Odyssey, which will result in prioritisation and advice. Advice generated by the tool may include: a face to face appointment at the centre, home visit, a routine GP appointment or self-care advice. Urgent care providers that utilise Odyssey in this way include: Mastercall (http://www.mastercall.org.uk/out-of-hours) and Practice Plus Group (PPG) https://practiceplusgroup.com/our-services/integrated-urgent-care/out-of-hours-services/.

This study ultimately aims to understand how clinician led digital triage is used in different settings and how this may be influenced by previous care advice through NHS 111 and the associated patient outcomes following triage.

The routine dataset being requested will enable several associations to be explored including the characteristics of repeat callers and access of services by particular subgroups. The study also enables the analysis of the impact of the COVID-19 pandemic on patient service use, which may provide insight on how services function in times of high service use and could feed into future planning and service delivery.

Use of data in the project:

Data is being requested for use in a PhD research project that investigates patient health and safety outcomes following the use of telephone based urgent care services. The project is led by the University of Warwick and is partly funded by a company called Advanced Health and Care ltd (https://www.oneadvanced.com/). Advanced provide digital triage software to NHS telephone based care services; the triage software is used by clinicians to generate algorithm based care advice based on a patients symptoms. Advanced will be providing the patient cohort, which is made up of patients who have been digitally triaged by a clinician.

Patient outcomes data (relating to A&E attendance and hospital admissions) linked to a cohort are being requested in this application. The research project will analyse two non-identifiable (pseudonymised) datasets: 1) patient outcomes (A&E attendance/hospital admissions) together with 2) data from telephone triage services. The project aims to understand patient safety and health outcomes, following the use of telephone based urgent care services, in order to make recommendations in how these services can be best delivered and improved.

The application is for Hospital Episodes Statistics (HES) Admitted Patient Care (APC) and Emergency Care (ECDS) datasets for a given cohort; the cohort is made up of recent users of telephone triage services in England. NHS Digital will link the patient identifiers for the cohort (provided by Advanced Health and Care) to patient outcomes in HES APC and ECDS datasets. NHS Digital will then remove the identifiable fields (NHS Numbers / data of births) before securely sending to University of Warwick as pseudonymised datasets.

The legal bases are;

GDPR article 6 (1)(f) as the data processing is necessary for a legitimate interest legal basis due to the commercial connection, this project investigates how digital triage and urgent care delivery can be improved to benefit patient care.

GDPR article 9(2)(j): as data processing is necessary for scientific research purpose.

There are minimal risks of harm; limited identifiable data will be handled by Advanced for data linkage purposes. Two identifiable data fields will be sent by Advanced to NHS Digital (DARS) for the data linkage: (NHS number and dates of births).

University of Warwick are the data controller who also process data, and are the storage location. University of Warwick are the data controller because they are conducting the research study by analysing the pseudonymised HES APC and ECDS data provided by NHS Digital. Advanced Health and Care Ltd will not have access to this data. The University of Warwick has all the prerequisite security and technical requirements and is experienced in research.

How will the data requested achieve the aim identified?

The pseudonymised HES APC and ECDS data will be linked to a separate pseudonymised triage dataset before analysis is conducted. This separate ‘triage dataset’ will be sent by Advanced to University of Warwick. The dataset is pseudonymised: all patient identifiers have been removed, leaving only a study ID corresponding to the study ID that is provided by Advanced to NHS Digital in the study cohort. The HES APC/ ECDS data will be linked to the triage dataset using this study ID. All other fields in this ‘triage dataset’ are non-identifiable, fields include: time of call to out of hours service, presenting symptoms e.g. ‘chest pain’, outcome of digital triage, which is the advice the patient is given over the telephone (e.g. urgent face to face appointment within 6 hours / routine GP appointment / self care)

The research aims to understand the rates of A&E attendances and hospital admissions in different groups of patients who have been digitally triaged by telephone based urgent care providers. ‘Groups of patients’ will be based on: presenting symptom (e.g. patients presenting with chest pain), age group (e.g. older adults, aged over 65), and sex. These groups will be identified from the triage dataset.

Linking the two datasets will allow comparison of the digital triage advice that patients are given over the phone (for example, to self care or attend A&E immediately), with their actual service use (e.g. attendance at an Emergency department or hospitalisation). For example, it will be possible to identify if patients who are recommended to “self care” when triaged over the phone, go on to attend A&E or are hospitalised; and whether this differs in groups with different presenting symptoms. This could help to identify areas for improvement in the digital triage tool (specifically the clinical content), which could improve patient safety.

Length of research:

The research will be conducted by as part of a PhD project, which commenced in October 2019 and will finish in March 2023. The PhD project is funded by University of Warwick and Advanced.

Collaborations/wider setting:

The PhD project has been designed independently by the research team (PhD student and supervisory team) at the University of Warwick. Advanced’s input is supporting with the provision of data for analysis in the research project. It is intended that the research findings will help to identify areas for improvement in the triage software, service delivery and to the international research community for the overall improvement in urgent care that is provided to patients.

The PhD project consists of a systematic review, to identify the areas of research focus; this is complete and has fed into the design of two parts of the project 1) The analysis of non-identifiable triage data and health outcomes, using data from 2019 – 2020 (as provided by Advanced) 2) An interview sub-study with service users, which are currently being planned and will take place in 2021. These two studies are independent.

Description of the data participants, including e.g. control and cohort groups:

The cohort is made up of patients/carers who have called and been triaged by a telephone based urgent care service, usually accessed via NHS 111, but in some cases patients may have called via the Covid-19 helpline (119) or have called the urgent care provider directly. This includes patients who were triaged over an 18 month period spanning 2019 – 2020. The patients have been triaged over the phone by a clinician using Odyssey which is a “digital triage tool” (a software based decision support tool, similar to ‘NHS Pathways’ used by NHS111).

Purpose of the request:

The research study investigates the use of telephone based digital triage and its impact on subsequent health care service use by patients and their health outcomes. This project is line with the Care Quality Commission’s(CQC) objectives for technology suppliers to work with researchers to better understand and improve patient outcomes relating to digital triage. There is no involvement of the CQC in this project.

Data that is required:

Data showing patients' use of A&E and hospital admissions is required for the project. The A&E / admissions data will be analysed in pseudonymised form together with a pseudonymised triage dataset, in order to investigate the relationships between triage outcomes (e.g. triage advice given to a patient) and patients compliance with advice, use of services and health outcomes.

Two datasets are being requested through DARS:

1. Admitted patient care (APC)

2. Emergency Care Dataset (ECDS)

Advanced will provide a cohort (including: pseudonymisation ID, NHS number and date of birth) for linking with the APC and ECDS data. The requested data will be linked and sent by DARS to University of Warwick with the NHS Numbers and date of births removed. The University of Warwick will keep this data for the duration of the DSA with NHS Digital and may request an extension if necessary.

Justification for the number of years requested:

Data spanning two years has been requested to allow for the analysis of data prior to and during the Covid-19 pandemic. The project has been designed with the input of a statistician and data over this time period data was deemed to be sufficient for this research.

Justification for the geographical spread of the data requested:

The cohort that will be provided to NHS Digital is based on the callers to urgent care providers that use the Odyssey digital triage tool. These urgent care providers are based in England; this is not based on any particular geographical spread.

As this study seeks to analyse a large amount of non-identifiable “historical” data (cohort size 231,419) it is not possible to collect A&E attendance / admissions data in an alternative way.

Details of any efforts taken to minimise the data required

The minimum amount of data for the research purpose has been requested. Two years of data have been requested; only the specific fields that will be used in the analysis have been requested. The requested fields have been reviewed by the supervisory team, which includes a statistician. No identifiable data has been requested in this application. The data requested relate to patients use of A&E and information about hospital admission (if any). Data requested from the ECDS relates to the patients arrival time at A&E, their chief complaint or injury, co-morbidities, whether they were admitted, investigations and treatments, and discharge.The data fields requested for admissions (APC) relate to when and how they were admitted, the reason they were admitted (diagnosis codes), procedures that were done when admitted, and the length of admission.

The cohort cannot be minimised as they are keen to discover evidence of clinical risk in the digital triage tool, which may be relatively small in size. Their initial analysis of the digital triage cohort data (which includes the referral advice that patients are given) indicates that fewer patients than anticipated are being advised to go to the ED following the digital assessment, which could mean that the system is working well or could mean that patients are being delayed access to ED. Therefore, the opportunity to include a larger cohort of patients is of value as it will lead to greater understanding of the safety of the digital triage process.

The exact cohort size was not known at the beginning of the application, and this was reflected in the CAG application, in which we stated: “A minimum of 100,000 triage cases will be analysed. This has been deemed to provide an appropriate number of A&E attendances based on previous studies. The actual number may be higher than 100,000 depending on data availability from the services however it has been established that a minimum of 100,000 cases will be available and appropriate for analysis.”

The sample size of 100,000 was a very conservative estimate to demonstrate that they did have enough power to conduct their analyses and they always expected more. In addition the sample size will increase the precision of their estimates. The increased sample size will also allow them to understand how triage works for certain patient groups, e.g. ethnic minorities.

Organisations involved

University of Warwick is the sole data controller who also process the data. University of Warwick will conduct the research, using non-identifiable data only.

The project has been designed and will be conducted independently by University of Warwick. Advanced have not been involved in the design and are not determining the purpose and the means of the data processing. They will not have any access to the linked data generated by NHS Digital. The only role that Advanced has is to provide the research cohort and supporting the research by providing non-conditional funding through the Warwick Collaborative Postgraduate Research Scholarship (WCPRS) scheme.

Advanced are a partial funder of the project; University of Warwick are the other funder.

Findings/Outputs will be presented in a PhD thesis.

Additionally, recommendations from this project will be presented in an oral presentation and a short written report that can feed into the design of digital triage tools, service delivery and healthcare policy. This will be in plain English (non-scientific format), which will be shared with all project stakeholders, key audiences and on social media.

Findings of the study will be disseminated through the following:

1. Oral presentations to Advanced who develop and provide the Odyssey digital triage tool. Audiences within Advanced will include leadership, clinical and product management teams. There will be an emphasis on areas for improvement in terms of service delivery and the clinical content within the digital triage tool.

2. Findings will be shared with the urgent care providers whose data is being analysed in the study: A report and presentation of findings will be offered to participating urgent care providers, including suggestions for service delivery improvements. Examples of the care providers include: Mastercall (http://www.mastercall.org.uk/services/out-of-hours-service), Bardoc (http://bardoc.co.uk/)

It is expected that audiences will include those responsible for managing service delivery, clinical staff (digital triage end users), and those responsible for training and quality assurance of digital triage tools. Findings from the research will be summarised in a PowerPoint presentation, which will be presented to audiences including policymakers and commissioners responsible for urgent care services, including members of NHS England and through special interest groups, for example the Society for Academic Primary Care (SAPC) special interest group: Digital Technologies in Primary Care.

Findings from the research will be summarised (in lay English) in a PowerPoint presentation. A written summary of findings report will also be created. The oral presentation and study reports will be shared with collaborating patient and public groups (for example: Healthwatch or GP practice patient groups) and service users who participated in the study who have indicated their interest in receiving a study report.

Findings will be written into academic research papers, which will be disseminated in peer reviewed journals (such as BJGP, BMC journals and BMJ open) and will be presented at conferences focussed on digital health and primary care. There are three anticipated papers: the systematic review, quantitative findings and mixed methods findings.

All outputs will contain only data that is aggregated with small numbers suppressed in line with the HES Analysis Guide.

There will be no suppression of results or outputs by the funders. This means that if the results show anything negative towards Advanced and the Odyssey digital triage tool, that these finding will still be published.

Outputs are expected to be complete by March 2023.

Yielded Benefits is not a requirement for new applications.