Evaluating prescribing safety indicators embedded in computerised clinical decision support software

Medication errors in general practice are an important and expensive preventable cause of safety incidents, illness, hospitalisations and deaths. IT-based tools are increasingly being used to support general practitioners in their clinical decision-making including safer prescribing. The National Institute of Health Research (NIHR) are therefore funding a programme of work called: 'Avoiding patient harm through the application of prescribing safety indicators in English general practices (acronym: PRoTeCT)'. Using a series of linked work packages, the PRoTeCT Programme aims to evaluate two large-scale interventions in English general practices that employ prescribing safety indicators to reduce hazardous prescribing and avoidable harm to patients: the clinical decision support software OptimiseRx, and a pharmacist-led IT-based intervention (PINCER).

One of these tools, OptimiseRx, alerts the prescribers of potential errors during the medication prescription process. However, despite its potential to improve prescribing safety and patient outcomes, the effectiveness of OptimiseRx has never been quantified. Such investigation is of substantial public interest for two reasons. Firstly, to make recommendations to improve patient safety in the future. If OptimiseRx is effective in aiding health professionals to prescribe more safely, it is something to recommend. Secondly, to understand if it is cost-effective for the NHS. As part of the PRoTeCT programme, the University of Manchester is leading on the aim to evaluate the effects of OptimiseRx on potentially hazardous prescribing in primary care and serious harm outcomes, as well as to evaluate the cost-effectiveness of OptimiseRx on behalf of NHS England.

This study aims to assess the effect of the implementation of OptimiseRx on potentially hazardous prescribing and associated adverse outcomes including hospitalisation and death; and to evaluate the cost-effectiveness of OptimiseRx to NHS England. This study will derive generalisable insights into the effectiveness and cost-effectiveness of point-of-prescription decision support systems.

To fulfil these aims, Hospital Episode Statistics (HES) Admitted Patient Care (APC), HES Critical Care (CC), and Civil Registrations (Deaths) Secondary Care Cut pseudonymised patient-level data is requested from NHS Digital. This data will be linked to pseudonymised patient-level primary care patient data from the ResearchOne clinical research database. ResearchOne is a health and care research database developed by a company called TPP, which holds clinical and administrative data drawn from electronic patient records currently held on the TPP SystmOne clinical system. Patient-level data from all general practices that use the SystmOne live environment and contribute their data to ResearchOne will be used for the linkage, irrespective of whether the practice has installed OptimiseRx.

This data linkage of ResearchOne primary care and NHS Digital HES APC and deaths data will allow an assessment of whether 79 different identified prescribing safety indicators in OptimiseRx lead to reductions in hospitalisations and death from a range of associated serious harm outcomes including: gastrointestinal bleed, exacerbation of asthma, heart failure, stroke, myocardial infarction, acute coronary syndrome, venous thromboembolism, arrhythmia, acute kidney injury, pneumonia, fractures, and rhabdomyolysis. The HES APC and HES CC data also enables economic analysis of the OptimiseRx tool from the perspective of NHS England to estimate cost per hazardous prescribing event avoided, and cost per serious harm outcome avoided. All NHS Digital linked data will only be used for the OptimiseRx evaluation, not for any other part of the PRoTeCT programme.

OptimiseRx is implemented at the Clinical Commissioning Group (CCG) level. For each CCG that has rolled out OptimiseRx, the University of Manchester will obtain from First Databank (industry partner; vendor of OptimiseRx) the start date for each of their practices when OptimiseRx was activated. If any practice has stopped using OptimiseRx since, First Databank will also provide their stop date. Data from ResearchOne covers both practices that have implemented OptimiseRx and practices that have not.

The study will investigate how often potentially hazardous prescribing events occur in the 24 months before and the 12 months after the practice starts using the OptimiseRx system. For each month, rates of potentially hazardous prescribing events will be estimated and compared to what would be expected from the trends before OptimiseRx was implemented. Using data from GP practices that are in ResearchOne but have not implemented OptimiseRx will help in understanding the secular trends and seasonal patterns of prescribing, driven by other sources of confounding such as changes in policy. The linked HES and mortality data at the patient-level will further enable investigation of any changes in serious harm outcomes associated with hazardous prescribing.

Only data for England will be required due to the inclusion of only including English practices. OptimiseRx was first implemented in September 2013. Given interest in the risks of serious harm outcomes associated with hazardous prescribing in the 24 months before and 12 months after the practices have implemented OptimiseRx, linkage to HES and mortality data from 2011/12 is requested. Data has been requested up to 2019/2020 for all three datasets, enabling analysis of patients from practices which implemented OptimiseRx in early 2019. Only the variables that are needed for analysis in each dataset have been selected and no outpatient or A&E data have been requested. There are no alternative, less intrusive ways of achieving the purpose of this study without the patient-level data as described in this agreement.

There is no direct linkage system between ResearchOne and HES/ mortality data. In order to link the data, TPP will send the relevant patient identifiers to NHS Digital. Approval from the Confidentiality Advisory Group (CAG) for Section 251 support to process routinely collected, identifiable patient data without consent has been obtained. Specifically, to enable ResearchOne and NHS Digital to generate project-specific patient pseudonyms, for ResearchOne to transfer the patient pseudonyms, gender, and date of birth of the patients to NHS Digital, and for NHS Digital to share non-identifiable health data with the University of Manchester for data linkage and analysis.

The University of Manchester and the University of Nottingham are joint Data Controllers. Only the University of Manchester will process the requested NHS Digital data. The University of Manchester lead on the OptimiseRx evaluation work package. The wider PRoTeCT programme is managed by the Principal Investigator based at the University of Nottingham. As the PRoTeCT programme lead, the University of Nottingham is ultimately responsible for all major decisions made regarding the OptimiseRx evaluation work led by the University of Manchester.

The PRoTeCT programme is a collaboration between the University of Manchester, University of Nottingham, University of Dundee, and University of Edinburgh. The role of team members from the Universities of Dundee and Edinburgh is advisory for the OptimiseRx evaluation study. The Universities of Dundee and Edinburgh do not determine the purpose or the means of the processing. The funder of the study, NIHR, have no role in the study design nor data analysis. Neither TPP (vendor of ResearchOne data) nor First Databank (vendor of OptimiseRx) are joint data controllers as they have no roles in determining the purposes and means of the processing of the NHS Digital data.

The legal bases for processing these data are:

- GDPR Article 6 (1)(e): "processing is necessary for the performance of a task carried out in the public interest or in the exercise of official authority vested in the controller";

- GDPR Article 9 (2)(j): "processing is necessary for archiving purposes in the public interest, scientific or historical research purposes or statistical purposes in accordance with Article 89(1) based on Union or Member State law which shall be proportionate to the aim pursued, respect the essence of the right to data protection and provide for suitable and specific measures to safeguard the fundamental rights and the interests of the data subject."

These legal bases for processing NHS Digital data apply to both the University of Manchester and the University of Nottingham.

The Universities of Manchester and Nottingham do not expect there to be any disadvantages or risks to individuals whose data will be used to understand patterns of potentially hazardous prescribing and associated outcomes. There will be no direct contact with patients and the research team will only have access to pseudonymised data. Only aggregated outputs will be shared with researchers outside of the University of Manchester and small numbers will be suppressed in line with the HES Analysis Guide.

All organisations party to this agreement must comply with the Data Sharing Framework Contract requirements, including those regarding the use (and purposes of that use) by “Personnel” (as defined within the Data Sharing Framework Contract ie: employees, agents and contractors of the Data Recipient who may have access to that data).

The steps described below will be used to enable linkage of data between First Databank (industry partner, vendor of OptimiseRx), TPP (industry partner, vendor of ResearchOne), and NHS Digital.

- The University of Manchester will generate two project-specific SALT strings. SALTs are strings of characters to be appended to the data that are being pseudonymised. SALT strings are a security tool for encrypting sensitive data. One of the SALT will be used to generate the general practice pseudonyms, and the other to generate the patient pseudonyms.

- The University of Manchester will transfer both SALTs to TPP and to First Databank.

- First DataBank will extract from their system the organisation data service (ODS) codes of the general practices, i.e. practices which have implemented OptimiseRx, together with their dates of implementation and end dates (if applicable).

- First DataBank will append one of the SALT strings to each ODS code and then apply a secure, one-way hash algorithm to create a ‘digest’ (practice pseudonym) – an alphanumeric string which cannot be reversed.

- First Databank will transfer the list of practice pseudonyms generated together with the OptimiseRx start/end dates to TPP.

- Using the same SALT that First Databank used, TPP will independently apply it to the ODS codes of the practices in the ResearchOne database. Since the same field (i.e. ODS code) and SALT are used by both First Databank and TPP, the same digest (practice pseudonym) can be understood by both parties. Therefore, comparison of the practice pseudonyms generated by TPP with those from First Databank will enable TPP to ascertain which practices have implemented OptimiseRx and are also contributing to ResearchOne.

- TPP will use the second SALT string to create a 'digest' for each patient (i.e. patient pseudonyms) in the ResearchOne database from their NHS number, and generate the primary care data extract. A bespoke study id for each patient (‘R1 IDs’) in the primary care data extract will also be generated.

- TPP will send the pseudonymised patient level primary care data extract to the University of Manchester, together with the R1 IDs. The SALT generated patient pseudonyms will not be included.

- TPP will transfer the SALT generated patient pseudonyms and R1 IDs to NHS Digital using the Secure Electronic File Transfer service (SEFT). The same file will also contain the patients' gender and date of birth. These variables are included to ensure accurate linkage between ResearchOne and HES/ mortality data.

- The University of Manchester will share the same SALT string that TPP used to generate the patient pseudonyms with NHS Digital. Using this SALT, NHS Digital will independently generate the patient pseudonyms in their database from the NHS numbers. These pseudonyms, together with the patients' gender and date of birth, will then be matched with the data from TPP. This will enable NHS Digital to ascertain those patients whose HES/ mortality data are required.

- NHS Digital will extract the HES/ mortality data for these patients for all data years requested.

- NHS Digital will transfer the pseudonymised patient level HES/ mortality data extracts containing the R1 IDs to the University of Manchester.

- Using the R1 IDs, the research team at the University of Manchester will link the primary care extract with the HES/ mortality data extracts. The merged file will then be used for analysis. There will be no requirement/attempt to re-identify individuals. There will be no subsequent flows of patient level data. NHS Digital data will only flow to the University of Manchester who will complete all data processing for the study.

Data storage and processing:

The research team at the University of Manchester will only have access to pseudonymised, record level data. All data received from TPP and NHS Digital will be stored in the University's Research Data Storage Service - an access restricted data share on the University network storage infrastructure, which is the recommended location for storing sensitive or critical University data. The storage infrastructure is hosted across two data centres for resilience and disaster recovery purposes.

The hardware in the data centres and the network infrastructure belong to the University of Manchester. Nothing is shared. There is dedicated space which is caged off. Dedicated University of Manchester staff manage the University infrastructure in both data centres. Physical access to the data centres is strictly limited to data centre staff and a limited number of authorised IT Services staff. The data centres are protected by physical and electronic access security systems, swipe card access in and out of the data centres and CCTV coverage.

Only authorised researchers at the University of Manchester employed for this study will have access to the data, and access control is managed via Active Directory groups and Unix groups. Users of the data have also all been trained in data protection and confidentiality and will adhere to the Data Protection Act 2018 when collecting, using, disclosing, retaining or disposing of personal data. The data will be processed on the University of Manchester interactive Computational Shared Facility (iCSF), which is a service designed specifically for interactive computationally-intensive work. The iCSF is only accessible on campus and exists on a private network. All data processing is done on the iCSF and no raw data will be transferred out. The workstations used for accessing the iCSF environment do not directly access the data. Access is via Virtual Desktop Infrastructure (VDI) technology to ensure the data is only processed within, and never leaves the virtual environment. No remnants of the data are ever stored on the user device through mechanisms such as temp files or browser caches.

A valid University of Manchester IT account is required to login to the iCSF. Account credentials are unique to each member of staff and only the account owner knows the password. All staff accessing the data will be substantive employees of the University of Manchester.

An interrupted time series (ITS) approach will be used to compare the incidence and prevalence of potentially hazardous prescribing events, and of associated serious harm outcomes. Practices are required to contribute at least 24 months of data to ResearchOne prior to their start date of OptimiseRx, and 12 months of data following their start date. The date of implementation of OptimiseRx will be considered as time point zero. For each month, the incidence and prevalence of potentially hazardous prescribing events as defined by the prescribing safety indicators will be estimated. The intervention effect will be calculated as the difference between the observed and predicted values, had the prior trends continued after OptimiseRx was turned on. Similarly, the patterns in serious adverse outcomes over the same time period will be estimated. Adverse events will be identified from primary care records using Read Clinical Terms Version 3 (CVT3) codes in ResearchOne, and ICD-10 codes from linked HES & mortality data. Practices contributing to ResearchOne that did not implement OptimiseRx will be used to explore and model secular trends in hazardous prescribing and serious harm outcomes, driven by other sources of confounding such as changes in policy. For the analysis of risks of serious harm outcomes associated with potentially hazardous prescribing, self-control case series method may also be explored. In this model, patients would act as their own control and risk of adverse outcomes during period of exposure to OptimiseRx intervention will be compared with the risk during all other observed time periods.

The economic evaluation will estimate, per practice, the difference in costs and outcomes generated for those practices implementing OptimiseRx, according to the proportion of hazardous prescribing detected and averted in a practice. The analysis will generate cost per hazardous prescribing event avoided, and estimates of cost per serious harm outcome avoided and hospitalisation avoided. Cost data will comprise the costs of providing OptimiseRX (from OptimiesRX data) and costs of hospitalisation (from NHS Digital HES data). Costs will be applied to hospital data via Healthcare Resource codes and the associated National Tariff price.

The data will be analysed using statistical packages. No record level data will be produced as an output at any stage; only aggregated results will be reported (with small numbers suppressed in line with the HES Analysis Guide). The outputs produced cannot be used to identify patients or sensitive information.

The study team will work with a number of organisations such as NHS England, NHS Improvement, The Health Foundation, Academic Health Science Networks, the Royal College of General Practitioners, and others to disseminate research findings. The expected output would include:

- Dissemination through meetings, project summaries, seminars, webinars, policy briefings and open access journal publications (e.g. leading medical journals such as Plos Medicine).

- Production of evidence that will help the NHS make investment decisions concerning prescribing safety innovations in general practice.

- Presenting at national conferences aimed at health care planners, practitioners and policy makers.

- Articles in CCG newsletters where appropriate.

- Developing strong links with a wide range of patient safety communities, e.g. The Health Foundation Q Community: https://q.health.org.uk/, National Medication Safety Network, Patient Safety Collaboratives: https://www.england.nhs.uk/patientsafety/collaboratives/.

Results of the cost-effectiveness analysis will form part of these outputs.

The study team also have experienced patient and public involvement members aligned to this research project and programme management. They are involved in all aspects of the study design, progress and dissemination. Two patient members of the wider PRoTeCT Programme Grant Management Group attend monthly meetings. They will contribute to the final report and be involved in dissemination to relevant patient and public audiences. Wider involvement of patients and the public will be obtained by continuing engagement with the Research Users Group of the NIHR Greater Manchester Primary Care Patient Safety Translational Research Centre, and the Patient and Public Involvement Senate of the East Midlands Academic Health Science Network.

Only aggregated results will be disseminated, with small numbers suppressed in line with the HES Analysis Guide. The completion dates for the outputs would be:

Production of final report to the NIHR – by 28/02/2023.

Write open-access journal articles – by 28/02/2023.

Disseminate findings to policy makers, managers and clinical leaders through project summaries, seminars, webinars and policy briefings – by 28/02/2023.

Any foreground intellectual property (IP) generated as a result of the project is assigned to Notts Healthcare NHS Trust as per the NIHR collaboration agreement for the wider ProTeCT programme, and will not be held by the commercial companies that are involved (TPP and First DataBank). Each party within the programme is however granted an irrevocable, non-transferable, royalty-free right to use all arising IP generated in the course of the project for academic teaching, research purposes and for non-commercial clinical purposes.

Reducing the incidence of avoidable harm including hazardous prescribing events and associated serious harm outcomes is an important priority for the NHS. It is estimated that 237 million medication errors occur at some point in the medication process in England annually, with 38.4% occurring in primary care contributing to 34% of all potentially clinically significant errors (Elliott et al, 2020). Definitely avoidable adverse drug events (ADE) are estimated to cost the NHS £98,462,582 per year, consuming 181,626 bed-days, and causing/contributing to 1708 deaths. This comprises primary care ADEs leading to hospital admission (£83.7 million; causing 627 deaths), and secondary care ADEs leading to longer hospital stay (£14.8 million; causing or contributing to 1081 deaths). In addition, a study of prescribing safety in 526 general practices in the UK based on 24 prescribing and medication monitoring indicators found that 49,927 of 949,552 patients at risk triggered at least one prescribing indicator and 21,501 of 182,721 triggered at least one monitoring indicator (Stocks et al, 2015).

There is therefore a need to develop interventions to reduce these avoidable events and OptimiseRx was developed with this purpose. However, despite its potential to improve prescribing safety, patient outcomes and cost to the NHS, any impact on these has never been quantified. This study aims to assess the effect of the implementation of OptimiseRx on potentially hazardous prescribing and associated adverse outcomes including hospitalisation and death; and to evaluate the cost-effectiveness of OptimiseRx to NHS England. The study’s findings will be used to make recommendations to improve patient safety in the future. If OptimiseRx is effective in aiding health professionals to prescribe more safely, it is something to recommend. Cost-effectiveness of OptimiseRx will also be taken into account in any recommendations to NHS England.

A systematic review published in 2009 showed the benefits of alerts at the point of prescribing in reducing hazardous prescriptions, but almost all the evidence was from studies in US hospitals with a lack of interventions in primary care (Schedlbauer et al, 2009). In addition, the prevalence of many of the 79 prescribing indicators included in this study are unknown. It is therefore difficult to estimate the magnitude of the potential effect of OptimiseRx on hazardous prescribing and serious harm outcomes. However, since medication errors in primary care is common, if OptimiseRx is effective in averting even a small percentage of these errors, the number of benefited patients and the potential saving to the NHS could be significant. Therefore, the outcomes/impacts would potentially include:

- Reduction in hazardous prescribing and avoidance of patient harm

- Enhanced knowledge sharing and shared decision-making with patients concerning the management of hazardous prescribing

- Improvements in prescribing safety in NHS general practice

References:

Elliott RA, Camacho E, Jankovic D, Sculpher MJ, Faria R. Economic analysis of the prevalence and clinical and economic burden of medication error in England. BMJ Qual Saf 2020:bmjqs-2019-010206.

Schedlbauer A, Prasad V, Mulvaney C, et al. What evidence supports the use of computerized alerts and prompts to improve clinicians' prescribing behavior? J Am Med Inform Assoc 2009;16:531-8.

Stocks SJ, Kontopantelis E, Akbarov A, Rodgers S, Avery AJ, Ashcroft DM. Examining variations in prescribing safety in UK general practice: cross sectional study using the Clinical Practice Research Datalink. BMJ 2015; 351:h5501.

Not stated in the register.