Investigating the Application of Causal Inference Methods for Modelling the Impact of Treatment Sequences in Health Economic Evaluations: Utilising Real-world Evidence from the English Cancer Registry

On 1 February 2023, NHS Digital merged with NHS England. NHS England has assumed responsibility for all activities previously undertaken by NHS Digital. The merger was completed by a statute change. Any reference made to NHS Digital within this Data Sharing Agreement is in reference to the merged organisation known as NHS England.

The University of Sheffield aims to investigate whether the English Cancer registry data is sufficient for reliably comparing the effectiveness of different sequential treatments in treating cancer patients in the NHS.

Patients can sometimes receive a series of treatments in a sequence instead of a single line of therapy. Alternating the order of treatments may result in different overall effectiveness and costs of medical treatments. Thus, it is essential to consider the sequence of treatments when making health resource allocation decisions, particularly for cancer treatments as they usually impact a patient’s survival.

Treatment effects are usually compared in clinical trials. However, a major limitation of some clinical trials is that they often do not provide details about patients’ treatment histories or treatment sequences. In this case, analysing routine healthcare data may help provide a better understanding of the effect of sequential treatments.

Analysing routine healthcare data without proper adjustments may lead to incorrect results in clinical trials. Therefore, several large international initiatives have been investigating methods to obtain reliable treatment effects using routine health care data. This generally involves replicating clinical trial results using routine healthcare data and further extrapolating the results to a wider population. None of the studies has examined the scenario of comparing the effectiveness of sequential treatments. Additionally, most of these studies were conducted in a non-UK setting. Thus, the project aims to fill this knowledge gap with two case studies comparing the effectiveness of different sequential treatments in treating prostate cancer and kidney cancer patients in the NHS. Specifically, the study plans to replicate the results of two clinical trials using the UK cancer registry data. For the purpose of clarity, the studies referenced above are not in any way allied to this project, and will not have access to data disseminated under this Agreement.

To support this work the University of Sheffield requests pseudonymised sub-sets of the following datasets:

• NDRS Cancer Registry

• NDRS Radiotherapy Dataset (RTDS)

• NDRS Linked Cancer Waiting Times (Treatments Only)

• NDRS Linked Hospital Episode Statistics (HES)- inclusive of the Admitted Patient Care, Accident and Emergency and Outpatient subsets

• NDRS Systemic Anti-Cancer Therapy (SACT) Dataset

All the above datasets will aid in determining what treatments a patient received, and the sequence in which a patient received the treatments. The cancer registry contains basic patient information at the time of diagnosis (e.g. age, gender), important prognostic factors, such as tumour stages and sizes, and patient performance status. Moreover, the Systemic Anti-Cancer Therapy (SACT) data set collects information on cancer therapies, including those available through the Cancer Drugs Fund (CDF). These NCRAS datasets can also be linked with NHS hospital records (e.g. Hospital Episode Statistics (HES)) to provide a complete picture of patients’ treatment trajectory.

To address the UK General Data Protection Regulation (GDPR) Principal of Data Minimisation the data requested is limited to individuals aged over 18 with a diagnosis of prostate cancer (C61) or renal cell carcinoma (C64), which is the minimum amount of data necessary to achieve the purposes outlined within this DSA. Patients below 18 will be excluded from the analysis due to the following reasons: (1) The study team aim to apply the same exclusion criteria from trials in the same diseases as this study is comparing results with those from existing trials. (2) Treatment patterns among younger patients may be significantly different from adults, and thus less comparable for the scope of this study. (3) The incidence of prostate cancer and renal cell carcinoma has been extremely low among adolescents and children in England. Excluding cases below 18 may help prevent small cell numbers being produced in our results. The study team have undertaken extensive work with the NHS Digital data production team to ensure compliance with the principle of data minimisation.

The data period requested to identify the primary patient cohort for both case studies is from January 1st 2011 to the latest NCRAS data available upon extraction. A subset of minimised historical data of the identified patient cohort dated back to 6 years prior to patient’s cancer diagnosis (i.e. since January 1st 2005 or the earliest available date of a dataset (if later than 2005)). The necessity for requesting over 10 years of data can be justified by the following feature of the study:

(1) This study will investigate the treatment effects on cancer survival and, therefore, long-term data are essential.

(2) This study involves emulating existing RCTs that have a minimum of 5 years of follow-up, and therefore long-term data is needed to make adequate comparisons between our analyses and those included in the existing RCTs.

(3) This study assesses the effectiveness of treatment sequences, which have changed over time. Therefore, it is important to have data over a prolonged time period.

There are no alternative, less intrusive ways of achieving the purposes outlined within this Data Sharing Agreement (DSA). The study has taken the appropriate steps to obtain ethical approval from an NHS Research Ethics Committee (REC).

The lawful basis for processing falls under the UK General Data Protection (GDPR) Article 6(1)(e), the processing is necessary for you to perform a task in the public interest and falls under the official functions outlined within the Universities Royal Charter. Processing of special category data falls under UK GDPR Article 9(2)(j), this processing is necessary for archiving purposes in the public interest, for scientific and statistical purposes in accordance with Article 89(1), based on Union law which shall be proportionate to the aim pursued, respect the essence of the right to data protection and provide for suitable and specific measures to safeguard the fundamental rights and the interests of the data subject.

This research has been deemed to be in the public interest as evidence of the effectiveness of different treatment sequences in treating cancer patients is scarce. Clinical trials are not usually designed to assess treatment sequences. However, in reality, most people with cancer receive a sequence of treatments and therefore evidence on the relative effectiveness of different treatment sequences would be valuable for patients, clinicians, and healthcare decision-makers. Information on patients who receive different treatment sequences is available in the cancer registry and linked datasets, and these data could be used to assess the relative effectiveness of different treatment sequences. However, analyses of observational data sources are prone to biases and our study aims to investigate this, to ascertain whether it is possible to estimate the relative effectiveness of different treatment sequences using an English cancer registry and linked datasets.

For the purposes of investigation, this study proposed using trial-mimicking procedures to identify the study cohort and to conduct statistical analyses. By mimicking the procedures of a randomised control trial as closely as possible, the study team may minimise biases associated with analysing observational data (e.g. confounding bias) so as to enhance the validity of our results. The study team aim to test the performance of different causal inference statistical methods for mimicking the randomisation procedures such as propensity score weighting and g-methods.

The study is in the public interest because it will provide evidence on whether it is possible to use these datasets to inform healthcare decision-making on treatment sequences. It is of scientific and statistical importance due to our proposed investigation of advanced statistical methods on routinely collected national datasets.

The University of Sheffield is the sole data controller who processes the data for the purposes outlined within this Agreement. The Wellcome Trust fund this work, other than funding the work the Wellcome Trust play no role in this project.

There is no flow of identifiable data into NHS Digital to support this request.

NHS Digital will flow pseudonymised sub-sets of the following datasets to the University of Sheffield via a secure file transfer system:

• NDRS Cancer Registry

• NDRS Radiotherapy Dataset (RTDS)

• NDRS Linked Cancer Waiting Times (Treatments Only)

• NDRS Linked Hospital Episode Statistics (HES)- inclusive of the Admitted Patient Care, Accident and Emergency and Outpatient subsets

• NDRS Systemic Anti-Cancer Therapy (SACT) Dataset

There will be no subsequent flows of data.

The University of Sheffield aims to determine if it can replicate results from two existing clinical trials using the National Cancer Registration and Analysis Service (NCRAS) data.

Treatment effectiveness will be compared between patients receiving different treatment sequences. The results of the study’s analyses will be compared with those from the clinical trials. If results are similar, the study may conclude that English cancer registry data is sufficient to obtain reliable comparative effectiveness of different sequential treatments.

The study will proceed to the second stage if the English cancer registry data is demonstrated to be sufficient. That is, the University of Sheffield will extrapolate the analyses to a broader population (i.e. removing the trial-matching exclusion criteria when selecting patients). This will allow the University of Sheffield to determine the effectiveness of sequential treatments being used in“a "non-trial” population.

The data being disseminated under this Agreement will not be linked to datasets not already referenced within this Agreement. The University of Sheffield is not permitted to reidentify individuals from the pseudonymised data.

All those processing the data disseminated under this Agreement will be either substantive employees of the University of Sheffield or University of Sheffield students with honorary status. Those accessing data will receive appropriate data protection and confidentiality training. Data will be processed by project team members only, and only for the purposes outlined within this DSA.

The data will be accessed via a secure virtual environment running on the University of Sheffield-owned and managed infrastructure in England. Analysts will conduct all data processing including the statistical analyses to fulfil the research objectives in this secure virtual environment. Off-site access is facilitated by a secure VPN (virtual private network) connection authenticated by a University username and remote password. This agreement prohibits the transfer of data from the virtual environment at the university to other machines. This service is maintained by the University’s Corporate Information and Computing Services. The University of Sheffield will comply with the Data Protection Act and the University's own Information Security and Data Protection Policies as well as the School of Health and Related Research (ScHARR) Information Governance Policy. In line with ScHARR’s Information Governance Policy. Only aggregated data/outputs, with small numbers suppressed, will leave the secure virtual environment.

Primarily, the results of this project will form part of a PhD thesis submission. The submission will be provided to the University of Sheffield and the study’s funder, the Wellcome Trust. The current submission date for this project is May 2023. The Wellcome Trust have an open source publishing policy (with funding) to increase the visibility of the research when its ready to be submitted to a peer-review journal.

The University of Sheffield also plans to submit the findings to peer-reviewed journals to ensure wider dissemination of the study findings.

The study team currently aims to present the study findings at international conferences, such as the International Society for Pharmacoeconomics and Outcomes Research Conference (May 2023, Boston; November 2023, Copenhagen) and the International Health Economics Association Congress (July 2023, Cape Town). Presenting at such conferences will allow the findings to reach scientists, innovative technology-focused organisations and policy-making communities.

The University aims to publish the study findings in a lay-friendly format on its web pages to reach that the results reach interested groups and civil society. Such publications may also be shared on social media.

Any data contained in the outputs references above will be aggregated with small numbers suppressed.

This dissemination has the potential to benefit the provision of health and social care in England, it may allow the study team to determine whether NCRAS data can be reliably used as a source of data when comparing the effectiveness of different treatments and sequences of treatments.

This dissemination may benefit decision-makers in health regulatory bodies:

1. The project aims to provide evidence on the value of using the NCRAS data as an alternative source of evidence when evaluating treatment sequences in health economic evaluations and clinical decision-making. That is, the study may be able to provide guidance on a systematic approach for providing reliable comparative effectiveness estimates of sequential treatments using UK routine health care data. Such guidance could be used to investigate many other questions that involve the comparison of treatment sequences. This can be important for decision problems where clinical trial data are lacking, but also for refining best practice guidance for research using national registry data.

2. If analyses are unsuccessful, the study will make suggestions as to whether the national health data collection may be improved to enable better analyses in the future.

This dissemination may produce several social benefits:

1. This project looks to examine whether data collected from NHS patients can be reliably used to support future decisions making for the patients in the NHS (i.e. using NHS data for NHS patients). Typically, health resource allocation decisions are heavily based on evidence from clinical trials. However, patients treated in the NHS are not always like those who are eligible to attend a clinical trial. Therefore, evidence from trials is often not generalisable, while evidence from the real world may offer additional insights. Ensuring the generalisability of trial results is essential to ensure that decisions are based on valid and reliable results.

The study team aim to liaise with the National Institute for Health and Care Excellence (NICE) decision support unit, as the study findings may be of relevance to the NICE Real-World Evidence Framework. The NICE Real-World Evidence Framework acknowledges the growing use of real-world data in the health technology assessment process but describes concerns around potential biases that can occur in analyses of non-randomised data. Issues around data quality are also discussed. The study will provide further information on the use of the English cancer registry and linked data to assess the effectiveness of cancer treatments used in clinical practice, which will help decision-makers such as NICE understand whether and how these datasets can be used to inform healthcare decision-making.

In summary, our study may demonstrate whether NCRAS data is of sufficient quality to allow reliable comparison of treatment sequences in treating prostate cancer and kidney cancer patients in healthcare care settings. As such, this study may extend the user experience of NCRAS data and provide public health benefits.

There is no specific target date or measure for the health and social benefits of this project. These benefits are likely to follow the study outputs described in the previous section.

The work the principle investigator is leading on, of which this project is part, has contributed to advancements in the use of real-world data for evaluating treatment sequences in health technology assessments. The project's findings are anticipated to offer additional insights that support and complement the recently published NICE Real World Evidence Framework. Furthermore, seminars and presentations planned both nationally and internationally will disseminate research and methodological developments from the lead applicant and the research group, which will inform subsequent development.