Patient Choice and Provider Quality - Why Patients Change GPs

The research team presenting this data application are based in the Economics and Public Policy (EPP) Department within the Business School at Imperial College London.

Informed choice is a key element of the rights and pledges outlined within the NHS constitution. Accordingly, the NHS has invested in a variety of online systems and programs to better assist patients in making informed choices about their healthcare. Among these is a review system that allows patients to view reviews of prospective General Practitioner Practices (GPPs), hospitals, and other services (this system was formerly called “NHS choices” and is currently called “The NHS website”).

Despite the focus on informed choice, and increasing efforts to utilize technology, relatively little is known about the efficacy of online systems in impacting patient choice and behaviours. Do patients consider reviews in their decision-making process? Does this lead them to choose practices that are better rated on the basis of other metrics? Understanding these questions is key to the design and development of new systems that allow the NHS to maintain informed choice. Understanding these questions is key to the improvement of existing systems and development of new systems that could allow the NHS to further promote informed choice by patients.

In January 2020, the NHS website removed the summary of reviews for each GPP, which was previously shown prominently at the top of each GPP’s page. The introduction, sustained use, and later removal of ratings and reviews information on the NHS website provides a rare opportunity to begin to understand the relationship between online information on healthcare quality and patient choice.

Detailed and continuously updated data on online reviews at the GPP level is available via NHS England. Information on GPP registration is available via the Patient Demographics Service. The combination of these two data sources allow for the estimation of associations between the public availability of online reviews and GPP enrolment growth rates.

The research team’s project will study how individuals use online reviews to choose their GPP. The analysis will provide a first test for the relationship between access to online patient reviews and better patient choices. Moreover, the analysis will consider differences in the strength of this relationship across various demographic characteristics defined at the Lower Layer Super Output Area (LSOA) level. By doing so, the study will be able to understand if, for instance, individuals in wealthier areas tend to benefit more from online information relative to individuals from poorer areas, even after taking into account the differences in GPP choices available in each area. In other words, the study will also shed light on how online information systems influence inequality in access to quality healthcare.

More specifically, the analysis will address the following questions:

1) To what extent do patients consider online reviews in their decision-making process?

The data provided will allow the research team to assess whether individuals choosing a new GPP (e.g., because of a change of address) are more likely to choose a GPP that has a higher star rating, amongst those GPPs available to them. There are many confounding factors that may explain the correlation between enrolment and GPP star ratings. The data provided will allow the research team to obtain credible causal estimates of the effect of the website start ratings on patient enrolment using two methods. First, the team will examine GPPs changes in star ratings over time. Second, the team will examine GPPs at the boundary of ratings.

2) How do different individuals use the information in the NHS website? For instance, do individuals in low income or low education LSOAs respond more or less to changes in reviews? What implications does this have for inequality in access to high-quality healthcare?

The data provided will allow the research team to answer this question by assessing whether individuals in high and low-income LSOA are equally likely to choose GPPs with higher star ratings, amongst those GPPs available to them.

3) Does this lead patients to choose practices that are better rated on the basis of other metrics (for instance, on the basis of clinical QOF scores)?

To answer this question, the research team will determine whether start ratings are correlated with objective measures of clinical quality like clinical QOF ratings and LSOA level improvements in health outcomes. Then, the data provided will allow the research team to determine whether the determinants of patient enrolment are strongly correlated with clinical quality.

4) What has been the effect on patient choices and outcomes of the website’s change in format in January 2020?

The data provided will allow the research team to answer this question by comparing the rate at which patients choose GPPs with high star ratings and high clinical outcomes measures, in the period immediately before and immediately after the change in the website format (the NHS website discontinued displaying overall star ratings in January 2020). In this context, the team will be particularly careful to avoid confounding effects due to the first wave of COVID-19 which occurred shortly after the website format change.

5) Besides the online rating, what other characteristics of GPPs do individuals value the most when choosing their GPP?

The data provided will allow the research team to answer this question by determining what characteristics of GPPs are predictive of patient enrolment. For instance, if patients are very likely to enrol in the GPP closest to the LSOA in which they reside, even if that GPP offers fewer services and has a lower start rating, this would allow the researchers to determine the extent to which patients value distance over and above GPP services and star ratings. In practice, the team will consider average demographic characteristics of each individual’s LSOA, since the data will be pseudonymized so that no individual can be identified in the data. The research team will take particular care to ensure that the model accounts for the fact that 1) different individuals have access to different sets of options from which to choose their GPPS (and some individuals might have a single option) and 2) switching GPPs is rare unless there is an event requiring an individual to choose a new GPP, such as moving to a new area or a GPP closure.

The answers to these questions have important consequences for the NHS. Appropriate design of systems to convey information is key to promoting informed choices by patients [b]. The study’s findings would give the NHS insight on how the previous NHS website influenced patient choices. The study will also determine the consequences of the January 2020 website format change and suggest ways to improve how information is conveyed to patients, with a view to improve patient outcomes and reduce inequality in access to quality primary care in the future. The study will also highlight which features of GPPs are most valued by patients, which can be used by GPPs to improve the quality of the services they provide. The authors have discussed their preliminary findings with members of an NHS England team responsible for the NHS website. The study has not been commissioned by NHS England but the NHS England team involved in the work area are interested in understanding the findings from the study, particularly understanding the consequences of the website’s change in format.

[b] See, for instance, Marsh C, Peacock R, Sheard L, Hughes L, Lawton R. Patient experience feedback in UK hospitals: What types are available and what are their potential roles in quality improvement (QI)? Health Expect. 2019;22:317–326.

The research team has produced some preliminary results, using publicly available but much less granular data aggregated at the GPP level. The initial analysis indicates a strong relationship between publicly observed quality (i.e., the star rating awarded to a GPP) and enrolment growth. The team’s preliminary analysis also suggests that patient reviews are correlated with other measures of GPP quality such as clinical QOF indicators, and that online information is most useful in areas where there are many GPPs to choose from.

However, the publicly available data is too aggregated to answer the team’s research questions. The requested granular level data is indispensable to answer the questions described above for the following reasons:

• The publicly available aggregate data contains only the total number of individuals registered at a GPP at each point in time. This aggregate data conceals a large number of the changes of GPPs done by patients, which biases the study’s results.

• Individuals switch GPPs relatively rarely, so an individual level data set is necessary to understand the determinants of such relatively rare events.

• The requested data will allow the research team to identify the effect of online information and those patients who are actively searching for a new GPP (e.g., due to a change in address). This population is the most relevant one since these patients are the ones most likely to benefit from additional information.

• The requested data will allow the research team to address confounding factors as described above. In addition, the research team will be able to distinguish GPP enrolment growth from broader population trends, which make it impossible to obtain clean statistical results when using aggregated data alone.

DATA MINIMISATION

The requested data will provide the minimal level of information necessary to track the choice of provider at the individual level over time, allowing the team to estimate the underlying drivers of GPP enrolment. The project requires a sample from the Personal Demographics Service (PDS) data for the period from April 2015 to March 2021. This time period would allow the team to study how patients use online information in general, but also to determine the specific effect of the change in website design that occurred in January 2020. The project requires individual level data on GPP choices because the statistical analysis relies on tracking individuals as they move from one GPP to another. For these reasons, there is no alternative data set which would allow for a robust and credible analysis of how individuals choose their GPP. The analysis is particularly concerned with individuals that switch GPPs, including those that switch residences. Given this, the data requested would identify one of 3 changes in the PDS data along with the month and year of the change:

1. Postcode is the same, GPP Code has changed

2. Both Postcode and GPP Code have changed

3. Postcode changed, GPP Code is the same

The researchers have minimized the data request as follows:

• The project requires data only for the period from April 2015 to March 2021. This is the period for which information is available regarding the NHS Choices website. In order to study the effect of the change in website format (which occurred in January 2020) the researchers require data for the period following the change, hence the request for the sample period to extent to March 2021.

• The data set requested does not require linking to Hospital Episode Statistics (HES) data set.

• The researchers do not require information on postcode (the research team will use information about an individual’s LSOA to link average demographic information from the 2011 Census).

• In order to avoid the possibility of identifying individuals that have had a changed address for such reasons as going on a witness protection programme or child protection programme, Data Production is requested to remove all such cases where this sensitive ‘suppression’ flag has been applied (Stop Note indicator).

• The project requires each patient’s GPP code only. The project does not require individual Practitioner codes.

• The project requires only the address LSOA for each individual. The project does not require full addresses. Unfortunately, the first 3 digits of postcode is not enough since this would give too coarse a view of individual choices.

• The project requires data only for England. However, it is necessary to obtain data for all of England since individuals are likely to move across the country.

• The researchers require only a Pseudo-ID for each individual. The project does not require individual NHS numbers and individuals will not be identifiable from the data.

• The project requires the month and year in which an individual changes GPP Codes or Postcode. The project does not require the exact day.

• The project requires only the month and year of death for each individual in the data who has died during the period covered by the sample. The project does not require the exact day of death. The project requires data on deaths because, without this data, the analysis would consider deceased individuals as actively choosing the same GPP, which would bias the results.

• The project requires age and sex for each individual. All ages are required from birth to death, but only in 5 years age bands. The sex variable will allow a better understanding of whether men and women choose GPPs differently.

• The project requires Country of Birth, as the analysis will explore the hypothesis that foreign nationals may make choices differently to those born in the UK.

• The project does not require records from individuals that are in small practices with less than 10 individuals. This is done to ensure the anonymity of individuals in the data.

Imperial College London is the sole Data Controller who also processes the data.

The legal basis for data processing is the GDPR Article 6(1)(e) processing is necessary for the performance of a task carried out in the public interest or in the exercise of official authority vested in the controller, and Article 9(2)(j). processing is necessary for archiving purposes in the public interest, scientific or historical research purposes or statistical purposes in accordance with Article 89(1) based on Union or Member State law which shall be proportionate to the aim pursued, respect the essence of the right to data protection and provide for suitable and specific measures to safeguard the fundamental rights and the interests of the data subject.

STUDY DESIGN

This study will use pre-existing pseudonymised historical data on patients registered in GP practices in England from April 2015 to March 2021 using the NHS Personal Demographics Service (PDS), extracted by NHS England using the following filters :

• GPP practice codes (we require only practice level identifier, not the individual physician identifiers)

• LSOA

• Pseudo-ID (individuals Study ID) for each record (we do not require individual’s real NHS numbers)

• Month and year in which individual’s change GPP Code or Postcode,

• Month and year of death for each individual in the data (if the individual has died within the period covered by the sample)

• Age (in 5-year bands) of each individual

• sex for each individual

• Country of birth

This will be linked to time series data on

1) average patient ratings of each GPP and

2) the aggregated (and rounded) summary ratings displayed on the NHS website between April 2015 to March 2021.

This information will be combined with baseline GPP registration statistics to calculate overall and LSOA specific enrolment growth rates. This linkage will not be at record-level and therefore will not add any identifiable information and hence will not generate an opportunity for re-identification. There will be no requirement or attempt to re-identify individuals. The data will not be made available to any third parties except in the form of aggregated outputs with small numbers suppressed in line with HES analysis guide.

METHOD FOR DATA EXTRACTION

- Leeds Data Production team extract a record-level cohort of patients registered in General Practitioner Practices (GPPs) in England only from PDS for each of 6 data periods according to bespoke filters and fields.

- Leeds Data Production remove all records with a sensitive suppression status (on a witness programme, etc).

- Leeds Data Production to map all the GPP codes (8 characters) to GPP Code using the date the record changed, so they are all in one format

- Leeds Data Production to filter out all cases where Stop Note indicator applied.

- Leeds Data Production team create a record level pseudonymised data extract for each PDS period and send this via SEFT to the research team at Imperial College London

The sole flow of data out of NHS England will be the one-time extraction of historic Personal Demographics Service (PDS) data outlined in this agreement. There will be no subsequent flows of data in or out of NHS England.

FINAL DATA FORMAT

The resulting data would consist of one row (per individual) for each change in Postcode or LSOA. This will mean multiple rows per each individual if the individual switches GPP and/or postcode multiple times.

Each row would contain the month and year of the event being recorded (for instance, change of GPP or change of address), as well as the variables described in the objective for processing section.

The primary statistical analysis will be of the relationship between GPP enrolment growth rates and average and summary reviews as displayed online. Both are continuous. The study team will primarily use linear regression analyses. Given the lack of any preliminary evidence on potential effect sizes, the study team cannot make meaningful estimates of the statistical power of the prospective study. Supplemental analyses will include tests of changes in these associations when summary reviews were taken offline, as well as subgroup analysis on the basis of LSOA and GPP level characteristics.

DATA PROCESSING

Data processing will be undertaken within Imperial College's Big Data and Analytical Unit Secure Environment (BDAU SE) which is hosted by Virtus SDC Limited in Slough, which will act as the secure storage and processing location. Data access is strictly controlled by the Imperial College's Big Data and Analytical Unit (BDAU) with stringent procedures including dataset registration process, limiting access to the data to the minimum numbers of the research team required and BDAU approved staff access facilities. Data analysis will only be undertaken through the BDAU. The data will only be used for the purpose outlined in this Data Sharing Agreement. Imperial College London staff are bound by all policies and regulations as substantive employees of the College.

All data will be stored and analysed within the Imperial College's Big Data and Analytical Unit Secure Environment (BDAU SE).

Data Processing is only carried out by substantive employees of Imperial College London who have been appropriately trained in data protection and confidentiality. All personnel will abide by Imperial College as well as national data privacy regulation. Imperial’s BDAU SE is a secure research environment, providing a standard operating/access model, secure data storage and processing environment and analysis software. It is ISO 27001 certified and also compliant with NHS England’s Data Security and Protection Toolkit. The BDAU SE can be accessed remotely by users using multi-factor authentication once the user registration process is completed. Data will only be provided once the appropriate dataset registration process is completed. All data files and directories will be encrypted using (ZFS) AES-256 encryption.

Virtus SDC Ltd are not considered a Data Processor as they do not access data held under this agreement. Virtus SDC Ltd are a data centre co-location provider (selling data centre space), not a cloud services provider. Imperial College rents space in Virtus data centres to host BDAU equipment. All data processing are performed by BDAU’s staff. Virtus SDC Ltd provide physical security, power and environmental controls. Therefore, any access to the data held under this agreement would be considered a breach of the agreement. This includes granting of access to the database[s] containing the data. Virtus have a number of quality credentials including ISO27001, certificate copies of which can be accessed or downloaded on the following page: https://virtusdatacentres.com/why-virtus/quality-credentials.’

The outputs of data processing will include summary statistics and the outputs of the statistical analysis (for instance, regression analyses and figures illustrating the study’s results). The study team expect to have an article ready for circulation to conferences within 12 months of receiving the data, with initial submissions to journals coming shortly thereafter. In all output, papers, working papers, and seminar slides only aggregate data will be shown, with small numbers suppressed in line with the HES Analysis Guide.

These outputs aim to be described and disseminated through a number of channels, including articles prepared for submission to peer-reviewed journals in economics, health economics, and industrial organization, presentations at academic seminars at Universities in the UK and abroad, and presentations at academic and policy conferences in the US, UK, and elsewhere. Conference submissions are particularly aimed at targeting health economics conferences within the UK to ensure maximum interaction with members of the policy making community.

To ensure information is available as freely as possible, working papers aim to be be circulated through the Social Science Research Network (SSRN), the National Bureau of Economic Research (NBER), the Center for Economic Policy Research (CEPR), or similar series. Additionally, economic models and results will be made available to the NHS and NHS England, for use in improving websites and other tools aimed at providing accurate and accessible information to patients.

Expected conference submissions include the International Industrial Organization Conference (IIOC), Econometric Society, The National Bureau of Economic Research Summer Institute in Industrial Organization, The National Bureau of Economic Research Summer Institute Health, The American Economic Association Annual Meetings, Health Economists' Study Group, ASHEcon.

Potential journal submissions include the Review of Economic Studies, the American Economic Review, The Rand Journal of Economics, Econometrica, The Journal of Health Economics, The Journal of the European Economic Association.

The research programme is expected to be directly beneficial to the NHS for several reasons. Most concretely, research outputs, including conference presentations and academic papers, will provide an explicit evaluation of the efficacy and impact of digital efforts by the NHS to provide accessible and relevant information to patients on GPPs.

The NHS has included the principle of informed choice within its Constitution for England. In addition to the legal right of patients to choose a GPP (unless there are reasonable grounds to refuse), the constitution notes the NHS’s pledge to

(i) inform patients about available healthcare services locally and nationally and

(ii) offer easily accessible, reliable, and relevant information in a form patients can understand, as well as support to use it.

However, there is no agreed-upon best practice for providing information about GPPs to patients. This is reflected in the changes in format to the NHS website, with a view of making the information more useful to patients . The research team’s study aims to be directly informative to NHS England as they seek to further improve the provision of accessible, reliable and relevant information to patients in an understandable form.

Furthermore, this research hopes to shed light on whether easily accessible information on GPPs helps to promote a well-functioning and high-quality provision of primary care. The study hope the analysis will also help understand to what extent does online information provision exacerbate or mitigate inequality in access to healthcare.

The study team at Imperial College London will remain in contact with the NHS in order to ensure that the study is as useful as possible.

The study’s results are hoped to have a number of important benefits for the NHS, as discussed above. The study’s findings would give the NHS insight on how the previous NHS website influenced patient choices. The study may also determine the consequences of the January 2020 website format change and suggest ways to improve how information is conveyed to patients, with a view to improve patient outcomes and reduce inequality in access to quality primary care in the future. The study may also highlight which features of GPPs are most valued by patients, which can be used by GPPs to improve the quality of the services they provide.

In sum, the benefits of the analysis are the following:

1) The analysis aims to provide empirical insight into the role of information barriers in patient choice of GPPs.

2) The analysis aims to determine the effect of the NHS website format change in January 2020, and thereby provide NHS England with feedback and recommendations to maximize the website’s efficacy.

3) The analysis hopes to give researchers and policy makers insight into what features of GPPs are most valued by patients, so that GPPs can use this information to provide better services to patients.

4) This study hopes to provide insights on how online information provision can be used to improve the quality of information provided to patients, so that patients can make better decisions about the GPP services they require and choose the best GPP to provide those services.

5) The study hopes to provide insights on how the provision of online information affects inequities in utilization of healthcare, so that the most disadvantaged patients can have improved access to the GPP services they require.

Our research is currently ongoing. We have communicated our initial findings to members of the NHS, specifically during the Economics Community Seminar at NHS England. Additionally, our preliminary results have been showcased at several economics seminars and conferences, including the NBER summer meetings in 2022 and presentations at academic venues such as HEC Paris and Essex University, among others.