Data for NHS hospital workforce retention project (determinants and effects on patients' outcomes).

The University of Surrey requires access to NHS England data for the purpose of the following research project:

Data for NHS hospital workforce retention project (determinants and effects on patients' outcomes).

The following is a summary of the aims of the research project provided by the University of Surrey:

To investigate the determinants of hospital workforce retention (HWR) and hospital staffing levels, and the effects of these variables on patient outcomes and hospital performance (i.e. quality, efficiency) measures. Workforce retention refers to the ability of an organization to retain its employees.

The study focuses on two research questions (RQ):

RQ1: What are the determinants of variations in NHS HWR, in both acute care (AC) and mental health (MH) hospitals? (what causes the differences in work force retention between different hospital settings)

RQ2: What are the causal effects of HWR on admitted patients’ health outcomes (mortality, emergency readmissions, length of stay, waiting times) in emergency, elective and MH care? (what is the impact on care).

RQ1 investigates the sources of variation in HWR for ambulance and clinical staff (doctors and nurses), i.e. which are the factors that affect the HWR within and amongst NHS hospital organizations and Ambulance Trusts, and whether different kind of factors affect differently the various categories of hospital workers (i.e. doctors, nurses and ambulance staff).

The baseline model makes use of a range of longitudinal data, including NHS E data, to uncover the association between HWR and factors determining its variations over time and across hospitals. Moreover, the variation stemming from a series of policies (i.e. the 2016 new junior doctor contract; the 2018 NHS Improvement Retention Support Programme; the 2018 hospital staff’s new pay/progression contract), political and historical events (Brexit 2016 referendum, 2021 EU withdrawal, 2020-2022 COVID-19 pandemic), together with the fact that such changes often affected unequally different groups of hospital workers, is used to estimate statistical models (e.g.: Interrupted Time Series (ITS) and difference-in-difference (DiD) models) to evaluate the effect of such ‘breaks’ on HWR. This analysis is not concerned with the identification of the retention behaviour of individual doctors, but with the identification of average retention behaviour for groups of doctors/nurses/ambulance workers amongst different NHS organization over time.

RQ2 relies on regression analysis to investigate the effect of HWR on patient outcomes, accounting for the possible endogeneity bias due to reverse causality (e.g. bad patients' outcomes leading to poor HWR). The changes in HWR caused either by policy changes or by the variation in the non-NHS wages (i.e. wages for workers with similar age/qualification profiles as NHS workers, but not working for the NHS) in the local labour market around NHS hospitals are used as sources of exogenous variation to identify plausible causal effects. This analysis is not concerned with the identification of the clinical performance of individual doctors, but with the identification of the average performance amongst NHS organisations with high vs low HWR rates and with the identification of the average performance of groups of doctors staying working in a hospital vs groups of doctors leaving a hospital, the effects of interest are at the very least aggregated at stayers vs leavers level, and never defined or reported for individual doctors.

This is a standalone project and has no links to other projects or collaborations. The project consists of two phases (related to each RQ), which will be carried out partly sequentially and partly in parallel:

Phase 1.

The first phase models the retention of the NHS hospital workforce. In this phase, the NHS England datasets requested are used to extract variables of interest that are likely factors associated with or causing changes in HWR patterns over time. The data obtained by the Department of Health and Social Care (Electronic Staff Records data, ESR) are used to define the main output variables for the analysis of the determinants of hospital workforce retention as well as some of the main variables of interest in the same analysis (e.g. average salary / gender pay gap / staff nationality / staff qualification / staff age). The ESR data are also used to define the main variables of interest (e.g. stability index, number of leavers) in the analysis of the effects of HWR on patient outcomes. The patient level data requested from NHS England are used to define some of the control variables in the analysis of the determinants of hospital workforce retention (e.g. weekly admissions to hospital, average age/comorbidities (state of having multiple medical conditions at the same time, especially when they interact with each other in some way)/procedures, number of competitors at NHS Trust level).

Phase 2.

The second phase analyses the effects of HWR on patient outcomes. In this phase, the NHS England datasets requested are used to extract variables that are either the patients’ health / process outcomes of interests (e.g. mortality, readmission, length of stay, waiting time) or characteristics of the patients (e.g. co-morbidities, age, economics deprivation, hospital attended, year, month, day of the week when admitted or treated). The patient level data requested from NHS England are used to define some of the patient-level control variables in the analysis of the effect of HWR on patient outcomes. (e.g. emergency or elective admission to hospital, patient age/comorbidities/procedures, number of competitors) as well as the main outcome variables (e.g. mortality, readmissions, length of stay, waiting times, change in the Oxford Hip/Knee score (a joint-specific, patient-reported outcome measure tool designed to assess disability in patients undergoing total hip replacement)).

This project has received approval from a Research Ethics Committee in February 2020. It was approved subject to a condition that the lay summary provided be revised suitably to be more lay friendly. This was revised, and full ethical approval was granted on August 10th 2020.

The following NHS England Data will be accessed:

• Hospital Episode Statistics

- Admitted Patient Care (HES APC)

- Critical Care (HES CC)

- Accident & Emergency (HES A&E)

• Emergency Care Data Set (ECDS)

• Civil Registrations of Death – Secondary Care Cut

• Mental Health and Learning Disabilities Data Set (MHLDDS)

• Mental Health Services Data Set (MHSDS)

• Mental Health Minimum Data Set (MHMDS)

• Patient Reported Outcome Measures (PROMS)

The data subjects are:

- all patients hospitalised in English NHS hospitals, from 2009/10 to 2025/26 for acute care, and from 2011/12 to 2025/26 for mental health care;

- the hospital consultants treating NHS inpatients for acute care and inpatients and outpatients in mental health (MH) care;

- the nurses and midwives working in NHS hospitals acute care wards and NHS mental health care hospitals (inpatients and outpatients wards);

- the NHS Ambulance Trusts workers.

The level of the Data will be pseudonymised.

The University of Surrey considered data minimisation, and provided the following summary to justify the volume of data and fields required:

The analysis is at national level and cannot be restricted by geographies. The analysis observes the full patient pathway; thus, it requires all clinical variables, episodes, dates of admissions and discharge.

The analysis needs to control for the total patients’ demand pressure on hospitals, and to compute concentration indices from patient flows as well as planned and unplanned patient readmissions to hospitals, so all diagnoses and procedures must be observed.

In order to produce a reliable analysis the projects relies on the richness of the dataset in all the fields related to the whole patient pathway - thus: Geographies, Demographics, Organization level fields, Administrative fields (i.e. episodes, episodes types, dates, diagnoses, procedures) are requested.

Historic data is required so that fixed effects can be estimated correctly and to look at impacts before/after implementation of policies.”

The lawful basis for processing personal data under the UK GDPR is:

Article 6(1)(e) - processing is necessary for the performance of a task carried out in the public interest or in the exercise of official authority vested in the controller.

The lawful basis for processing special category data under the UK GDPR is:

Article 9(2)(j) - processing is necessary for archiving purposes in the public interest, scientific or historical research purposes or statistical purposes in accordance with Article 89(1) based on Union or Member State law which shall be proportionate to the aim pursued, respect the essence of the right to data protection and provide for suitable and specific measures to safeguard the fundamental rights and the interests of the data subject.

This processing is in the public interest because it adheres to the UK Policy Framework for Health and Social Care Research, which protects and promotes the interests of patients, service users and the public, and aims to produce generalisable and publicly available information to inform future decisions over patients’ treatments or care.

The funding is provided by The Health Foundation. The funding is specifically for the study described. Funding is in place until 30th June 2025.

The funder will have no ability to suppress or otherwise limit the publication of findings.

There are no other organisations acting as a processor.

Co-investigators from the University of Leeds and City University London contribute intellectually to the writing of the reports and papers only. Co-investigators do not have access to Data provided by NHS England and cannot process data provided by NHS England.

Data will be accessed by:

- Substantive employees of the University of Surrey who have authorisation from the Principle Investigator.

- PhD students affiliated with the University of Surrey. The individuals accessing the data will do so under the supervision of a substantive employee of the University of Surrey. The University of Surrey would be responsible and liable for any work carried out by the individual. PhD students would only work on the data for the purposes described in this Data Sharing Agreement (DSA). The total number of PhD students working on the data will never be higher than 5 students, and their research work is within the remit of the research purpose stated in this Agreement.

- Individuals with honorary contracts with the University of Surrey. Honorary contract holders include: Former substantive employees of the University of Surrey who have moved to another institution; and, visiting academics who collaborate to the project and have a recognised status of honorary research fellows at the University of Surrey.

PhD students and individuals holding honorary contracts have completed mandatory data protection and confidentiality training and are subject to the same University of Surrey policies on data protection and confidentiality as substantive employees.

The study findings resulting from the data processing has contributed / will contributed to the production of:

1. Reports to the Funder / working papers;

2. Submissions to peer reviewed journals;

3. Presentations to seminars and conferences / policy briefs reports;

4. Conferences.

The research outputs are never reported at individual patient / worker level. The research outputs are always reported as aggregate quantities, e.g. the average number of patients with condition X (e.g. heart attack) across NHS hospitals, by year. Categories with small numbers of observations are suppressed / not reported.

All outputs produced using NHS England data will only contain aggregated results with small number suppression applied.

To disseminate the results of the research, the research team carries out the following activities:

- Draft non-technical blogs. The Investigators in research team have considerable experience of presenting research to varied audiences. One such example is given by the following: https://www.economicsobservatory.com/will-the-nhs-long-term-workforce-plan-solve-the-current-crisis

- Hold a launch event at the end of the first 4 years of funding. This has occurred on 19th and 20th June 2023 (https://drive.google.com/file/d/1YjdbG5oygZZJIJWbeQbjCe-oRZmmPmvP/view). A follow-up event will be organized in 2024 or 2025.

- Make use of the University of Surrey media team and press release the research work. Examples of press releases stemming from the project are:

i) https://www.surrey.ac.uk/news/keeping-nurses-happy-their-job-helps-retain-doctors-new-surrey-study-finds, which has been also reported by the Guardian (https://www.theguardian.com/commentisfree/2022/oct/23/consultants-wont-stay-in-hospitals-where-their-nurses-arent-happy) and other media;

ii) https://www.surrey.ac.uk/news/can-non-monetary-benefits-improve-nhs-nurse-retention-rates.

- Present evidence from research to experts and policymakers. During the last three years, the Principal Investigator has presented findings from this research three times at NHS England, 11 times at international conferences, and 11 times at different academic institutions (both in UK and abroad).

The team will also seek opportunities to write for other blogs both general outlets such as The Conversation and those addressed to more specialized audiences such as The Health Foundation’s own blog series.

Beyond the launch event and associated activities, the research team seeks to publish the project’s findings in high quality journals. This will establish the rigor of the research and ensure its lasting academic influence. Potential target journals are Journal of Human Resources (JHR), Economic Journal (EJ), Journal of Public Economics (JPubEcon), Journal of Health Economics (JHE), Journal of Economic Behaviour and Organization (JEBO), Health Economics (HE), Social Science and Medicine (SSM), BMJ, BMJ Open, Scientific Reports, Plos One.

The overall communications objectives are to:

1. Engage key stakeholders with the project at its inception, enabling the research team members to understand their concerns and draw on their specialist knowledge to shape the research strategy.

2. Create awareness of the research project and expertise among a broad range of interested parties.

3. Gain valuable feedback from academics and stakeholders as the project results approach their final version.

4. Disseminate the research findings widely among stakeholders, health researchers and the wider academic community.

5. Present clear and relevant policy implications to both national and local decision-makers.

In the setup phase (during the first 2 years of the project) the research team have:

- Conducted a scoping exercise to ensure the fully understanding of the key organizations (and individuals within them) interested in HWFR. This activity has helped the research team ensuring the engagement of valuable experts on the project Steering Group, such as Michelle Lee from NHS Improvement, John Stock at Health Education England, and Dr. Adrian Boyle (President of the Royal College of Emergency Medicine).

- Shared their research via team members’ Twitter (now, X) accounts.

The former activities enable the research team to engage the interest of relevant project stakeholders, and it allows the stakeholders to interact with the research team in the way that suits them best.

- Hold project Steering Group meetings. This enables the research team to form deeper connections with key stakeholders and gain valuable feedback about the proposed methodology. The first three Steering Group meetings have taken place on 15/03/2019, 02/10/2020 and 05/11/2021.

After the initial set-up phase, the research team members have:

- Continued to use Twitter, an especially helpful tool at this stage of the project. As the project networks are established, Twitter is an effective way of sharing the team’s growing involvement with stakeholders and establishing the research team as an authority on this topic.

- Presented the emerging findings at seminars and conferences. Some of the intended events are specialized in the health field (Health Economists’ Study Group, European Health Econometrics workshop, European Health Economics Association conference) or aimed at policy makers (NHS England workforce directorate internal seminars) while others are more general (Royal Economics Society and International Association for Applied Econometrics conferences) and give scope for gaining wide academic feedback.

- Made available the project’s research papers as discussion papers disseminated as through international channels (https://docs.iza.org/dp15480.pdf; https://docs.iza.org/dp15638.pdf; https://docs.iza.org/dp16379.pdf).

With regards to access to journal articles, Green open access will be provided as the base case. Gold open access will be provided subject to the will of the funder to pay for the Gold open access charges. In any case, the publications pre-prints will always be freely available to the public. Gold open access is where an author publishes their article in an online open access journal. In contrast, green open access is where an author publishes their article in any journal and then self-archives a copy in a freely accessible institutional or specialist online archive known as a repository, or on a website.

Subject to any third-party rights, the data and knowledge generated by the study will belong to the project partners, who will also:

- manage the data and knowledge produced;

- administer the access rights to the study and its results;

- together with The Health Foundation, arrange the possibility of making the publications of the study available as Open access. With respect to utilization rights, The Health Foundation has requested a license to use the work produced by the project partners for its public benefit purposes.

The Health Foundation is under an obligation to ensure that the outputs of the project are applied for the public good. Therefore, the funder has requested a license to use, for its public benefit purposes, the outputs generated by the Recipient under the Project. Subject to any third-party rights, the project partners (i.e. the Principal Investigator and co-Investigators) have granted the funder a royalty-free, non-exclusive, world-wide license to use the outputs generated by the project partners under the project for its own charitable public benefit purposes. The funder, where reasonable, will discuss with the Principal Investigator prior to using the outputs for public benefit. All outputs shared with the Funder will be aggregated with small numbers suppressed.

Relevant Target Dates:

- Submission to peer-review of at least one paper related the first research question by March 2022 (achieved). Submission to peer-review of at least one paper related to the second research question by July 2023 (achieved).

- Organization of launch event of the project by June 2023 (achieved).

- Peer-reviewed publication of as many research outcomes as possible by December 2024. Four research outputs are under review at peer-reviewed journals. [Please notice that publications in Economics have a very long turnover and so they require several years to get peer-reviewed, revised and published. A paper publication in a 4 stars peer-reviewed journal in Economics can take also several years, due to previous rejections and the time for the peer-reviewers to provide comments].

The project has achieved the following outputs to date:

1) Publication of three discussion papers:

https://docs.iza.org/dp15480.pdf

https://docs.iza.org/dp15638.pdf

https://docs.iza.org/dp16379.pdf)

2) Publication of two press releases:

https://www.surrey.ac.uk/news/can-non-monetary-benefits-improve-nhs-nurse-retention-rates

https://www.surrey.ac.uk/news/keeping-nurses-happy-their-job-helps-retain-doctors-new-surrey-study-finds

3) Publication of one non-technical blog: https://www.economicsobservatory.com/will-the-nhs-long-term-workforce-plan-solve-the-current-crisis

4) Research reported by newspapers and online blogs:

https://www.theguardian.com/commentisfree/2022/oct/23/consultants-wont-stay-in-hospitals-where-their-nurses-arent-happy

https://healthcare-in-europe.com/en/news/happy-nurses-are-key-for-doctors-retention-study-finds.html ,

https://medicalxpress.com/news/2022-03-nurses-happy-jobs-retain-doctors.html ,

https://nursingnotes.co.uk/news/research/study-finds-keeping-nurses-happy-in-their-jobs-helps-retain-senior-doctors/ ,

https://www.nursingtimes.net/news/research-and-innovation/nurse-engagement-linked-to-improved-retention-of-both-nurses-and-doctors-17-03-2022/ ,

https://rcni.com/emergency-nurse/newsroom/news/high-nurse-retention-makes-doctors-want-to-stay-their-jobs-too-183466 )

5) Presentations to policymakers; international academic conferences; and, academic seminars.

Yielded benefits to date:

- The Principle Investigator and a post-doctoral researcher in the project were invited, and are taking part in National Institute for Health Research (NIHR) funding committees as Subject Matters Experts.

- A post-doctoral researcher in the project, has secured a position as a lecturer at the University of Aberdeen also thanks to the work on the project.

- The Principle Investigator has secured two funding extensions of the project from the Funder, for the overall duration of extra 24-months - bringing the end of the funded activities of the project to the end of June 2025. This funding will support the research of two early-career researchers, one research assistant and one postdoctoral researcher.

The study team envisage more benefits will be achieved as the research is published in peer-reviewed outlets, as this might enhance trust in the research and inform policymakers, and hospital trust leader decisions in relation to improving health and social care in England and Wales. Both investigation and peer-reviewed publication are lengthy processes, which have been further delayed by several complications, including (but not limited to): COVID-19 pandemic and related work difficulties; maternity leave of one post-doctoral team member; changes in the level of commitment to the project from some of initial team members; and a resupply of Civil Registration Deaths data by NHS England in March 2023.

The University of Surrey requires access to NHS England data for the purpose of the following research project:

Data for NHS hospital workforce retention project (determinants and effects on patients' outcomes).

The following is a summary of the aims of the research project provided by the University of Surrey:

To investigate the determinants of hospital workforce retention (HWR) and hospital staffing levels, and the effects of these variables on patient outcomes and hospital performance (i.e. quality, efficiency) measures. Workforce retention refers to the ability of an organization to retain its employees.

The study focuses on two research questions (RQ):

RQ1: What are the determinants of variations in NHS HWR, in both acute care (AC) and mental health (MH) hospitals? (what causes the differences in work force retention between different hospital settings)

RQ2: What are the causal effects of HWR on admitted patients’ health outcomes (mortality, emergency readmissions, length of stay, waiting times) in emergency, elective and MH care? (what is the impact on care).

RQ1 investigates the sources of variation in HWR for ambulance and clinical staff (doctors and nurses), i.e. which are the factors that affect the HWR within and amongst NHS hospital organizations and Ambulance Trusts, and whether different kind of factors affect differently the various categories of hospital workers (i.e. doctors, nurses and ambulance staff).

The baseline model makes use of a range of longitudinal data, including NHS E data, to uncover the association between HWR and factors determining its variations over time and across hospitals. Moreover, the variation stemming from a series of policies (i.e. the 2016 new junior doctor contract; the 2018 NHS Improvement Retention Support Programme; the 2018 hospital staff’s new pay/progression contract), political and historical events (Brexit 2016 referendum, 2021 EU withdrawal, 2020-2022 COVID-19 pandemic), together with the fact that such changes often affected unequally different groups of hospital workers, is used to estimate statistical models (e.g.: Interrupted Time Series (ITS) and difference-in-difference (DiD) models) to evaluate the effect of such ‘breaks’ on HWR. This analysis is not concerned with the identification of the retention behaviour of individual doctors, but with the identification of average retention behaviour for groups of doctors/nurses/ambulance workers amongst different NHS organization over time.

RQ2 relies on regression analysis to investigate the effect of HWR on patient outcomes, accounting for the possible endogeneity bias due to reverse causality (e.g. bad patients' outcomes leading to poor HWR). The changes in HWR caused either by policy changes or by the variation in the non-NHS wages (i.e. wages for workers with similar age/qualification profiles as NHS workers, but not working for the NHS) in the local labour market around NHS hospitals are used as sources of exogenous variation to identify plausible causal effects. This analysis is not concerned with the identification of the clinical performance of individual doctors, but with the identification of the average performance amongst NHS organisations with high vs low HWR rates and with the identification of the average performance of groups of doctors staying working in a hospital vs groups of doctors leaving a hospital, the effects of interest are at the very least aggregated at stayers vs leavers level, and never defined or reported for individual doctors.

This is a standalone project and has no links to other projects or collaborations. The project consists of two phases (related to each RQ), which will be carried out partly sequentially and partly in parallel:

Phase 1.

The first phase models the retention of the NHS hospital workforce. In this phase, the NHS England datasets requested are used to extract variables of interest that are likely factors associated with or causing changes in HWR patterns over time. The data obtained by the Department of Health and Social Care (Electronic Staff Records data, ESR) are used to define the main output variables for the analysis of the determinants of hospital workforce retention as well as some of the main variables of interest in the same analysis (e.g. average salary / gender pay gap / staff nationality / staff qualification / staff age). The ESR data are also used to define the main variables of interest (e.g. stability index, number of leavers) in the analysis of the effects of HWR on patient outcomes. The patient level data requested from NHS England are used to define some of the control variables in the analysis of the determinants of hospital workforce retention (e.g. weekly admissions to hospital, average age/comorbidities (state of having multiple medical conditions at the same time, especially when they interact with each other in some way)/procedures, number of competitors at NHS Trust level).

Phase 2.

The second phase analyses the effects of HWR on patient outcomes. In this phase, the NHS England datasets requested are used to extract variables that are either the patients’ health / process outcomes of interests (e.g. mortality, readmission, length of stay, waiting time) or characteristics of the patients (e.g. co-morbidities, age, economics deprivation, hospital attended, year, month, day of the week when admitted or treated). The patient level data requested from NHS England are used to define some of the patient-level control variables in the analysis of the effect of HWR on patient outcomes. (e.g. emergency or elective admission to hospital, patient age/comorbidities/procedures, number of competitors) as well as the main outcome variables (e.g. mortality, readmissions, length of stay, waiting times, change in the Oxford Hip/Knee score (a joint-specific, patient-reported outcome measure tool designed to assess disability in patients undergoing total hip replacement)).

This project has received approval from a Research Ethics Committee in February 2020. It was approved subject to a condition that the lay summary provided be revised suitably to be more lay friendly. This was revised, and full ethical approval was granted on August 10th 2020.

The following NHS England Data will be accessed:

• Hospital Episode Statistics

- Admitted Patient Care (HES APC)

- Critical Care (HES CC)

- Accident & Emergency (HES A&E)

• Emergency Care Data Set (ECDS)

• Civil Registrations of Death – Secondary Care Cut

• Mental Health and Learning Disabilities Data Set (MHLDDS)

• Mental Health Services Data Set (MHSDS)

• Mental Health Minimum Data Set (MHMDS)

• Patient Reported Outcome Measures (PROMS)

The data subjects are:

- all patients hospitalised in English NHS hospitals, from 2009/10 to 2025/26 for acute care, and from 2011/12 to 2025/26 for mental health care;

- the hospital consultants treating NHS inpatients for acute care and inpatients and outpatients in mental health (MH) care;

- the nurses and midwives working in NHS hospitals acute care wards and NHS mental health care hospitals (inpatients and outpatients wards);

- the NHS Ambulance Trusts workers.

The level of the Data will be pseudonymised.

The University of Surrey considered data minimisation, and provided the following summary to justify the volume of data and fields required:

The analysis is at national level and cannot be restricted by geographies. The analysis observes the full patient pathway; thus, it requires all clinical variables, episodes, dates of admissions and discharge.

The analysis needs to control for the total patients’ demand pressure on hospitals, and to compute concentration indices from patient flows as well as planned and unplanned patient readmissions to hospitals, so all diagnoses and procedures must be observed.

In order to produce a reliable analysis the projects relies on the richness of the dataset in all the fields related to the whole patient pathway - thus: Geographies, Demographics, Organization level fields, Administrative fields (i.e. episodes, episodes types, dates, diagnoses, procedures) are requested.

Historic data is required so that fixed effects can be estimated correctly and to look at impacts before/after implementation of policies.”

The lawful basis for processing personal data under the UK GDPR is:

Article 6(1)(e) - processing is necessary for the performance of a task carried out in the public interest or in the exercise of official authority vested in the controller.

The lawful basis for processing special category data under the UK GDPR is:

Article 9(2)(j) - processing is necessary for archiving purposes in the public interest, scientific or historical research purposes or statistical purposes in accordance with Article 89(1) based on Union or Member State law which shall be proportionate to the aim pursued, respect the essence of the right to data protection and provide for suitable and specific measures to safeguard the fundamental rights and the interests of the data subject.

This processing is in the public interest because it adheres to the UK Policy Framework for Health and Social Care Research, which protects and promotes the interests of patients, service users and the public, and aims to produce generalisable and publicly available information to inform future decisions over patients’ treatments or care.

The funding is provided by The Health Foundation. The funding is specifically for the study described. Funding is in place until July 2024.

The funder will have no ability to suppress or otherwise limit the publication of findings.

There are no other organisations acting as a processor.

Co-investigators from the University of Leeds and City University London contribute intellectually to the writing of the reports and papers only. Co-investigators do not have access to Data provided by NHS England and cannot process data provided by NHS England.

Data will be accessed by:

- Substantive employees of the University of Surrey who have authorisation from the Principle Investigator.

- PhD students affiliated with the University of Surrey. The individuals accessing the data will do so under the supervision of a substantive employee of the University of Surrey. The University of Surrey would be responsible and liable for any work carried out by the individual. PhD students would only work on the data for the purposes described in this Data Sharing Agreement (DSA). The total number of PhD students working on the data will never be higher than 5 students, and their research work is within the remit of the research purpose stated in this Agreement.

- Individuals with honorary contracts with the University of Surrey. Honorary contract holders include: Former substantive employees of the University of Surrey who have moved to another institution; and, visiting academics who collaborate to the project and have a recognised status of honorary research fellows at the University of Surrey.

PhD students and individuals holding honorary contracts have completed mandatory data protection and confidentiality training and are subject to the same University of Surrey policies on data protection and confidentiality as substantive employees.

The study findings resulting from the data processing has contributed / will contributed to the production of:

1. Reports to the Funder / working papers;

2. Submissions to peer reviewed journals;

3. Presentations to seminars and conferences / policy briefs reports;

4. Conferences.

The research outputs are never reported at individual patient / worker level. The research outputs are always reported as aggregate quantities, e.g. the average number of patients with condition X (e.g. heart attack) across NHS hospitals, by year. Categories with small numbers of observations are suppressed / not reported.

All outputs produced using NHS England data will only contain aggregated results with small number suppression applied.

To disseminate the results of the research, the research team carries out the following activities:

- Draft non-technical blogs. The Investigators in research team have considerable experience of presenting research to varied audiences. One such example is given by the following: https://www.economicsobservatory.com/will-the-nhs-long-term-workforce-plan-solve-the-current-crisis

- Hold a launch event at the end of the first 4 years of funding. This has occurred on 19th and 20th June 2023 (https://drive.google.com/file/d/1YjdbG5oygZZJIJWbeQbjCe-oRZmmPmvP/view). A follow-up event will be organized in 2024 or 2025.

- Make use of the University of Surrey media team and press release the research work. Examples of press releases stemming from the project are:

i) https://www.surrey.ac.uk/news/keeping-nurses-happy-their-job-helps-retain-doctors-new-surrey-study-finds, which has been also reported by the Guardian (https://www.theguardian.com/commentisfree/2022/oct/23/consultants-wont-stay-in-hospitals-where-their-nurses-arent-happy) and other media;

ii) https://www.surrey.ac.uk/news/can-non-monetary-benefits-improve-nhs-nurse-retention-rates.

- Present evidence from research to experts and policymakers. During the last three years, the Principal Investigator has presented findings from this research three times at NHS England, 11 times at international conferences, and 11 times at different academic institutions (both in UK and abroad).

The team will also seek opportunities to write for other blogs both general outlets such as The Conversation and those addressed to more specialized audiences such as The Health Foundation’s own blog series.

Beyond the launch event and associated activities, the research team seeks to publish the project’s findings in high quality journals. This will establish the rigor of the research and ensure its lasting academic influence. Potential target journals are Journal of Human Resources (JHR), Economic Journal (EJ), Journal of Public Economics (JPubEcon), Journal of Health Economics (JHE), Journal of Economic Behaviour and Organization (JEBO), Health Economics (HE), Social Science and Medicine (SSM), BMJ, BMJ Open, Scientific Reports, Plos One.

The overall communications objectives are to:

1. Engage key stakeholders with the project at its inception, enabling the research team members to understand their concerns and draw on their specialist knowledge to shape the research strategy.

2. Create awareness of the research project and expertise among a broad range of interested parties.

3. Gain valuable feedback from academics and stakeholders as the project results approach their final version.

4. Disseminate the research findings widely among stakeholders, health researchers and the wider academic community.

5. Present clear and relevant policy implications to both national and local decision-makers.

In the setup phase (during the first 2 years of the project) the research team have:

- Conducted a scoping exercise to ensure the fully understanding of the key organizations (and individuals within them) interested in HWFR. This activity has helped the research team ensuring the engagement of valuable experts on the project Steering Group, such as Michelle Lee from NHS Improvement, John Stock at Health Education England, and Dr. Adrian Boyle (President of the Royal College of Emergency Medicine).

- Shared their research via team members’ Twitter (now, X) accounts.

The former activities enable the research team to engage the interest of relevant project stakeholders, and it allows the stakeholders to interact with the research team in the way that suits them best.

- Hold project Steering Group meetings. This enables the research team to form deeper connections with key stakeholders and gain valuable feedback about the proposed methodology. The first three Steering Group meetings have taken place on 15/03/2019, 02/10/2020 and 05/11/2021.

After the initial set-up phase, the research team members have:

- Continued to use Twitter, an especially helpful tool at this stage of the project. As the project networks are established, Twitter is an effective way of sharing the team’s growing involvement with stakeholders and establishing the research team as an authority on this topic.

- Presented the emerging findings at seminars and conferences. Some of the intended events are specialized in the health field (Health Economists’ Study Group, European Health Econometrics workshop, European Health Economics Association conference) or aimed at policy makers (NHS England workforce directorate internal seminars) while others are more general (Royal Economics Society and International Association for Applied Econometrics conferences) and give scope for gaining wide academic feedback.

- Made available the project’s research papers as discussion papers disseminated as through international channels (https://docs.iza.org/dp15480.pdf; https://docs.iza.org/dp15638.pdf; https://docs.iza.org/dp16379.pdf).

With regards to access to journal articles, Green open access will be provided as the base case. Gold open access will be provided subject to the will of the funder to pay for the Gold open access charges. In any case, the publications pre-prints will always be freely available to the public. Gold open access is where an author publishes their article in an online open access journal. In contrast, green open access is where an author publishes their article in any journal and then self-archives a copy in a freely accessible institutional or specialist online archive known as a repository, or on a website.

Subject to any third-party rights, the data and knowledge generated by the study will belong to the project partners, who will also:

- manage the data and knowledge produced;

- administer the access rights to the study and its results;

- together with The Health Foundation, arrange the possibility of making the publications of the study available as Open access. With respect to utilization rights, The Health Foundation has requested a license to use the work produced by the project partners for its public benefit purposes.

The Health Foundation is under an obligation to ensure that the outputs of the project are applied for the public good. Therefore, the funder has requested a license to use, for its public benefit purposes, the outputs generated by the Recipient under the Project. Subject to any third-party rights, the project partners (i.e. the Principal Investigator and co-Investigators) have granted the funder a royalty-free, non-exclusive, world-wide license to use the outputs generated by the project partners under the project for its own charitable public benefit purposes. The funder, where reasonable, will discuss with the Principal Investigator prior to using the outputs for public benefit. All outputs shared with the Funder will be aggregated with small numbers suppressed.

Relevant Target Dates:

- Submission to peer-review of at least one paper related the first research question by March 2022 (achieved). Submission to peer-review of at least one paper related to the second research question by July 2023 (achieved).

- Organization of launch event of the project by June 2023 (achieved).

- Peer-reviewed publication of as many research outcomes as possible by December 2024. Four research outputs are under review at peer-reviewed journals. [Please notice that publications in Economics have a very long turnover and so they require several years to get peer-reviewed, revised and published. A paper publication in a 4 stars peer-reviewed journal in Economics can take also several years, due to previous rejections and the time for the peer-reviewers to provide comments].

The project has achieved the following outputs to date:

1) Publication of three discussion papers:

https://docs.iza.org/dp15480.pdf

https://docs.iza.org/dp15638.pdf

https://docs.iza.org/dp16379.pdf)

2) Publication of two press releases:

https://www.surrey.ac.uk/news/can-non-monetary-benefits-improve-nhs-nurse-retention-rates

https://www.surrey.ac.uk/news/keeping-nurses-happy-their-job-helps-retain-doctors-new-surrey-study-finds

3) Publication of one non-technical blog: https://www.economicsobservatory.com/will-the-nhs-long-term-workforce-plan-solve-the-current-crisis

4) Research reported by newspapers and online blogs:

https://www.theguardian.com/commentisfree/2022/oct/23/consultants-wont-stay-in-hospitals-where-their-nurses-arent-happy

https://healthcare-in-europe.com/en/news/happy-nurses-are-key-for-doctors-retention-study-finds.html ,

https://medicalxpress.com/news/2022-03-nurses-happy-jobs-retain-doctors.html ,

https://nursingnotes.co.uk/news/research/study-finds-keeping-nurses-happy-in-their-jobs-helps-retain-senior-doctors/ ,

https://www.nursingtimes.net/news/research-and-innovation/nurse-engagement-linked-to-improved-retention-of-both-nurses-and-doctors-17-03-2022/ ,

https://rcni.com/emergency-nurse/newsroom/news/high-nurse-retention-makes-doctors-want-to-stay-their-jobs-too-183466 )

5) Presentations to policymakers; international academic conferences; and, academic seminars.

Yielded benefits to date:

- The Principle Investigator and a post-doctoral researcher in the project were invited, and are taking part in National Institute for Health Research (NIHR) funding committees as Subject Matters Experts.

- A post-doctoral researcher in the project, has secured a position as a lecturer at the University of Aberdeen also thanks to the work on the project.

- The Principle Investigator has secured a funding extension of 13-months for the project from the Funder until the end of July 2024. This funding will support the research of two early-career researchers, one research assistant and one postdoctoral researcher.

The study team envisage more benefits will be achieved as the research is published in peer-reviewed outlets, as this might enhance trust in the research and inform policymakers, and hospital trust leader decisions in relation to improving health and social care in England and Wales. Both investigation and peer-reviewed publication are lengthy processes, which have been further delayed by several complications, including (but not limited to): COVID-19 pandemic and related work difficulties; maternity leave of one post-doctoral team member; changes in the level of commitment to the project from some of initial team members; and a resupply of Civil Registration Deaths data by NHS England in March 2023.

AIM AND PURPOSE

The aim of this project is to investigate the determinants and effects of hospital workforce retention (WFR). Workforce retention refers to the ability of a workforce to retain its employees). This project is led by University of Surrey (UoS) and funded by The Health Foundation (the Funder). The project is of interest for the research team, the Funder, and the wider community of researchers and healthcare policy-makers, with an expected positive impact on the knowledge of the economics of healthcare workforce and its effects on hospital performance and patients’ outcomes. The added contribution generated by the project is hoped to help improve the sustainability of the English NHS.

The lawful basis for processing personal data is Article 6(1)(e), in that processing is necessary for the performance of a task carried out in the public interest or in the exercise of official authority vested in the controller. The lawful basis for processing Special Category Data is Article (9)(2)(j), in that processing is necessary for scientific research purposes in accordance with Article 89(1). The University of Surrey is a public authority responsible for conducting scientific research for academic and public benefit. Data in the ‘Hospital workforce retention and patient outcomes’ study is processed to enable the University of Surrey to perform its public task. The University of Surrey rely on GDPR Article 6 (1) (e), to carry out its public task and for special categories of data (including health information and pathways, and information concerning ethnicity); GDPR Article 9.2(j), for archiving, research and statistics, as the study is a research project which will use data and statistics, in accordance with Article 89(1).

The following data products are requested:

1) Hospital Episode Statistics (HES) Admitted Patient Care: Financial Years 2009/10 to 2021/22;

2) Patient Reported Outcome Measures (PROMs): Financial Years 2009/10 to 2021/22;

3) Civil Registration - Deaths (CR-D): Financial Years 2009/10 to 2021/22;

4) HES Critical Care (HES CC): Financial Years 2017/18 to 2021/22;

5) HES Accidents and Emergencies (HES A&E): Financial Years 2009/10 to 2018/19;

6) Emergency Care Data Set (ECDS): Financial Years 2018/19 to 2021/22;

7) Mental Health Minimum DataSet (MHMDS), Mental Health Learning Disabilities (MHLDDS), Mental Health Services DataSet (MHSDS): Financial Years 2011/12 to 2021/22;

8) Bridge files: Hospital Episode Statistics to Mental Health Minimum Data Set

9) Bridge files: Hospital Episode Statistics to Mental Health Services Data Set

10) Mapping files: MHSDS to MHMDS / MHLDDS

11) Bridge files: PROMS to Hospital Episode Statistics

The data requested will allow UoS to investigate the association of hospital WFR for different categories of hospital workers. These workers include consultants, nurses, ambulance staff with patients’ outcomes in different type of hospital care, length of stay in acute emergency care, unplanned re-admissions in acute elective care, and unplanned re-admission to inpatient mental health wards for mental health care.

The study will focus on two research questions (RQ).

RQ1: What are the determinants of variations in NHS WFR, in both acute care (AC) and mental health (MH) hospitals? (what causes the differences in work force retention between different hospital settings)

RQ2: What are the causal effects of WFR on admitted patients’ health outcomes (mortality, emergency readmissions, length of stay, waiting times) in emergency, elective and MH care? (what is the impact on care).

RQ1 investigates the sources of variation in hospital WFR for ambulance and clinical staff (doctors and nurses), i.e. which are the factors that affect the hospital WFR within and amongst NHS hospital organizations and Ambulance Trusts, and whether different kind of factors affect differently the various categories of hospital workers (i.e. doctors, nurses and ambulance staff).

The baseline model will make use of a range of longitudinal data, including NHS Digital data, to uncover the association between WFR and factors determining its variations over time and across hospitals. Moreover, the variation stemming from a series of policies (i.e. the 2016 new junior doctor contract; the 2018 NHS Improvement Retention Support Programme; the 2018 hospital staff’s new pay/progression contract) and political events (Brexit 2016 poll and 2019 EU withdrawal) and the fact that such changes often affected unequally different groups of hospital workers, will be used to estimate statistical models (e.g.: Interrupted Time Series (ITS) and difference-in-difference (DiD) models) to evaluate the effect of such ‘breaks’ on hospital WFR. This analysis is not concerned with the identification of the retention behaviour of individual doctors, but with the identification of average retention behaviour for groups of doctors/nurses/ambulance workers amongst different NHS organization over time.

RQ2 will rely on a simple (linear regression) analysis to uncover how the effect of WFR on patients' outcomes can lead to estimates biased by endogeneity due to reverse causality (e.g. bad patients' outcomes leading to poor WFR), so the changes in hospital WFR caused either by policy changes and the Brexit shock or by the variation in the non-NHS wages (i.e. wages for workers with similar age/qualification profiles as NHS workers, but not working for the NHS) and business turnover in the local labour market around NHS hospitals. The aim is to plausibly identify causal effects.

This analysis is not concerned with the identification of the clinical performance of individual doctors, but with the identification of the average performance amongst NHS organisations with high vs low WFR rates and with the identification of the average performance of groups of doctors staying working in a hospital vs groups of doctors leaving a hospital, the effects of interest are at the very least aggregated at stayers vs leavers level, and never defined or reported for individual doctors.

This is a standalone project and has no links to other projects or collaborations. The project consists of two phases (related to each RQ), which will be carried out partly sequentially and partly in parallel.

Phase 1.

The first phase will model the retention of the NHS hospital workforce. In this phase, the NHSD datasets requested will be used to extract variables of interest that are likely to be factors associated with or causing changes in hospital WFR patterns over time. The data obtained by the Department of Health and Social Care (Electronic Staff Records data, ESR) will be used to define the main output variables for the analysis of the determinants of hospital workforce retention as well as some of the main variables of interest in the same analysis (e.g. average salary / gender pay gap / staff nationality / staff qualification / staff age). The ESR data will also be used to define the main variables of interest (e.g. stability index, number of leavers) in the analysis of the effects of hospital WFR on patient outcomes. The patient level data requested from NHSD will be used to define some of the control variables in the analysis of the determinants of hospital workforce retention (e.g. weekly admissions to hospital, average age/comorbidities (state of having multiple medical conditions at the same time, especially when they interact with each other in some way)/procedures, number of competitors at NHS Trust level).

Phase 2.

The second phase will analyse the effects of hospital WFR on patients’ outcomes. In this phase, the NHSD datasets requested will be used to extract variables that are either the patients’ health / process outcomes of interests (e.g. mortality, readmission, length of stay, waiting time) or characteristics of the patients (e.g. co-morbidities, age, economics deprivation, hospital attended, year, month, day of the week when admitted or treated).

The patient level data requested from NHSD will be used to define some of the patient-level control variables in the analysis of the effect of hospital WFR on patient outcomes. (e.g. emergency or elective admission to hospital, patient age/comorbidities/procedures, number of competitors) as well as the main outcome variables (e.g. mortality, readmissions, length of stay, waiting times, change in the Oxford Hip/Knee score ( a joint-specific, patient-reported outcome measure tool designed to assess disability in patients undergoing total hip replacement)).

The most important academic references to this project are the following published studies:

Propper, C., & Van Reenen, J. (2010). Can pay regulation kill? Panel data evidence on the effect of labour markets on hospital performance. Journal of Political Economy, 118(2), 222-273].

Shields, M. A., & Ward, M. (2001). Improving nurse retention in the National Health Service in England: the impact of job satisfaction on intentions to quit. Journal of health economics, 20(5), 677-701.

Newman, K., & Maylor, U. (2002). The NHS Plan: nurse satisfaction, commitment and retention strategies. Health Services Management Research, 15(2), 93-105.

Other important healthcare policy references on the economics of the NHS healthcare workforce are:

Health Education England. (2017). Facing the Facts, Shaping the Future. A draft health and care workforce strategy for England to 2027.

Charlesworth, A., & Lafond, S. (2017). Shifting from Undersupply to Oversupply: Does NHS Workforce Planning Need a Paradigm Shift?. Economic Affairs, 37(1), 36-52.

Buchan, J., Charlesworth, A., Gershlick, B., & Seccombe, I. (2017). Rising pressure: the NHS workforce challenge.

Nuffield Trust (2017). Creating a sustainable workforce: The long-term sustainability of the NHS.

PARTICIPANTS:

- all patients hospitalized in English NHS hospitals, from 2009/10 to 2021/22 for acute care, and from 2011/12 to 2021/22 for mental health care;

- the hospital consultants treating NHS inpatients for acute care and inpatients and outpatients in mental health (MH) care;

- the nurses working in NHS hospitals acute care wards and NHS mental health care hospitals (inpatients and outpatients wards);

- the NHS Ambulance Trusts workers.

The data requested from NHS Digital is related only to the patients admitted to acute and/or MH care, as well as the hospital consultant codes of the consultants treating the patients in hospitals.

For all data products requested, the full datasets are required, including all admissions to hospital care (& community care for MH patients). The data needed to deliver the project cannot be limited to a cohort of patients with a specific condition, procedure or age range and there are several reasons for this. To study both the determinant factors of hospital WFR and its effects on hospital patient outcomes, data is needed to:

1) Create variables to proxy healthcare demand pressure at provider-level (or at department-level within each provider). These variables depend on the sum of all admissions recorded, and not by a single cohort of patients.

2) Define clinically and policy-makers relevant health outcomes like emergency readmissions to hospital following a previous hospital discharge, where the diagnosis for the readmission spell need not be the same as the diagnosis of the index admission spell. This requires having records for all the patients.

3) Define market concentration variables for non-emergency admissions, e.g. the HHI index. The Herfindahl-Hirschman Index (HHI) is a commonly accepted measure of market concentration - measure of the size of firms in relation to the industry and an indicator of the amount of competition among them. Computation at provider level requires to observe the spectrum of all non-emergency admissions in a given year or month for all providers in England, and so it requires the records of all elective patients in England.

4) Define clinically relevant case-mix variables to control for patient severity like the number of emergency admissions to hospital within a given period (e.g. one year, two years), where the diagnosis for the emergency admission can be of any type. This requires observing records for all the emergency patients.

The University of Surrey is the sole data controller and the sole data processor for this agreement.

There are co-investigators from University of Leeds and City University London involved in this project. The University of Leeds and City University London do not have access to or process NHS Digital data and will only contribute to the writing of the reports and papers. The co- investigators belonging to these organizations are only contributing intellectually and in the writing of reports/papers to the project. But they are not involved in determining the means by which the data are being processed.

FUNDER

The Health Foundation is funding this project and their role is to ensure and facilitate the delivery of this project, but The Health Foundation has no data controlling or data processing roles within the project.

The Health Foundation is a major stakeholder in the project and it has funded this research project, along with several other projects from other institutions, under a funding call for their Efficiency Research Programme (https://www.health.org.uk/sites/default/files/ERP%202018%20Call%20for%20applications.pdf) which is targeted to investigate the under-researched themes of labour productivity and workforce retention in health and social care. As such, the remit of RQ1 and RQ2 of this project fall under the remit of the funding call issued by the Funder.

The Funder organises an Advisory meeting for the research projects which facilitates the circulation of ideas among researchers, their collaboration and so the development of the research project. The Funder is a very known think-tank in the UK and has an extensive network of professionals that supports the public good and public health mission. As such, using its network, The Health Foundation is going to help the research team circulate the findings of the research, prior to and along with any other dissemination channel (e.g. peer-reviewed journals, conferences, etc…).

The Health Foundation has no control of the data that is released by NHS Digital.

The Health Foundation will have access to research outputs, aggregated with small numbers suppressed, in terms of graphs, tables and paper to be produced by the UoS research team, which will not be able to be published or used without the UoS research team’s explicit consent.

The Health Foundation will act as an additional dissemination channel, e.g. similarly to posting a working paper from the project on the Surrey project website.

Ethical Approval.

This project has received approval from a Research Ethics Committee in February 2020. It was approved subject to a condition that the lay summary provided be revised suitably to be more lay friendly. This was revised, and full ethical approval was granted on August 10th 2020.

The study findings resulting from the data processing will contribute to the production of:

1. Reports to the Funder / working papers;

2. Submissions to peer reviewed journals;

3. Presentations to seminars and conferences / policy briefs reports;

4. Conferences.

The research outputs will never be reported at individual patient / worker level. The research outputs will always be reported as aggregate quantities, e.g. the average number of patients with condition X (e.g. heart attack) across NHS hospitals, by year. Categories with small numbers of observations will be suppressed / not reported.

All outputs that will be produced using the NHS Digital data will only contain aggregated results with small number suppression applied.

To disseminate the results of the research, the research team will:

- Draft at least two non-technical briefing papers (one on the factors affecting staff retention and the other on its consequences for patient welfare). The Investigators in research team have considerable experience of presenting research to varied audiences.

- Hold a launch event at the end of the 4 years of funding, inviting the project’s key stakeholders and wider networks, including representatives of individual Trusts. This will be timed to coincide with the actions at the point below.

- Publication of non-technical papers on the project’s website and through Twitter. They will be accompanied by a blog and animation.

- The research team will make use of the University of Surrey media team and press release the research work. This press release team assisted in writing and placing the article on the free entitlement in The Daily Telegraph [2]. The team will also seek opportunities to write for other blogs both general outlets such as The Conversation and those addressed to more specialized audiences such as The Health Foundation’s own blog series. The research team is also willing to provide guidance to organisations, such as Trusts through NHS Improvement. One of the co-Is, has written NICE guidelines so he has experience of turning research results into a specific product.

Beyond the launch event and associated activities, the research team will seek to publish the project’s findings in high quality journals. This will establish the rigor of the research and ensure its lasting academic influence. Potential target journals are Journal of Human Resources (JHR), Economic Journal (EJ), Journal of Public Economics (JPubEcon), Journal of Health Economics (JHE), Journal of Economic Behaviour and Organization (JEBO), Health Economics (HE), Social Science and Medicine (SSM).

[1] Cookson, R. and Moscelli, G. (2018) Are Angioplasty Waiting Times Growing Again. Centre for Health Economics. https://www.york.ac.uk/media/che/documents/policybriefing/Angioplasty.pdf

[2] Blanden, J. (2016). X-Factor Over Evidence: The Failure of Early Years’ Education. The Daily Telegraph, 22nd October. https://www.telegraph.co.uk/education/educationopinion/11177381/X-Factor-over-evidence-the-failure-of-early-years-education.html

The overall communications objectives are to:

1. Engage key stakeholders with the project at its inception, enabling the research team members to understand their concerns and draw on their specialist knowledge to shape the research strategy.

2. Create awareness of the research project and expertise among a broad range of interested parties.

3. Gain valuable feedback from academics and stakeholders as the project results approach their final version.

4. Disseminate the research findings widely among stakeholders, health researchers and the wider academic community.

5. Present clear and relevant policy implications to both national and local decision-makers.

In the setup phase (during the first 2 years of the project) the research team will:

- Conduct a scoping exercise to ensure the fully understanding of the key organizations (and individuals within them) interested in HWFR. This will help the research team ensure they are inviting exactly the right people to be on the project Steering Group. It will also grow the project wider network by subscribing to the right mailing lists and following the right Twitter feeds to be appraised of relevant events. The research team will also contact the most important individuals by email. [This activity has already taken place over the course of Summer and Fall 2019]

- Set up the project website at the University of Surrey, using as potential models the websites of previous research projects like Better for Less (https://www.surrey.ac.uk/better-for-less) and the Centre for Vocational Education Research (http://cver.lse.ac.uk/), which saw the involvement of one team-member of this project.

- Establish a Twitter account. The research team can share the blog through this channel as well as using it to provide brief comment and link the project to related activity.

The former activities will enable the research team to engage interested parties at the start of the project, and it allows these parties to interact with the research team in the way that suits them best.

- Form and hold the first meeting of the project Steering Group. This enables the research team to form deeper connections with key stakeholders and gain valuable feedback about the proposed methodology. Members will encourage the research team to think of ways of addressing their concerns and are likely to be able to provide valuable information about institutions, policy detail and data. The first project Steering Group has already taken place on 15/03/2019. During such meeting, the Investigators have received valuable suggestions how to set up the analysis and which data may be useful for the investigation.

After the initial set-up phase, the research team will sustain interest in the research without over-burdening its audience. In this phase the research team will:

- Continue to use Twitter, an especially helpful tool at this stage of the project. As the project networks are established, Twitter is an effective way of sharing the team’s growing involvement with stakeholders and establishing the research team as an authority on this topic.

- Update the website and add blogs if appropriate.

- Continue the meetings with the Steering Group. These will provide the research team with valuable feedback as the research findings begin to emerge and will enable the research team to discuss potential robustness checks and extensions. The Steering Group’s input will also be invaluable in helping to understand the results and their policy implications, particularly as the part of the Cost-Benefit analysis is approached.

- Begin to present the emerging findings at seminars and conferences. Some of the intended events are specialized in the health field (Health Economists’ Study Group, European Health Econometrics workshop, European Health Economics Association conference) or aimed at policy makers (DHSC analytical lunchtime seminars) while others are more general (Royal Economics Society and International Association for Applied Econometrics conferences) and give scope for gaining wide academic feedback.

- The project’s research papers will be made available as Discussion Papers on the project’s website once they are complete. The work on the determinants of staff retention will be completed first, in accordance with the research agenda and scheduled plan.

In regards to access to journal articles - Green open access will be provided as the base case. Gold open access will be provided subject to the will of the funder to pay for the Gold open access charges. In any case, the publications pre-prints will always be freely available to the public. Gold open access is where an author publishes their article in an online open access journal. In contrast, green open access is where an author publishes their article in any journal and then self-archives a copy in a freely accessible institutional or specialist online archive known as a repository, or on a website.

Subject to any third party rights, the data and knowledge generated by the study will belong to the project partners (from UoS, University of Leeds and City University London), who will also:

- manage the data and knowledge produced;

- administer the access rights to the study and its results;

- together with The Health Foundation, arrange the possibility of making the publications of the study available as Open access. With respect to utilization rights, The Health Foundation has requested a license to use the work produced by the project partners for its public benefit purposes.

The Health Foundation is under an obligation to ensure that the outputs of the project are applied for the public good. Therefore, the funder has requested a license to use, for its public benefit purposes, the outputs generated by the Recipient under the Project. Subject to any third party rights, the project partners (i.e. the Principal Investigator and co-Investigators) have granted the funder a royalty-free, non-exclusive, world-wide license to use the outputs generated by the project partners under the project for its own charitable public benefit purposes. The funder, where reasonable, will discuss with the Principal Investigator prior to using the outputs for public benefit. All outputs shared with the Funder will be aggregated with small numbers suppressed.

Relevant Target Dates:

Submission to peer-review of at least one paper related the first research question by December 2021;

Submission to peer-review of at least one paper related to the second research question by June 2023;

Organization of launch event of the project by June 2023;

Publication of at least 60% of the outcomes of the project by December 2023. [Please notice that publications in Economics have a very long turnover and so they require several years to get peer-reviewed, revised and published. A paper publication in a 4 stars peer-reviewed journal in Economics can take also several years, due to previous rejections and the time for the peer-reviewers to provide comments].

EU funding is not applicable.

AIM AND PURPOSE

The aim of this project is to investigate the determinants and effects of hospital workforce retention (WFR). Workforce retention refers to the ability of a workforce to retain its employees). This project is led by University of Surrey (UoS) and funded by The Health Foundation (the Funder). The project is of interest for the research team, the Funder, and the wider community of researchers and healthcare policy-makers, with an expected positive impact on the knowledge of the economics of healthcare workforce and its effects on hospital performance and patients’ outcomes. The added contribution generated by the project is hoped to help improve the sustainability of the English NHS.

The lawful basis for processing personal data is Article 6(1)(e), in that processing is necessary for the performance of a task carried out in the public interest or in the exercise of official authority vested in the controller. The lawful basis for processing Special Category Data is Article (9)(2)(j), in that processing is necessary for scientific research purposes in accordance with Article 89(1). The University of Surrey is a public authority responsible for conducting scientific research for academic and public benefit. Data in the ‘Hospital workforce retention and patient outcomes’ study is processed to enable the University of Surrey to perform its public task. The University of Surrey rely on GDPR Article 6 (1) (e), to carry out its public task and for special categories of data (including health information and pathways, and information concerning ethnicity); GDPR Article 9.2(j), for archiving, research and statistics, as the study is a research project which will use data and statistics, in accordance with Article 89(1).

The following data products are requested:

1) Hospital Episode Statistics (HES) Admitted Patient Care: Financial Years 2009/10 to 2021/22;

2) Patient Reported Outcome Measures (PROMs): Financial Years 2009/10 to 2021/22;

3) Civil Registration - Deaths (CR-D): Financial Years 2009/10 to 2021/22;

4) HES Critical Care (HES CC): Financial Years 2017/18 to 2021/22;

5) HES Accidents and Emergencies (HES A&E): Financial Years 2009/10 to 2018/19;

6) Emergency Care Data Set (ECDS): Financial Years 2018/19 to 2021/22;

7) Mental Health Minimum DataSet (MHMDS), Mental Health Learning Disabilities (MHLDDS), Mental Health Services DataSet (MHSDS): Financial Years 2011/12 to 2021/22;

8) Bridge files: Hospital Episode Statistics to Mental Health Minimum Data Set

9) Bridge files: Hospital Episode Statistics to Mental Health Services Data Set

10) Mapping files: MHSDS to MHMDS / MHLDDS

11) Bridge files: PROMS to Hospital Episode Statistics

The data requested will allow UoS to investigate the association of hospital WFR for different categories of hospital workers. These workers include consultants, nurses, ambulance staff with patients’ outcomes in different type of hospital care, length of stay in acute emergency care, unplanned re-admissions in acute elective care, and unplanned re-admission to inpatient mental health wards for mental health care.

The study will focus on two research questions (RQ).

RQ1: What are the determinants of variations in NHS WFR, in both acute care (AC) and mental health (MH) hospitals? (what causes the differences in work force retention between different hospital settings)

RQ2: What are the causal effects of WFR on admitted patients’ health outcomes (mortality, emergency readmissions, length of stay, waiting times) in emergency, elective and MH care? (what is the impact on care).

RQ1 investigates the sources of variation in hospital WFR for ambulance and clinical staff (doctors and nurses), i.e. which are the factors that affect the hospital WFR within and amongst NHS hospital organizations and Ambulance Trusts, and whether different kind of factors affect differently the various categories of hospital workers (i.e. doctors, nurses and ambulance staff).

The baseline model will make use of a range of longitudinal data, including NHS Digital data, to uncover the association between WFR and factors determining its variations over time and across hospitals. Moreover, the variation stemming from a series of policies (i.e. the 2016 new junior doctor contract; the 2018 NHS Improvement Retention Support Programme; the 2018 hospital staff’s new pay/progression contract) and political events (Brexit 2016 poll and 2019 EU withdrawal) and the fact that such changes often affected unequally different groups of hospital workers, will be used to estimate statistical models (e.g.: Interrupted Time Series (ITS) and difference-in-difference (DiD) models) to evaluate the effect of such ‘breaks’ on hospital WFR. This analysis is not concerned with the identification of the retention behaviour of individual doctors, but with the identification of average retention behaviour for groups of doctors/nurses/ambulance workers amongst different NHS organization over time.

RQ2 will rely on a simple (linear regression) analysis to uncover how the effect of WFR on patients' outcomes can lead to estimates biased by endogeneity due to reverse causality (e.g. bad patients' outcomes leading to poor WFR), so the changes in hospital WFR caused either by policy changes and the Brexit shock or by the variation in the non-NHS wages (i.e. wages for workers with similar age/qualification profiles as NHS workers, but not working for the NHS) and business turnover in the local labour market around NHS hospitals. The aim is to plausibly identify causal effects.

This analysis is not concerned with the identification of the clinical performance of individual doctors, but with the identification of the average performance amongst NHS organisations with high vs low WFR rates and with the identification of the average performance of groups of doctors staying working in a hospital vs groups of doctors leaving a hospital, the effects of interest are at the very least aggregated at stayers vs leavers level, and never defined or reported for individual doctors.

This is a standalone project and has no links to other projects or collaborations. The project consists of two phases (related to each RQ), which will be carried out partly sequentially and partly in parallel.

Phase 1.

The first phase will model the retention of the NHS hospital workforce. In this phase, the NHSD datasets requested will be used to extract variables of interest that are likely to be factors associated with or causing changes in hospital WFR patterns over time. The data obtained by the Department of Health and Social Care (Electronic Staff Records data, ESR) will be used to define the main output variables for the analysis of the determinants of hospital workforce retention as well as some of the main variables of interest in the same analysis (e.g. average salary / gender pay gap / staff nationality / staff qualification / staff age). The ESR data will also be used to define the main variables of interest (e.g. stability index, number of leavers) in the analysis of the effects of hospital WFR on patient outcomes. The patient level data requested from NHSD will be used to define some of the control variables in the analysis of the determinants of hospital workforce retention (e.g. weekly admissions to hospital, average age/comorbidities (state of having multiple medical conditions at the same time, especially when they interact with each other in some way)/procedures, number of competitors at NHS Trust level).

Phase 2.

The second phase will analyse the effects of hospital WFR on patients’ outcomes. In this phase, the NHSD datasets requested will be used to extract variables that are either the patients’ health / process outcomes of interests (e.g. mortality, readmission, length of stay, waiting time) or characteristics of the patients (e.g. co-morbidities, age, economics deprivation, hospital attended, year, month, day of the week when admitted or treated).

The patient level data requested from NHSD will be used to define some of the patient-level control variables in the analysis of the effect of hospital WFR on patient outcomes. (e.g. emergency or elective admission to hospital, patient age/comorbidities/procedures, number of competitors) as well as the main outcome variables (e.g. mortality, readmissions, length of stay, waiting times, change in the Oxford Hip/Knee score ( a joint-specific, patient-reported outcome measure tool designed to assess disability in patients undergoing total hip replacement)).

The most important academic references to this project are the following published studies:

Propper, C., & Van Reenen, J. (2010). Can pay regulation kill? Panel data evidence on the effect of labour markets on hospital performance. Journal of Political Economy, 118(2), 222-273].

Shields, M. A., & Ward, M. (2001). Improving nurse retention in the National Health Service in England: the impact of job satisfaction on intentions to quit. Journal of health economics, 20(5), 677-701.

Newman, K., & Maylor, U. (2002). The NHS Plan: nurse satisfaction, commitment and retention strategies. Health Services Management Research, 15(2), 93-105.

Other important healthcare policy references on the economics of the NHS healthcare workforce are:

Health Education England. (2017). Facing the Facts, Shaping the Future. A draft health and care workforce strategy for England to 2027.

Charlesworth, A., & Lafond, S. (2017). Shifting from Undersupply to Oversupply: Does NHS Workforce Planning Need a Paradigm Shift?. Economic Affairs, 37(1), 36-52.

Buchan, J., Charlesworth, A., Gershlick, B., & Seccombe, I. (2017). Rising pressure: the NHS workforce challenge.

Nuffield Trust (2017). Creating a sustainable workforce: The long-term sustainability of the NHS.

PARTICIPANTS:

- all patients hospitalized in English NHS hospitals, from 2009/10 to 2021/22 for acute care, and from 2011/12 to 2021/22 for mental health care;

- the hospital consultants treating NHS inpatients for acute care and inpatients and outpatients in mental health (MH) care;

- the nurses working in NHS hospitals acute care wards and NHS mental health care hospitals (inpatients and outpatients wards);

- the NHS Ambulance Trusts workers.

The data requested from NHS Digital is related only to the patients admitted to acute and/or MH care, as well as the hospital consultant codes of the consultants treating the patients in hospitals.

For all data products requested, the full datasets are required, including all admissions to hospital care (& community care for MH patients). The data needed to deliver the project cannot be limited to a cohort of patients with a specific condition, procedure or age range and there are several reasons for this. To study both the determinant factors of hospital WFR and its effects on hospital patient outcomes, data is needed to:

1) Create variables to proxy healthcare demand pressure at provider-level (or at department-level within each provider). These variables depend on the sum of all admissions recorded, and not by a single cohort of patients.

2) Define clinically and policy-makers relevant health outcomes like emergency readmissions to hospital following a previous hospital discharge, where the diagnosis for the readmission spell need not be the same as the diagnosis of the index admission spell. This requires having records for all the patients.

3) Define market concentration variables for non-emergency admissions, e.g. the HHI index. The Herfindahl-Hirschman Index (HHI) is a commonly accepted measure of market concentration - measure of the size of firms in relation to the industry and an indicator of the amount of competition among them. Computation at provider level requires to observe the spectrum of all non-emergency admissions in a given year or month for all providers in England, and so it requires the records of all elective patients in England.

4) Define clinically relevant case-mix variables to control for patient severity like the number of emergency admissions to hospital within a given period (e.g. one year, two years), where the diagnosis for the emergency admission can be of any type. This requires observing records for all the emergency patients.

The University of Surrey is the sole data controller and the sole data processor for this agreement.

There are co-investigators from University of Leeds and City University London involved in this project. The University of Leeds and City University London do not have access to or process NHS Digital data and will only contribute to the writing of the reports and papers. The co- investigators belonging to these organizations are only contributing intellectually and in the writing of reports/papers to the project. But they are not involved in determining the means by which the data are being processed.

FUNDER

The Health Foundation is funding this project and their role is to ensure and facilitate the delivery of this project, but The Health Foundation has no data controlling or data processing roles within the project.

The Health Foundation is a major stakeholder in the project and it has funded this research project, along with several other projects from other institutions, under a funding call for their Efficiency Research Programme (https://www.health.org.uk/sites/default/files/ERP%202018%20Call%20for%20applications.pdf) which is targeted to investigate the under-researched themes of labour productivity and workforce retention in health and social care. As such, the remit of RQ1 and RQ2 of this project fall under the remit of the funding call issued by the Funder.

The Funder organises an Advisory meeting for the research projects which facilitates the circulation of ideas among researchers, their collaboration and so the development of the research project. The Funder is a very known think-tank in the UK and has an extensive network of professionals that supports the public good and public health mission. As such, using its network, The Health Foundation is going to help the research team circulate the findings of the research, prior to and along with any other dissemination channel (e.g. peer-reviewed journals, conferences, etc…).

The Health Foundation has no control of the data that is released by NHS Digital.

The Health Foundation will have access to research outputs, aggregated with small numbers suppressed, in terms of graphs, tables and paper to be produced by the UoS research team, which will not be able to be published or used without the UoS research team’s explicit consent.

The Health Foundation will act as an additional dissemination channel, e.g. similarly to posting a working paper from the project on the Surrey project website.

Ethical Approval.

This project has received approval from a Research Ethics Committee in February 2020. It was approved subject to a condition that the lay summary provided be revised suitably to be more lay friendly. This was revised, and full ethical approval was granted on August 10th 2020.

The study findings resulting from the data processing will contribute to the production of:

1. Reports to the Funder / working papers;

2. Submissions to peer reviewed journals;

3. Presentations to seminars and conferences / policy briefs reports;

4. Conferences.

The research outputs will never be reported at individual patient / worker level. The research outputs will always be reported as aggregate quantities, e.g. the average number of patients with condition X (e.g. heart attack) across NHS hospitals, by year. Categories with small numbers of observations will be suppressed / not reported.

All outputs that will be produced using the NHS Digital data will only contain aggregated results with small number suppression applied.

To disseminate the results of the research, the research team will:

- Draft at least two non-technical briefing papers (one on the factors affecting staff retention and the other on its consequences for patient welfare). The Investigators in research team have considerable experience of presenting research to varied audiences.

- Hold a launch event at the end of the 4 years of funding, inviting the project’s key stakeholders and wider networks, including representatives of individual Trusts. This will be timed to coincide with the actions at the point below.

- Publication of non-technical papers on the project’s website and through Twitter. They will be accompanied by a blog and animation.

- The research team will make use of the University of Surrey media team and press release the research work. This press release team assisted in writing and placing the article on the free entitlement in The Daily Telegraph [2]. The team will also seek opportunities to write for other blogs both general outlets such as The Conversation and those addressed to more specialized audiences such as The Health Foundation’s own blog series. The research team is also willing to provide guidance to organisations, such as Trusts through NHS Improvement. One of the co-Is, has written NICE guidelines so he has experience of turning research results into a specific product.

Beyond the launch event and associated activities, the research team will seek to publish the project’s findings in high quality journals. This will establish the rigor of the research and ensure its lasting academic influence. Potential target journals are Journal of Human Resources (JHR), Economic Journal (EJ), Journal of Public Economics (JPubEcon), Journal of Health Economics (JHE), Journal of Economic Behaviour and Organization (JEBO), Health Economics (HE), Social Science and Medicine (SSM).

[1] Cookson, R. and Moscelli, G. (2018) Are Angioplasty Waiting Times Growing Again. Centre for Health Economics. https://www.york.ac.uk/media/che/documents/policybriefing/Angioplasty.pdf

[2] Blanden, J. (2016). X-Factor Over Evidence: The Failure of Early Years’ Education. The Daily Telegraph, 22nd October. https://www.telegraph.co.uk/education/educationopinion/11177381/X-Factor-over-evidence-the-failure-of-early-years-education.html

The overall communications objectives are to:

1. Engage key stakeholders with the project at its inception, enabling the research team members to understand their concerns and draw on their specialist knowledge to shape the research strategy.

2. Create awareness of the research project and expertise among a broad range of interested parties.

3. Gain valuable feedback from academics and stakeholders as the project results approach their final version.

4. Disseminate the research findings widely among stakeholders, health researchers and the wider academic community.

5. Present clear and relevant policy implications to both national and local decision-makers.

In the setup phase (during the first 2 years of the project) the research team will:

- Conduct a scoping exercise to ensure the fully understanding of the key organizations (and individuals within them) interested in HWFR. This will help the research team ensure they are inviting exactly the right people to be on the project Steering Group. It will also grow the project wider network by subscribing to the right mailing lists and following the right Twitter feeds to be appraised of relevant events. The research team will also contact the most important individuals by email. [This activity has already taken place over the course of Summer and Fall 2019]

- Set up the project website at the University of Surrey, using as potential models the websites of previous research projects like Better for Less (https://www.surrey.ac.uk/better-for-less) and the Centre for Vocational Education Research (http://cver.lse.ac.uk/), which saw the involvement of one team-member of this project.

- Establish a Twitter account. The research team can share the blog through this channel as well as using it to provide brief comment and link the project to related activity.

The former activities will enable the research team to engage interested parties at the start of the project, and it allows these parties to interact with the research team in the way that suits them best.

- Form and hold the first meeting of the project Steering Group. This enables the research team to form deeper connections with key stakeholders and gain valuable feedback about the proposed methodology. Members will encourage the research team to think of ways of addressing their concerns and are likely to be able to provide valuable information about institutions, policy detail and data. The first project Steering Group has already taken place on 15/03/2019. During such meeting, the Investigators have received valuable suggestions how to set up the analysis and which data may be useful for the investigation.

After the initial set-up phase, the research team will sustain interest in the research without over-burdening its audience. In this phase the research team will:

- Continue to use Twitter, an especially helpful tool at this stage of the project. As the project networks are established, Twitter is an effective way of sharing the team’s growing involvement with stakeholders and establishing the research team as an authority on this topic.

- Update the website and add blogs if appropriate.

- Continue the meetings with the Steering Group. These will provide the research team with valuable feedback as the research findings begin to emerge and will enable the research team to discuss potential robustness checks and extensions. The Steering Group’s input will also be invaluable in helping to understand the results and their policy implications, particularly as the part of the Cost-Benefit analysis is approached.

- Begin to present the emerging findings at seminars and conferences. Some of the intended events are specialized in the health field (Health Economists’ Study Group, European Health Econometrics workshop, European Health Economics Association conference) or aimed at policy makers (DHSC analytical lunchtime seminars) while others are more general (Royal Economics Society and International Association for Applied Econometrics conferences) and give scope for gaining wide academic feedback.

- The project’s research papers will be made available as Discussion Papers on the project’s website once they are complete. The work on the determinants of staff retention will be completed first, in accordance with the research agenda and scheduled plan.

In regards to access to journal articles - Green open access will be provided as the base case. Gold open access will be provided subject to the will of the funder to pay for the Gold open access charges. In any case, the publications pre-prints will always be freely available to the public. Gold open access is where an author publishes their article in an online open access journal. In contrast, green open access is where an author publishes their article in any journal and then self-archives a copy in a freely accessible institutional or specialist online archive known as a repository, or on a website.

Subject to any third party rights, the data and knowledge generated by the study will belong to the project partners (from UoS, University of Leeds and City University London), who will also:

- manage the data and knowledge produced;

- administer the access rights to the study and its results;

- together with The Health Foundation, arrange the possibility of making the publications of the study available as Open access. With respect to utilization rights, The Health Foundation has requested a license to use the work produced by the project partners for its public benefit purposes.

The Health Foundation is under an obligation to ensure that the outputs of the project are applied for the public good. Therefore, the funder has requested a license to use, for its public benefit purposes, the outputs generated by the Recipient under the Project. Subject to any third party rights, the project partners (i.e. the Principal Investigator and co-Investigators) have granted the funder a royalty-free, non-exclusive, world-wide license to use the outputs generated by the project partners under the project for its own charitable public benefit purposes. The funder, where reasonable, will discuss with the Principal Investigator prior to using the outputs for public benefit. All outputs shared with the Funder will be aggregated with small numbers suppressed.

Relevant Target Dates:

Submission to peer-review of at least one paper related the first research question by December 2021;

Submission to peer-review of at least one paper related to the second research question by June 2023;

Organization of launch event of the project by June 2023;

Publication of at least 60% of the outcomes of the project by December 2023. [Please notice that publications in Economics have a very long turnover and so they require several years to get peer-reviewed, revised and published. A paper publication in a 4 stars peer-reviewed journal in Economics can take also several years, due to previous rejections and the time for the peer-reviewers to provide comments].

EU funding is not applicable.

Yielded Benefits is not a requirement for new applications.