Nuffield RSET DSA Amendment 2025

The Nuffield Trust for Research and Policy Studies in Health Services (The Nuffield Trust) is an independent health research charity overseen by a board of Trustees including a number of senior NHS clinicians, managers and academics. The Nuffield Trust aims to improve the quality of health and in the UK by providing evidence-based research and policy analysis and informing and generating debate. It provides a trusted and respected voice at a time of unprecedented challenge to the NHS and social care system.

The Nuffield Trust has a number of projects in progress (covered under existing Data Sharing Agreements) to investigate the effectiveness and impact of various health service interventions, including ones focussing on Covid-19 and Covid-19 recovery. Those that were part of evaluation work for the NIHR-funded Rapid Service Evaluation Team (2018 to 2023) included:

• a youth violence intervention at a London hospital trust,

• analysis of the Care Quality Commission's special measures for quality regime,

• an analysis of outpatient attendances to identify variations in activity by trust and specialty,

• an evaluation of outpatient-based patient-initiated follow-up services, and

• analysis of a national rollout of home-based oximetry monitoring for Covid patients.

In everything the Nuffield Trust does, they strive to be:

• independent and free from vested interests;

• rigorous, robust and evidence-based in the work they undertake;

• relevant, supportive but also challenging when they need to be;

• open and engaging with all those they come into contact with;

an organisation that makes a difference to the quality of policy-making and practice in the UK.

Mindful of the quote by the UK Chief Medical Adviser, that "Research is of no use unless it gets to the people who need to use it", Nuffield Trust's strategy to maximise impact is as follows:

• Investigate relevant research questions that are important to patients, commissioners, providers and other stakeholders;

• Co-produce research with patients, the public, professionals and managers to address these questions using rigorous and appropriate research methods applied to the best available data;

• Identify the implications of this research for current and future service delivery for different stakeholders;

• Present the evidence in an appropriate format for the range of target audiences;

• Be guided by input from evidence users, including patients and the public, in all of the above.

Challenges to health and care systems such as increases in life expectancy and multi-morbidities, developments in treatments and technologies, and wider economic pressures have resulted in a drive at national and local levels to develop and accelerate service innovations to benefit patients and the public. Currently, innovations do not spread as fast nor have the same degree of impact as in other sectors, particularly in terms of rapidly shaping frontline service delivery. There is therefore a need for new approaches to evaluation that provide robust evidence to meet these needs in a timely fashion.

To address this need, the National Institute for Health Research (NIHR) Health Services and Delivery Research (HS&DR) programme have re-commissioned a Rapid Service Evaluation Team (NIHR RSET, 2023 to 2028).

This follows the success of the first NIHR RSET (2018 to 2023). Within NIHR RSET (2018 to 2023) ten projects were completed over the five years of funding:

1. Special Measures for Quality (SMQ)

2. Identification and prioritisation of research topics in adult social care and social work

3. Virtual Wards/remote home monitoring for COVID-19 patients (Phase 1)

4. Innovations in Outpatient Services (including evaluation of the implementation of patient-initiated follow-up (PIFU))

5. Youth violence intervention programme (Redthread) for vulnerable young people (11-24 years) attending Emergency Departments

6. Virtual Wards/remote home monitoring for COVID-19 patients "Covid Oximetry @ Home" (Phase 2)

7. Covid Oximetry @home Care home extension

8. Pre-hospital triage for stroke patients

9. Peer supported social care in prisons

10. Centralisation of specialist healthcare services: a mixed methods programme

HES and ECDS data were used under a previous iteration of this DSA (DARS-NIC-194629-S4F9X-v4) for projects numbered 1, 4, 5 and 6.

With respect to the outpatient innovations project (number 4) the Nuffield Trust have undertaken an analysis of shifts in outpatient attendances during Covid-19. The findings of the analysis are publicly available here https://www.medrxiv.org/content/10.1101/2021.04.28.21256176v1.

A major phase of this study used national monthly HES and ECDS data alongside data from the Provider Elective Recovery Outpatients Collection (made available to us by NHS England) to test whether there was evidence of an impact of PIFU services on the volume of follow-up outpatient attendances, and A&E visits. As of October 2023, a full study report has been submitted to NIHR, and will be published in due course. Some results from an early phase are available in a slide pack https://www.nuffieldtrust.org.uk/sites/default/files/2022-12/nhse-pifu-evaluation-phase-1-final-main-v2-73-read-only.pdf, and a final slide pack with full findings will be published in the coming months.

Monthly HES data received for the Covid oximetry @ home project (number 6) allowed the Nuffield Trust to keep NHS England (and NHSX and NHS Digital at the time) as up to date as possible on impacts of this high priority scheme. The Nuffield Trust's work may help NHS England to plan its approach to home monitoring of other conditions, in the post-Covid service.

With respect to the evaluation of the Redthread youth violence programme (number 5), the intervention itself (based in Emergency Departments) was interrupted by Covid, and the monthly data was valuable in helping the Nuffield Trust to study implications of the re-launched service rapidly.

The Nuffield Trust was part of this previous collaborative team funded by NIHR (2018 to 2023), and they are part of the new 5-year collaboration (2023 to 2028) NIHR have recently funded.

The Nuffield Trust's partners in the NIHR Rapid Service Evaluation Team (NIHR RSET) study are researchers from the Department of Applied Health Research (DAHR) at University College London (UCL) and the Department of Public Health and Primary Care at the University of Cambridge. Note however, that the data accessed through this agreement will be managed by the Nuffield Trust, and will not be shared with UCL or Cambridge colleagues, or with any other third-parties (unless aggregated in accordance with standard disclosure rules).

The NIHR RSET team have a remit to produce timely findings of national relevance and immediate use to decision makers.

The service innovations studied may be 'combinatorial' and encompass several aspects of innovation such as technological, financial and service model re-design. To be successfully introduced and adopted by the health system, they need to demonstrate added value, cost-effectiveness, benefit to patients over current practice, and feasibility. At the same time, innovations generally require flexible and varied financial structures, and support mechanisms, to ensure they are scaled up and spread beyond local settings, whilst new service delivery models may demand innovative governance and payment arrangements. For these reasons, a flexible approach to applying new evaluation methods is important, in order to accommodate changing innovation processes and pathways. Developing these methods is necessary to address important research questions within this programme but will also be beneficial for future research.

The Nuffield Trust's and its collaborators’ approach to evaluation will combine the questions, 'What works at what cost?' with 'How and why?' using a mix of innovative quantitative and qualitative methods. The team will enable provision of formative, as well as summative, feedback to people implementing innovations in health and care services. The data sets accessed through this agreement will be crucial to answering questions about the impact of these schemes on health and care. This will happen through high quality, timely research undertaken with a variety of methods.

Over the five-year period 2023 to 2028 the NIHR RSET team will evaluate approximately ten innovations. These will be identified by the funder NIHR. Some evaluation projects will aim to complete within 6 to 9 months. Others may have a slightly longer timeframe, but the period from inception to analysis will necessarily be short, to allow for timely formative feedback to the areas undertaking innovations.

All evaluations will conform to the following scope:

- The subject of the evaluation will be a health or care provider (or group of health and/or care providers) which has/have adopted a new way of working which differs to those that have been widely adopted by the health system - i.e. an 'innovation'. Health or care providers may be NHS Trusts, primary care providers, Local Authorities or, for example, groups of district nurses. The innovation may be use of new technology, or a financial and/or service model redesign.

- The HES and ECDS data will be used to calculate relevant metrics that those implementing the technology would be seeking to affect - for example - short term readmissions to hospital for the targeted groups in those areas. HES data would not necessarily be used in all of the evaluations. It would only be used where it would add value to the evaluation.

- Community Services Data (CSDS) has been requested to provide a more complete view of healthcare related activity. The Rapid Response team has a broad remit to analyse the impact of innovation on access to and use of services. Access to CSDS will allow the research team to investigate areas such as displacement of activity where trusts may be successfully reducing re-attendances by displacing activity into other services such as community care. Having access to CSDS will provide visibility of such activity.

- The evaluations will attempt to determine whether there were any detectable changes in the metrics over time that might be associated with the introduction of the innovation.

- This would be done using a variety of techniques such as comparing metrics for the health care provider(s) pre and post introduction of the innovation or comparing the metrics of the health care provider(s) which adopted the innovation with health care provider(s) where the innovation is not in use.

- Standard hospital activity costing methods would be used to calculate the impact in terms of costs or savings to commissioners and hospitals and to model the impact of the innovation if adopted in larger areas (e.g. nationally).

Alongside this quantitative analysis, local qualitative information-gathering and analysis would be carried out to understand how and why the innovations might have had their particular impact in those local health economies.

The NIHR RSET team aim to minimise any delays (such as the approval process for data access) as much as possible in order to maximise time spent on each evaluation, to maximise the efficient use of researcher time and to maximise cost effective use of the NIHR grant. Note that the funders NIHR explicitly set up and expanded this programme to promote much more rapid analysis of service innovations, that might more quickly benefit health care services nationally.

The Nuffield Trust will be responsible for ensuring that all uses of the data will be within the agreed scope above.

The NIHR RSET team will be overseen by a Stakeholder Advisory Board. This Board will draw together expert leaders from across relevant sectors, including NIHR, and will have an independent Chair. The SAB will perform two main functions: 1) programme oversight; and 2) ongoing advice and support for the programme.

• Oversight: the SAB will meet online twice a year to review programme and project progress and the programme's priority topics, proposing amendments if appropriate. Recommendations will be recorded and shared with NIHR.

• Ongoing, rapid advice: the SAB members will provide ad hoc advice reflecting their expertise,

including identification of and access to services, inequalities, public and patient involvement and engagement (PPIE), relevant datasets, and optimal settings for and approaches to dissemination of findings.

The approach to public and patient involvement and engagement (PPIE) has been strengthened significantly in the new NIHR RSET approach. PPIE will shape all aspects of the team’s work, including: strategic leadership, programme oversight, project delivery, and enabling impact. While rapid evaluations can limit the time available to develop ways of working with patient representatives, the study team are able to draw on a co-produced strategy from NIHR RSET (2018 to 2023) to provide a foundation for new relationships in NIHR RSET (2023 to 2028). The study team will have a programme-level PPIE panel, made up of 8 to 10 members in order to increase capacity and diversity (including young people, those new to PPIE, and groups who struggle to access services or experience health inequalities). The PPIE Panel will meet 2-4 times per year and feed into all aspects of ongoing and developing RSET2 projects. the study team will also have project-specific PPIE input, with PPIE partners attending project meetings to guide delivery of the project, including decision-making, interpretation of findings, and co-production of outputs. The NIHR RSET (2023 to 2028) management team itself also includes two PPIE co-leads, one an experienced patient representative.

Unlike NIHR RSET (2018 to 2023), in NIHR RSET (2023 to 2028) the funder NIHR will mainly identify evaluation topics. However, upon receiving a topic, the study team will work with PPIE and evidence users to conduct an initial 2-4 week feasibility assessment. This will assess whether a rapid evaluation is required to inform policy and practice, whether the topic is amenable to rapid evaluation, and likely contribution of rapid evaluation to delivery of the innovation.

If the topic is deemed to be appropriate and amenable for rapid evaluation, the study team will conduct a period of iterative scoping work and protocol development. This stage will explore what is known about the subject and to identify gaps, in order to guide evaluation design. Throughout, the study team will be guided by in depth collaboration with PPIE and evidence users, and a review of available evidence.

PPIE and evidence users will be involved in several interrelated processes. the study team will learn about the innovation, its context, and activity to date; explore what data would be available for analysis; and coproduce research questions.

In NIHR RSET (2018 to 2023) NIHR commissioned some projects including projects 1, 6 and 7 listed above.

When deciding whether to proceed with a rapid evaluation the Nuffield Trust and its partners will consider:

- Whether rapid evaluation is needed and justified (Are there relevant gaps in evidence? Are lessons required urgently (e.g. to inform policy or practice rapidly)? Do stakeholders/evidence-users/PPIE agree on the need for the evaluation and its focus, scope, and timescale?)

- Whether there is sufficient stakeholder engagement (Do national/local stakeholders confirm support for evaluation? e.g. do they agree to join study group/steering panel, or act as local contact?)

- Whether there are no methodological barriers preventing rigorous rapid evaluation (Are sufficient quantitative and/or qualitative data available on the innovation, or can such data be collected in the given timeframe? Can research questions be answered with sufficient methodological rigour within the required timeframe?)

Where the decision is made to proceed to evaluation, a detailed research protocol will be developed by the NIHR RSET team (the Nuffield Trust and its UCL and Cambridge partners). This protocol develops the evaluation research questions and outlines in detail the design and methods proposed to answer the research questions including the team's plans for data processing. The protocol will include references to the evaluation team's approach to PPIE, ethical issues, data and information management, risk management and dissemination of findings. The protocols will be reviewed by independent experts in healthcare organisation and management, and also by project stakeholders, NIHR and patient representatives. The aim of this stage of review will be to advise the NIHR RSET team of any shortcomings in the proposed work, or of additional research questions that NIHR RSET might find valuable to address.

In developing the evaluation protocols the Nuffield Trust lead will ensure that any proposed use of HES, ECDS and CSDS data conforms to the agreed scope above, with appropriate input from the Nuffield Trust's Data Protection Officer (DPO). The Nuffield Trust project lead will ensure that:

- Projects have clearly defined objectives and analysis plans;

- In each case, the use of the HES/ECDS/CSDS data is necessary and proportionate to the purpose of the project - this will include consideration of the necessity for use of each individual HES dataset; the number of years of data; the sizes of any cohorts or control cohorts derived from the data, and the inclusion and exclusion criteria (such as presence of specific diagnostic or procedure codes);

- Appropriate safeguards are in place to protect confidentiality; minimise risks of re-identification and use of excessive data beyond necessity

The individual or individuals within Nuffield Trust who will carry out the project will be bound by an analysis plan detailing what data is permitted for use in the project and how it shall be processed. The Nuffield Trust aim to start each project (or sub-project workstream, as appropriate) with an assessment of the data necessary for analysis, and the anticipated analytical steps.

The Nuffield Trust, UCL and the University of Cambridge are joint data controllers for the data under this Data Sharing Agreement. As described above, the funder NIHR selects evaluation projects, and the Stakeholder Advisory Board and the PPIE panel offer advice on how the study team might evaluate and disseminate results.

However, decisions in respect of use of HES/ECDS/CSDS data under this Agreement are taken jointly by the Nuffield Trust, UCL and University of Cambridge without obligation to the funder or any other body. The Nuffield Trust, UCL and University of Cambridge will have joint responsibility for determining the purposes for processing the data under this Agreement to support the delivery of the projects and for determining the manner in which the data will be processed.

With reference to the Information Commissioner's published guidance on data controllership, the Nuffield Trust, UCL and University of Cambridge can confirm that these three organisations are jointly responsible for directing the research project. Specifically they decide:

- to collect the data under this Agreement and the legal basis for doing so;

- which items of data to collect, i.e. the content of the data

- the purposes the data are to be used for;

- which individuals to collect data about;

- whether to disclose the data - the data will not be disclosed to any third party;

- whether subject access and other individuals’ rights apply; and

- how long to retain the data

The nature of this NIHR RSET programme (2023 to 2028) is that its projects are not currently specified in this Agreement but will be identified at relatively short notice until the end of the NIHR RSET funding (end September 2028). This Agreement permits the Nuffield Trust to use subsets of the data for the purposes of the projects specified above and for the purposes of the projects to be identified for the purpose of this NIHR programme which must be compliant with the description of such projects given above.

The Nuffield Trust will send NHS England details of each NIHR RSET project's progress as part of an annual update.

Some of the innovations the Nuffield Trust will look at will inevitably need to be analysed with reference to long term historic data and so the Nuffield Trust will require Community Services Data Set, HES Accident and Emergency, HES Admitted Patient Care, HES Outpatients and ECDS data for all of England from 2013/14 to 2028/29.

The lawful basis for the Nuffield Trust to process personal data under this Agreement is under General Data Protection Regulation Article 6 (1) (f) and Article 9 (2) (j). The lawful basis for University College London to process personal data under this Agreement is under General Data Protection Regulation Article 6 (1) (e) and Article 9 (2) (j). The lawful basis for University of Cambridge to process personal data under this Agreement is under General Data Protection Regulation Article 6 (1) (e) and Article 9 (2) (j). The reasons described above are the Nuffield Trust's legitimate interests which provide a lawful basis for processing personal data under the General Data Protection Regulation (GDPR). Specifically, that the data is used to track changes in health care activity over time - this enables analysis of the impact of changes in health services; conduct detailed analysis of particular health events used to identify particular issues with quality of care; and study variation between hospitals or areas in use of services which can identify populations where there are gaps in care, and also areas delivering high quality care from which the NHS can learn more widely.

The funding for the RSET project comes to an end on 30 September 2028. The Agreement expiry date of 1/4/2027 has been chosen as the maximum period available.

The Nuffield Trust and the wider NIHR RSET team are highly aware of the need to balance rapid formative feedback (to areas implementing innovations) with the need to produce summative findings (shared locally and nationally, including in academic publications).

The NIHR RSET team will use innovative approaches to sharing findings, including formative feedback locally during scoping and the full evaluation, and active dissemination of findings more widely so that they can be taken up quickly in other parts of the health and care system. This will be supported by advice from the team’s Stakeholder Advisory Board (meeting twice yearly) and their PPIE Panel (meeting 2-4 times per year).

The Nuffield Trust will use their extensive communications facilities & networks for dissemination (including professionals in the fields of media relations, public affairs, digital communications and event management), working with the UCL and Cambridge communications teams, NIHR Dissemination Centre, and NIHR Health Services and Delivery Research (HS&DR) programme to maximise the impact of evaluation findings.

The NIHR RSET team's communications/dissemination strategy will:

• Identify the audience: who needs to know?

• Set a timetable for dissemination outputs: weeks, not months

• Decide on appropriate media and form of communications

• Ensure written outputs are as simple as possible

• Provide context and other evidence relevant to each evaluation.

The NIHR RSET team will develop a dissemination strategy for the overall programme and each project in collaboration with NIHR, PPIE & end users, including stakeholder dissemination list; social media strategy; bespoke events; conference presentations; open-access journal articles; sharing findings with trade press, such as the Health Service Journal and National Health Executive.

All NIHR RSET projects will have PPIE input. Patients and the public will be involved in the programme PPIE panel and in project-specific panels. An experienced patient representative is on the NIHR RSET management team, and was instrumental in developing some of the new team’s processes.

NIHR RSET's PPIE strategy was revised in early 2021, in collaboration with PPIE representatives, and new PPI members were subsequently recruited. Previous NIHR RSET projects involved patients and the public in the following ways:

• PPIE representatives were involved in designing the COVID Oximetry @home questionnaires, as well as providing feedback on the early findings from the workforce and patient surveys.

• A new PPIE representative was recruited for the Innovations in Outpatient Services project to provide perspectives on their experiences of care and different outpatients services, and two PPIE representatives commented on the project protocol.

The NIHR RSET team will plan outputs and events to stimulate formative learning over the course of evaluations. The NIHR RSET team will reach target audiences through a variety of tailored methods, from graphics and posts on social media, to blogs and Q&As, accessible summaries, reviews, webinars, short films and visualisations to traditional media coverage and targeted two-way conversation events through, for example, Nuffield Trust's popular annual rapid evaluation conference.

The NIHR RSET team have set up a micro site on the Nuffield Trust website (https://www.nuffieldtrust.org.uk/rset-rapid-evaluations-of-new-ways-of-providing-care) devoted to the whole NIHR RSET programme. This serves as a focus for reaching out to the NHS and other organisations to advertise the programme and its findings. A full, and growing, list of NIHR REST outputs can be found at https://www.nuffieldtrust.org.uk/rset-rapid-evaluations-of-new-ways-of-providing-care/publications-and-outputs-0-0 .

The Nuffield Trust will review and evaluate the impact of their communications activity and refine their approach for each new output.

Examples of completed projects under NIHR RSET (2018 to 2023), and their outputs:

I. Special Measures for Quality (SMQ)

The team carried out a considerable amount of dissemination and engagement with key stakeholders (Department of Health and Social Care, Care Quality Commission, NHS Improvement/England). One recent output was a rapid literature review published in the International Journal of Health Policy and Management (IJHPM) please see https://www.ijhpm.com/article_3943.html), and a Q&A blogpost on the impact of improvement interventions in NHS trusts (please see - https://www.nuffieldtrust.org.uk/news-item/special-measures-for-quality-a-q-a-on-the-impact-of-improvement-interventions-in-nhs-trusts). A final report to the funder has been submitted and will be published in the coming months.

II. Identification and prioritisation of research topics in adult social care and social work

The paper outlining adaptation of the James Lind Alliance (JLA) method for rapid identification was published in the Health Research Policy and Systems (HARPS) in March 2021 (please see - https://health-policy- systems.biomedcentral.com/articles/10.1186/s12961-021-00693-2).

III. Virtual Wards/remote home monitoring for COVID-19 patients

The team presented findings from both phases of the project to national leaders including Public Health England, NHS England, NHS Digital, and National Institute for Health Research and in webinars in all English regions. Further, the team presented the findings to over 400 people around the country planning and developing virtual wards, including the NHS@Home Pulse Oximetry Learning Network, COVID-19 Virtual Wards Community of Practice, Liverpool City Region/Northwest Coast, and London COVID Care at Home (Virtual Ward) Webinar. In addition to these published presentations, and planned publicstions, up to ten peer reviewed papers from both phases of the study. During 2021 a Community of Practice consisting of clinicians and other healthcare professionals from within the NHS met to discuss and share plans for the home monitoring of other conditions, building on their experiences during the Covid 19 pandemic. Outputs from the RSET project on remote monitoring of COVID-19 patients were disseminated to this group to provide learning about how they might best benefit patients.

A report has also been published for the Youth violence intervention programme (Redthread), and reports for Investigating innovations in outpatient services (with a special focus on Patient-Initiated Follow-up) and Centralisation of specialist services have been submitted to the funder, and will be published shortly.

The Outpatients work team have worked closely with NHSE’s Outpatient Transformation team, and have also fed in to the newly developing RCP and NHSE Outpatient Strategy.

All Nuffield Trust outputs for NIHR RSET projects will be aggregated with small numbers suppressed in line with the HES Analysis Guide.

All NIHR RSET data work is kept within specific, named folders on the secure server, distinct from other project folders for projects that fall under other Data Sharing Agreements. Despite the joint controllership with University College London and University of Cambridge for this Agreement, no data are shared, unless aggregated and suitably disclosure controlled.

An external review of NIHR RSET (2018 to 2023) interviewed 18 significant stakeholders at the mid-way point of its funding, and summarised the team’s impacts as follows:

“RSET is having impact. The findings from completed evaluations have influenced national policy and informed service delivery. After working with RSET, evidence users have developed and are continuing to apply their knowledge of evaluation methods. Members of the team see potential for RSET to help improve the healthcare sector’s evaluation capacity, particularly for service innovations with evolving data collection systems. RSET is helping NIHR to respond more quickly to the needs of evidence users, and demand for this service is increasing as RSET’s reputation for the quick deployment of a skilled and credible team grows.”

The impact of NIHR RSET’s findings (“heightened by the team’s reputation for delivering robust and credible work, succinct presentation of findings and relationship-building and networking skills”, according the review) were particularly noted in cases where the intervention was of national interest and importance.

The analysis of the Care Quality Commission's special measures for quality regime led to the project team presenting findings to several groups including senior staff at the Department for Health and Social care and NHSEI's Joint strategic oversight group. This work has influenced the development of the new Intensive Support regime. The benefits are a more effective regime for supporting trusts that are underperforming with respect to quality of care to enable them to provide better care for patients.

With respect to the Covid oximetry evaluation, feedback from NHS England has recognised the contribution the evaluation team made to establishing the Covid oximetry at home programme, and the programme had a direct impact on improving treatment options available to Covid patients. Building on this programme, NHS England have outlined a 'national ambition' to have in place 40-50 virtual wards beds per 100,000 population for non-Covid conditions, by the end of 2023. The work ensures that systems are able to understand the current evidence basis on the implementation of, and impact of such schemes, as they invest in setting these up. The national and international relevance of this work is revealed by over 100 citations of the evaluation team's first two research articles, with several other publications having also been published. It is expected that the work is, and will be, influencing not just national, but international implementations of home monitoring schemes, and evaluations of such schemes.

Another NIHR RSET evaluation which has made use of HES data - the Outpatients innovation evaluation - has led to a relationship with NHS England's Outpatient transformation programme to discuss potential rapid evaluation support around the national ambition to move a proportion of outpatient attendances to 'Patient Initiated Follow Up' pathways.

The Nuffield Trust for Research and Policy Studies in Health Services (The Nuffield Trust) is an independent health research charity overseen by a board of Trustees including a number of senior NHS clinicians, managers and academics. The Nuffield Trust aims to improve the quality of health and in the UK by providing evidence-based research and policy analysis and informing and generating debate. It provides a trusted and respected voice at a time of unprecedented challenge to the NHS and social care system.

The Nuffield Trust have a number of projects in progress (covered under existing Data Sharing Agreements) to investigate the impact of Covid-19 on the use of health services, and strategies for post Covid-19 recovery.

Those that are part of the evaluation work for the NIHR- funded Rapid Service Evaluation Team include:

• an analysis of outpatient attendances to identify variations in activity by trust and specialty,

• a potential evaluation of outpatient-based patient initiated follow up services, and

• analysis of a national rollout of home-based oximetry monitoring for Covid patients.

In everything the Nuffield Trust does, they strive to be:

• independent and free from vested interests;

• rigorous, robust and evidence-based in the work they undertake;

• relevant, supportive but also challenging when they need to be;

• open and engaging with all those they come into contact with;

• an organisation that makes a difference to the quality of policy-making and practice in the UK.

Mindful of the quote by the Chief Scientific Adviser for the Department of Health and Social Care that "Research is of no use unless it gets to the people who need to use it", Nuffield Trust's strategy to maximise impact is as follows:

• Investigate relevant research questions that are important to patients, commissioners, providers and other stakeholders;

• Co-produce research with patients, the public, professionals and managers to address these questions using rigorous and appropriate research methods applied to the best available data;

• Identify the implications of this research for current and future service delivery for different stakeholders;

• Present the evidence in an appropriate format for the range of target audiences;

• Be guided by input from evidence users, including patients and the public, in all of the above.

Challenges to health and care systems such as increases in life expectancy and multi-morbidities, developments in treatments and technologies, and wider economic pressures have resulted in a drive at national and local levels to develop and accelerate service innovations to benefit patients and the public. Currently, innovations do not spread as fast nor have the same degree of impact as in other sectors, particularly in terms of rapidly shaping frontline service delivery. There is therefore a need for new approaches to evaluation that provide robust evidence to meet these needs in a timely fashion. To address this need, the National Institute for Health Research (NIHR) Health Services and Delivery Research (HS&DR) programme have commissioned a Rapid Service Evaluation Team (RSET).

Within NIHR RSET three projects have been completed and seven are ongoing.

Completed Projects:

1. Special Measures for Quality (SMQ)

2. Identification and prioritisation of research topics in adult social care and social work

3. Virtual Wards/remote home monitoring for COVID-19 patients (Phase 1)

Ongoing Projects:

4. Innovations in Outpatient Services

5. Youth violence intervention programme (Redthread) for vulnerable young people (11-24 years) attending Emergency Departments

6. Virtual Wards/remote home monitoring for COVID-19 patients ""Covid Oximetry @ Home"" (Phase 2)

7. COVID Oximetry @home Care home extension

8. Pre-hospital triage for stroke patients

9. Peer supported social care in prisons

10. Centralisation of specialist healthcare services: a mixed methods programme

HES data have been, or are being used for projects numbered 1, 4, 5, 6, and may be used for a latter stage of 10.

With respect to the outpatient project (see 4 in list above) the Nuffield Trust have undertaken an analysis of shifts in outpatient attendances during Covid-19. The findings of the analysis are available in the preprint and the preprint is publicly available here https://www.medrxiv.org/content/10.1101/2021.04.28.21256176v1. The Nuffield Trust are currently updating this with the latest data, and will submit for publication soon.

The monthly data for the Covid oximetry @ home project (6; details above) has allowed the Nuffield Trust to keep NHS England (and NHSX and NHS Digital) as up to date as possible on possible impacts of this high priority scheme. The Nuffield Trust's work may help NHS England to plan its approach to home monitoring of other conditions, post-Covid.

With respect to the evaluation of the Redthread youth violence programme (5; details above), this is one where the intervention itself (based in Emergency Departments) was interrupted by Covid, and the monthly data has been valuable in helping the Nuffield Trust to study possible implications of the re-launched service rapidly.

The Nuffield Trust is part of this collaborative team funded by NIHR over 5 years (2018/9 to 2022/23) to conduct rapid evaluations of health and care service innovations, in close partnership with those who deliver, manage and use these services.

The service innovations may be 'combinatorial' and encompass several aspects of innovation such as technological, financial and service model re-design. To be successfully introduced and adopted by the health system, they need to demonstrate added value, cost-effectiveness, benefit to patients over current practice, and feasibility. At the same time, innovations generally require flexible and varied financial structures, and support mechanisms, to ensure they are scaled up and spread beyond local settings, whilst new service delivery models may demand innovative governance and payment arrangements. For these reasons, a flexible approach to applying new evaluation methods is important, in order to accommodate changing innovation processes and pathways. Developing these methods is necessary to address important research questions within this programme but will also be beneficial for future research.

The Nuffield Trust's partners in the Rapid Service Evaluation Team (RSET) study are researchers from the Department of Applied Health Research (DAHR) at University College London (UCL) - although the data accessed through this agreement will be managed by the Nuffield Trust, and will not be shared with UCL colleagues or any other third-parties (unless aggregated in accordance with standard disclosure rules). The RSET team have a remit to produce timely findings of national relevance and immediate use to decision makers.

The Nuffield Trust's and UCL's approach to evaluation will combine the questions, 'What works at what cost?' with 'How and why?' using a mix of innovative quantitative and qualitative methods. The team will enable provision of formative, as well as summative, feedback to people implementing innovations in health and care services. The data sets accessed through this agreement will be crucial to answering questions about the impact of these schemes on health and care. This will happen through high quality, timely research undertaken with a variety of methods. Over the five-year period the RSET team will identify approximately ten innovations to evaluate. NIHR, NHS England, DHSC and the RSET team's stakeholder advisory board (which includes clinicians, managers, and patient and public representatives) will have advisory input into identifying potential innovations. There will be a focus on rapid evaluation of these schemes with some projects aiming to complete evaluation within 6 to 9 months. Others may have a slightly longer timeframe, but the period from inception to analysis will necessarily be short, to allow for timely formative feedback to the areas undertaking innovations.

All evaluations will conform to the following scope:

• The subject of the evaluation will be a health or care provider (or group of health and/or care providers) which has/have adopted a new way of working which differs to those that have been widely adopted by the health system - i.e. an 'innovation'. Health or core providers may be NHS Trusts, Local Authorities or, for example, a group of district nurses. The innovation may be use of new technology or a financial and/or service model redesign.

• The HES data would potentially be used to calculate relevant metrics that those implementing the technology would be seeking to affect - for example - short term readmissions to hospital for the targeted groups in those areas. HES data would not necessarily be used in all of the evaluations. It would only be used where it would add value to the evaluation.

• Community Services Data (CSDS) has been requested to provide a more complete view of healthcare related activity. The Rapid Response team has a broad remit to analyse the impact of innovation on access to and use of services. Access to CSDS will allow the research team to investigate areas such as displacement of activity where trusts may be successfully reducing re-attendances by displacing activity into other services such as community care. Having access to CSDS will provide visibility of such activity.

• The evaluations will attempt to determine whether there were any detectable changes in the metrics over time that might be associated with the introduction of the innovation.

• This would be done using a variety of techniques such as comparing metrics for the health care provider(s) pre and post introduction of the innovation or comparing the metrics of the health care provider(s) which adopted the innovation with health care provider(s) where the innovation is not in use.

• Standard hospital activity costing methods would be used to calculate the impact in terms of costs or savings to commissioners and hospitals and to model the impact of the innovation if adopted in larger areas (e.g. nationally).

Alongside this quantitative analysis, local qualitative information gathering and analysis would be carried out to understand how and why the innovations might have had their particular impact in those local health economies.

The RSET team aim to minimise any delays (such as the approval process for data access) as much as possible in order to maximise time spent on each evaluation, to maximise the efficient use of researcher time and to maximise cost effective use of the NIHR grant. Note that the funders NIHR explicitly set up this programme to promote much more rapid analysis of service innovations, that might more quickly benefit health care services nationally.

The Nuffield Trust will be responsible for ensuring that all uses of the data will be within the agreed scope above.

The selection of research projects to be undertaken by the RSET team will be guided using a partnership approach, involving close working with the funder (NIHR), but also with other stakeholders via the RSET team's Advisory Board, Patient and Public Involvement (PPI) input and other networks.

The Stakeholder Advisory Board will draw together expert leaders from across relevant sectors, including NIHR, and will have an independent Chair. The SAB will perform two main functions: 1) programme oversight; and 2) ongoing advice and support for the programme.

• Oversight: the SAB will meet face-to-face annually to review programme and project progress and the programme’s priority topics, proposing amendments if appropriate. Recommendations will be recorded and shared with NIHR.

• Ongoing, rapid advice: the SAB members will advise on: horizon scanning; current innovations and who to contact in relation to these; additional sites that are implementing innovations that we are evaluating; and on effective dissemination of findings.

The PPI is an effective and meaningful involvement of service-users, carers, patients and members of the public who make for a stronger rapid evaluation team, more focused research and more accessible and user-friendly outputs. The Nuffield Trust worked with patient and public partners on their bid, and plan to involve people throughout, to ensure their work benefits from:

• Experience of health/social care services as service user, patient, carer, or relative (including parents)

• Knowledge and understanding of patient, public and service user perspectives

• A unique voice to evaluations of innovation and service development – in particular when evaluating involvement and engagement activity related to innovation

• Knowledge and understating of successful involvement/engagement

The topics will be selected by initially producing a long list of potential research projects based on recommendations from the funder, the Advisory Board, and the PPI panel. This will be supplemented with a range of activities by the RSET team to identify topics of current interest that it would be timely to investigate. These activities will include desk-based research of published outputs, analysis of social media, interaction with stakeholders at dissemination events, contacting networks, and by people alerting the RSET team to innovations through their interactive website.

A shortlist of potential research projects will be produced using pre-determined criteria to be agreed with the funder, the Advisory Board, and PPI panel. These criteria are likely to include considerations of timeliness, strategic importance, and feasibility. Topics identified by the funder will be prioritised.

NIHR have commissioned some projects including projects 1, 6 and 7 listed above. A significant scoping phase is undertaken, and all projects require NIHR approval before progressing.

The criteria are:

- Strategic importance (Is the innovation within scope? How large an impact could the innovation have? Are other evaluations being conducted? Or have they been completed already? Would the results of the evaluation be of national relevance? If it is a local or regional evaluation, is there be enough potential for others to learn from it? Would the evidence be of immediate use? Would the evidence have long-lasting usefulness?)

- Feasibility (Is it possible to evaluate the innovation within a reasonable timeframe? Could a 'useful' evaluation be completed within the scale of RSET? Is (enough) data available? Does data of sufficient granularity exist? Covering the right areas? Over the right period? Is it of sufficient quality? If data does not exist, can it be collected? Is the innovation ready to be evaluated? Would the Nuffield Trust be evaluating a stable innovation, or is the intervention in continuous change? Would there be sufficient support from participating services for the evaluation? Would there be sufficient support from the funder and patient and public involvement group for the evaluation? How would the evaluation fit within the RSET portfolio of evaluations? Are the Nuffield Trust covering a sufficient variety of service innovations? How would appropriate team resources be allocated? Is there the right capacity to do the evaluation right now?)

- Other criteria (Are there any ethical issues to consider with the innovation? Would there be any ethical issues to consider when conducting an evaluation of this innovation? Who stands to benefit from the innovation? What would the equity health impact be? Does the evaluation afford the team the opportunity to push the methodological boundaries of health service research? Is there a potential for the innovation to generate any other positive or negative externalities that the Nuffield Trust should consider?).

Once a project has been identified and reviewed by the funder (to advise the RSET team as to whether the project is within the scope of the research grant), the project will enter a scoping phase. A 'topic specification form' is filled in at this point. This includes a description of the motivation for undertaking the research, the research questions, the proposed work and the timescales. This is reviewed by the Nuffield Trust alongside its RSET partners and the funder, and a decision is taken by the RSET team as to whether the project should proceed to a full evaluation.

Where the decision is made to proceed to evaluation, a detailed research protocol is developed by the RSET team (the Nuffield Trust and its UCL partners). This protocol develops on the evaluation research questions and outlines in detail the design and methods proposed to answer the research questions including the team's plans for data processing. The protocol will include references to the evaluation team's approach to PPI, ethical issues, data and information management, risk management and dissemination of findings. The protocols will be reviewed by independent experts in healthcare organisation and management, and also by project stakeholders, NIHR and patient representatives. The aim of this stage of review will be to advise the RSET team of any shortcomings in the proposed work, or of additional research questions that the RSET might find valuable to address.

In developing the evaluation protocols the Nuffield Trust lead will ensure that any proposed use of HES data conforms to the agreed scope above, with appropriate input from the Nuffield Trust's Data Protection Officer (DPO). The Nuffield Trust project lead will ensure that:

• Projects have clearly defined objectives and analysis plans;

• In each case, the use of the HES data is necessary and proportionate to the purpose of the project - this will include consideration of the necessity for use of each individual HES dataset; the number of years of data; the sizes of any cohorts or control cohorts derived from the data, and the inclusion and exclusion criteria (such as presence of specific diagnostic or procedure codes);

• Appropriate safeguards are in place to protect confidentiality; minimise risks of re-identification and use of excessive data beyond necessity

The individual or individuals within Nuffield Trust who will carry out the project will be bound by an analysis plan detailing what data is permitted for use in the project and how it shall be processed. The Nuffield Trust aim to start each project (or sub-project workstream, as appropriate) with an assessment of the data necessary for analysis, and the anticipated analytical steps.

The Nuffield Trust and UCL are joint data controllers for the data under this Data Sharing Agreement. As described above, the funder, the Advisory Board and the PPI panel offer advice on what service evaluation projects might be undertaken. Additionally, the funder may have a role in determining that a given project is not within scope of the research grant.

However, decisions in respect of use of the data under this Agreement are taken jointly by the Nuffield Trust and UCL without obligation to the funder or any other body. The Nuffield Trust and UCL will have joint responsibility for determining the purposes for processing the data under this Agreement to support the delivery of the projects and for determining the manner in which the data will be processed.

With reference to the Information Commissioner's published guidance on data controllership, the Nuffield Trust and UCL can confirm that these two organisations are jointly responsible for directing the research project. Specifically they decide:

• to collect the data under this Agreement and the legal basis for doing so;

• which items of data to collect, i.e. the content of the data

• the purposes the data are to be used for;

• which individuals to collect data about;

• whether to disclose the data - the data will not be disclosed to any third party;

• whether subject access and other individuals' rights apply; and

• how long to retain the data

Specific projects completed, or currently in progress, that make use of HES have been listed above.

The nature of this NIHR programme is that further projects are not currently specified in this Agreement but will be identified at relatively short notice until the end of the NIHR RSET funding (end March 2023). This Agreement permits the Nuffield Trust to use subsets of the data for the purposes of the projects specified above and for the purposes of the projects to be identified for the purpose of this NIHR programme which must be compliant with the description of such projects given above.

The Nuffield Trust will send NHS Digital details of each RSET project's progress as part of an annual update.

Some of the innovations the Nuffield Trust will look at will inevitably need to be analysed with reference to long term historic data and so the Nuffield Trust will require Community Services Data Set, HES Accident and Emergency, HES Admitted Patient Care, HES Outpatients and ECDS data for all of England from 2008/09 to 2022/23.

The lawful basis for the Nuffield Trust to process personal data under this Agreement is under General Data Protection Regulation Article 6 (1) (f) and Article 9 (2) (j). The lawful basis for University College London to process personal data under this Agreement is under General Data Protection Regulation Article 6 (1) (e) and Article 9 (2) (j). The reasons described above are the Nuffield Trust's legitimate interests which provide a lawful basis for processing personal data under the General Data Protection Regulation (GDPR). Specifically, that the data is used to track changes in health care activity over time enables analysis of the impact of changes in health services; conduct detailed analysis of particular health events used to identify particular issues with quality of care; and study variation between hospitals or areas in use of services which can identify populations where there are gaps in care, and also areas delivering high quality care from which the NHS can learn more widely.

The funding for the RSET project comes to an end on 31/3/23. Access is however required to year-end data to provide the opportunity for the research team to re-work their analysis with the year-end data in submitting to journals. The Agreement expiry date of 30/6/2025 takes into account a 3 month period to submit outputs to journals together with a 2 year retention period to respond to questions and challenge.

The Nuffield Trust and the wider RSET team are highly aware of the need to balance rapid formative feedback (to areas conducting innovations) with the need to produce summative findings (shared locally and nationally, including academic publications).

The RSET team will use innovative approaches to sharing findings, including formative feedback locally during scoping and the full evaluation, and active dissemination of findings more widely so that they can be taken up quickly in other parts of the health and care system.

The Trust will use their extensive communications facilities & networks for dissemination (including professionals in the fields of media relations, public affairs, digital communications and event management), working with the UCL communications team, National Institute for Health Research (NIHR) Dissemination Centre, and Health Services and Delivery Research (HS&DR) programme to maximise the impact of evaluation findings.

The RSET team's communications/dissemination strategy will:

• Identify the audience: who needs to know?

• Set a timetable for dissemination outputs: weeks, not months

• Decide on appropriate media and form of communications

• Ensure written outputs are as simple as possible

• Provide context and other evidence relevant to each evaluation.

The RSET team will develop a communication strategy for the overall programme and each project in collaboration with NIHR, PPI & end users, including stakeholder dissemination list; social media strategy; bespoke events; conference presentations; open-access journal articles; sharing findings with trade press, such as the Health Service Journal and National Health Executive.

GP = General practitioners; STP = Sustainability and transformation partnerships; CEOs = Chief Executives.

All RSET projects have PPI input. Patients and the public have been involved in the following ways:

• PPI representatives were involved in designing the COVID Oximetry @home questionnaires, as well as providing feedback on the early findings from the workforce and patient surveys.

• A new PPI representative was recruited for the Innovations in Outpatient Services project to provide perspectives on their experiences of care and different outpatients services, and two PPI representatives commented on the project protocol. RSET's PPI strategy has been revised, in collaboration with PPI representatives. The revised strategy was distributed in January 2021. Since then, three new PPI members have been recruited.

The RSET team will plan outputs and events to stimulate formative learning over the course of evaluations. The RSET team will reach target audiences through a variety of tailored methods, from graphics and tweetchats on social media, to blogs, vlogs and blogshots, opinion articles, short films and visualisations to traditional media coverage and targeted two-way conversation events through, for example, Nuffield Trust's extensive learning sets (involving GPs, STP leads, acute trust CEOs etc).

The RSET team have also set up a micro site on the Nuffield Trust website (https://www.nuffieldtrust.org.uk/project/rset- the-rapid-service-evaluation-team) devoted to the whole RSET programme. This will serve as a focus for reaching out to

the NHS and other organisations to advertise the RSET programme and its findings.

The Trust will review and evaluate the impact of their communications activity and refine their approach for each new output.

Completed Projects:

I. Special Measures for Quality (SMQ)

The team carried out a considerable amount of dissemination and engagement with key stakeholders (Department of Health and Social Care, Care Quality Commission, NHS Improvement/England). The latest output (since July 2020) are a rapid literature review published in the International Journal of Health Policy and Management (IJHPM) please see https://www.ijhpm.com/article_3943.html), and a Q&A blogpost on the impact of improvement interventions in NHS trusts (please see - https://www.nuffieldtrust.org.uk/news-item/special-measures-for-quality-a-q-a-on-the-impact-of- improvement-interventions-in-nhs-trusts). The team are currently working on further papers for submission in the next few months and continuing with dissemination remotely to relevant stakeholders.

II. Identification and prioritisation of research topics in adult social care and social work

The paper outlining adaptation of the James Lind Alliance (JLA) method for rapid identification was published in the Health Research Policy and Systems (HARPS) in March 2021 (please see - https://health-policy- systems.biomedcentral.com/articles/10.1186/s12961-021-00693-2).

III. Virtual Wards/remote home monitoring for COVID-19 patients (Phase 1)

The team presented findings from phase I to national leaders including Public Health England, NHS England, NHS Digital, and National Institute for Health Research and in webinars in all English regions. Further, the team presented the findings to over 400 people around the country planning and developing virtual wards, including the NHS@Home Pulse Oximetry Learning Network, COVID-19 Virtual Wards Community of Practice, Liverpool City Region/Northwest Coast, and London COVID Care at Home (Virtual Ward) Webinar.

All outputs will be aggregated with small numbers suppressed in line with the HES Analysis Guide.

Reports are currently in development (including in preprint form) for projects: Virtual Wards/remote home monitoring for COVID-19 patients (Phase 2), Innovations in outpatient services, and Youth violence intervention programme (Redthread).

During 2021 a Community of Practice consisting of clinicians and other healthcare professionals from within the NHS met to discuss and share plans for the home monitoring of other conditions, building on their experiences during the Covid 19 pandemic. Outputs from the RSET project on remote monitoring of COVID-19 patients were disseminated to this group to provide learning about how they might best benefit patients.

All RSET data work is kept within specific, named folders on the secure server, distinct from other project folders for projects that fall under other Data Sharing Agreements. Despite the joint controllership with University college London for this Agreement, no data are shared, unless aggregated and suitably disclosure controlled.

The analysis of the Care Quality Commission's special measures for quality regime carried out by the RSET team has led led to the project team presenting findings to several groups including senior staff at the Department for Health and Social care and NHSEI's Joint strategic oversight group. This work has influenced the development of the new Intensive Support regime. The benefits are a more effective regime for supporting trusts that are underperforming with respect to quality of care to enable them to provide better care for patients.

With respect to the Covid oximetry evaluation, feedback from NHS England has recognised the contribution the evaluation team made to establishing the Covid oximetry at home programme, and the programme had a direct impact on improving treatment options available to Covid patients. Building on this programme, NHS England have outlined a ‘national ambition’ to have in place 40-50 virtual wards beds per 100,000 population for non-Covid conditions, by the end of 2023. The work ensures that systems are able to understand the current evidence basis on the implementation of, and impact of such schemes, as they invest in setting these up. The national and international relevance of this work is revealed by 50+ citations of the evaluation team’s first two research articles, with the bulk of publications (up to 8 more papers) still to come. It is expected that the work is, and will be, influencing not just national, but international implementations of home monitoring schemes, and evaluations of such schemes.

Two further RSET evaluations have made use of HES data (Outpatients innovations evaluation, and Redthread Youth Violence Intervention at UCLH), but have yet to reach the stage of findings being published. However, one of these – the Outpatients innovation evaluation – has led to a relationship with NHS England’s Outpatient transformation programme to discuss potential rapid evaluation support around the national ambition to move a proportion of outpatient attendances to ‘Patient Initiated Follow Up’ pathways.

June 2021 - Monthly data dissemination are requested to gain access to more timely data (especially related to COVID-19 research) from 1/7/21 to 30/6/22.

Justification:

Since the Nuffield Trust received monthly data at the end of 2020, the Trust have used this for Quality Watch annual data visualisation on the impact of the rapid growth in remote care delivery during Covid-19. This considered the long-term impact on quality of care, and what issues need to be evaluated and monitored in the longer term. As well as releasing the data story and blog, the Nuffield Trust also presented on this issue to the Rewired digital health conference. Having monthly data enabled updates to the analysis for the conference.

The Nuffield Trust have also used the data in the chart of the week series (please see here - https://www.nuffieldtrust.org.uk/spotlight/chart-of-the-week), for example for tracking length of stay of covid-19 patients during the pandemic. Further charts of the week are planned using monthly data. The next one to use HES data is likely to be examining the impact on emergency admissions for respiratory conditions during the pandemic, and what the lessons are for the longer term. The monthly data has also enabled the Nuffield Trust to track cancelled operations for Quality Watch, data which contributes to the analysis of how the NHS is managing during the pandemic, and what the impacts are on health care for non-Covid conditions.

The Nuffield Trust have also the data for scoping potential analysis on “lost” planned activity because of Covi19, and what the implications are for recovery. These outputs fall within the Quality and equity programme, and are also relevant to Politics, legislation, and governance work, where the Nuffield Trust consider issues which are a priority for politicians and government.

The need for monthly data also supports RSET projects led by the Nuffield Trust. Please see here - https://www.nuffieldtrust.org.uk/rset-the-rapid-service-evaluation-team.

The Nuffield Trust have a number of projects planned (covered under existing data sharing agreements) to investigate the impact of Covid-19 on the use of health services which are dependent on or would benefit from access to more timely (i.e. monthly rather than quarterly) HES data. These include, for example, an analysis of outpatient attendances with a view to identify variations in activity by trust and specialty over the early pandemic period as part of evaluation work for the NIHR-funded Rapid Service Evaluation Team. The Trust are also starting a project to understand the profile of patients discharged before and during the pandemic, and their subsequent use of services in the community. As part of the Trusts Quality Watch programme the Trust are also planning more wide-ranging analysis of hospital activity and performance over the pandemic period and especially in the run up to, and over, the winter period for which again, more timely HES data would be of benefit.

In addition, from the out-patient project (4; details below) the Nuffield Trust have undertaken an analysis of shifts in outpatient attendances during Covid-19, which is available as a pre-print, and which the Nuffield Trust hope will be published soon. This will be a report that the Nuffield Trust update with the latest data soon.

The monthly data for the Covid oximetry @ home project (6; details below) has allowed the Nuffield Trust to keep NHS England (and NHSX and NHS Digital) as up to date as possible on possible impacts of this high priority scheme. The Nuffield Trust’s work may help NHS England to plan its approach to home monitoring of other conditions, post-Covid.

With respect to the evaluation of the Redthread youth violence programme (5; details below), this is one where the intervention itself (based in ED) was interrupted by Covid, and the monthly data has been valuable in helping us to study the recently re-launched service rapidly.

_____________________________________________________________________________________________________

The Nuffield Trust for Research and Policy Studies in Health Services (The Nuffield Trust) is an independent health research charity overseen by a board of Trustees including a number of senior NHS clinicians, managers and academics. The Nuffield Trust aims to improve the quality of health care that improves the health of people in the UK by providing evidence-based research and policy analysis and informing and generating debate. The Nuffield Trust undertakes work for the public good and within a research governance framework. These are the Nuffield Trust’s legitimate interests which provide a lawful basis for processing personal data under the General Data Protection Regulation (GDPR).

In everything the Nuffield Trust does, they strive to be:

• independent and free from vested interests;

• rigorous, robust and evidence-based in the work they undertake;

• relevant, supportive but also challenging when they need to be;

• open and engaging with all those they come into contact with;

• an organisation that makes a difference to the quality of policy-making and practice in the UK.

Mindful of the quote by the Chief Scientific Adviser for the Department of Health and Social Care that “Research is of no use unless it gets to the people who need to use it”, Nuffield Trust’s strategy to maximise impact is as follows:

• Investigate relevant research questions that are important to patients, commissioners, providers and other stakeholders;

• Co-produce research with patients, the public, professionals and managers to address these questions using rigorous and appropriate research methods applied to the best available data;

• Identify the implications of this research for current and future service delivery for different stakeholders;

• Present the evidence in an appropriate format for the range of target audiences;

• Be guided by input from evidence users, including patients and the public, in all of the above.

Challenges to health and care systems such as increases in life expectancy and multi-morbidities, developments in treatments and technologies, and wider economic pressures have resulted in a drive at national and local levels to develop and accelerate service innovations to benefit patients and the public. Currently, innovations do not spread as fast nor have the same degree of impact as in other sectors, particularly in terms of rapidly shaping frontline service delivery. There is therefore a need for new approaches to evaluation that provide robust evidence to meet these needs in a timely fashion. To address this need, the National Institute for Health Research (NIHR) Health Services and Delivery Research (HS&DR) programme have commissioned a Rapid Evaluation Research Team (RSET).

Three projects have been completed and seven are ongoing.

Completed Projects:

1) Special Measures for Quality (SMQ)

2) Identification and prioritisation of research topics in adult social care and social work

3) Virtual Wards/remote home monitoring for COVID-19 patients (Phase 1)

Ongoing Projects:

4) Innovations in Outpatient Services

5) Youth violence intervention programme (Redthread) for vulnerable young people (11-24 years) attending Emergency Departments

6) Virtual Wards/remote home monitoring for COVID-19 patients ""Covid Oximetry @ Home"" (Phase 2)

7) COVID Oximetry @home Care home extension

8) Pre-hospital triage for stroke patients

9) Peer supported social care in prisons

10) Centralisation of specialist healthcare services: a mixed methods programme (long project)

HES data have been, or are being used for projects numbered 1, 4, 5, 6, and will likely be used for a latter stage of 10.

The Nuffield Trust is part of this collaborative team funded by NIHR over 5 years (2018/9 to 2022/23) to conduct rapid evaluations of health and care service innovations, in close partnership with those who deliver, manage and use these services.

The service innovations may be ‘combinatorial’ and encompass several aspects of innovation such as technological, financial and service model re-design. To be successfully introduced and adopted by the health system, they need to demonstrate added value, cost-effectiveness, benefit to patients over current practice, and feasibility. At the same time, innovations generally require flexible and varied financial structures, and support mechanisms, to ensure they are scaled up and spread beyond local settings, whilst new service delivery models may demand innovative governance and payment arrangements. For these reasons, a flexible approach to applying new evaluation methods is important, in order to accommodate changing innovation processes and pathways. Developing these methods is necessary to address important research questions within this programme but will also be beneficial for future research.

The Nuffield Trust’s partners in the Rapid Service Evaluation Team (RSET) study are researchers from the Department of Applied Health Research (DAHR) at University College London (UCL) - although the data accessed through this agreement will be managed by the Nuffield Trust, and will not be shared with UCL colleagues or any other third-parties (unless aggregated in accordance with standard disclosure rules). The RSET team have a remit to produce timely findings of national relevance and immediate use to decision makers.

The Nuffield Trust’s and UCL's approach to evaluation will combine the questions, ‘What works at what cost?’ with ‘How and why?’ using a mix of innovative quantitative and qualitative methods. The team will enable provision of formative, as well as summative, feedback to people implementing innovations in health and care services. The data sets accessed through this agreement will be crucial to answering questions about the impact of these schemes on health and care. This will happen through high quality, timely research undertaken with a variety of methods. Over the five-year period the RSET team will identify approximately ten innovations to evaluate. NIHR, NHS England, DHSC and the RSET team’s stakeholder advisory board (which includes clinicians, managers, and patient and public representatives) will have advisory input into identifying potential innovations. There will be a focus on rapid evaluation of these schemes with some projects aiming to complete evaluation within 6 to 9 months. Others may have a slightly longer timeframe, but the period from inception to analysis will necessarily be short, to allow for timely formative feedback to the areas undertaking innovations.

All evaluations will conform to the following scope:

• The subject of the evaluation will be a health or care provider (or group of health and/or care providers) which has/have adopted a new way of working which differs to those that have been widely adopted by the health system – i.e. an ‘innovation’. Health or core providers may be NHS Trusts, Local Authorities or, for example, a group of district nurses. The innovation may be use of new technology or a financial and/or service model redesign.

• The HES data would potentially be used to calculate relevant metrics that those implementing the technology would be seeking to affect – for example – short term readmissions to hospital for the targeted groups in those areas. HES data would not necessarily be used in all of the evaluations. It would only be used where it would add value to the evaluation.

• The evaluations will attempt to determine whether there were any detectable changes in the metrics over time that might be associated with the introduction of the innovation.

• This would be done using a variety of techniques such as comparing metrics for the health care provider(s) pre and post introduction of the innovation or comparing the metrics of the health care provider(s) which adopted the innovation with health care provider(s) where the innovation is not in use.

• Standard hospital activity costing methods would be used to calculate the impact in terms of costs or savings to commissioners and hospitals and to model the impact of the innovation if adopted in larger areas (e.g. nationally).

• Alongside this quantitative analysis, local qualitative information gathering and analysis would be carried out to understand how and why the innovations might have had their particular impact in those local health economies.

The RSET team aim to minimise any delays (such as the approval process for data access) as much as possible in order to maximise time spent on each evaluation, to maximise the efficient use of researcher time and to maximise cost effective use of the NIHR grant. Note that the funders NIHR explicitly set up this programme to promote much more rapid analysis of service innovations, that might more quickly benefit health care services nationally.

The Nuffield Trust will be responsible for ensuring that all uses of the data will be within the agreed scope above.

The selection of research projects to be undertaken by the RSET team will be guided using a partnership approach, involving close working with the funder (NIHR), but also with other stakeholders via the RSET team’s Advisory Board, Patient and Public Involvement (PPI) input and other networks. The topics will be selected by initially producing a long list of potential research projects based on recommendations from the funder, the Advisory Board, and the PPI panel. This will be supplemented with a range of activities by the RSET team to identify topics of current interest that it would be timely to investigate. These activities will include desk-based research of published outputs, analysis of social media, interaction with stakeholders at dissemination events, contacting networks, and by people alerting the RSET team to innovations through their interactive website.

A shortlist of potential research projects will be produced using pre-determined criteria to be agreed with the funder, the Advisory Board, and PPI panel. These criteria are likely to include considerations of timeliness, strategic importance, and feasibility. Topics identified by the funder will be prioritised.

NIHR have commissioned some projects including projects 1, 6 and 7 listed above. A significant scoping phase is undertaken, and all projects require NIHR approval before progressing.

The criteria are:

- Strategic importance (Is the innovation within scope? How large an impact could the innovation have? Are other evaluations being conducted? Or have they been completed already? Would the results of the evaluation be of national relevance? If it is a local or regional evaluation, is there be enough potential for others to learn from it? Would the evidence be of immediate use? Would the evidence have long-lasting usefulness?)

- Feasibility (Is it possible to evaluate the innovation within a reasonable timeframe? Could a ‘useful’ evaluation be completed within the scale of RSET? Is (enough) data available? Does data of sufficient granularity exist? Covering the right areas? Over the right period? Is it of sufficient quality? If data does not exist, can it be collected? Is the innovation ready to be evaluated? Would the Nuffield Trust be evaluating a stable innovation, or is the intervention in continuous change? Would there be sufficient support from participating services for the evaluation? Would there be sufficient support from the funder and patient and public involvement group for the evaluation? How would the evaluation fit within the RSET portfolio of evaluations? Are the Nuffield Trust covering a sufficient variety of service innovations? How would appropriate team resources be allocated? Is there the right capacity to do the evaluation right now?)

- Other criteria (Are there any ethical issues to consider with the innovation? Would there be any ethical issues to consider when conducting an evaluation of this innovation? Who stands to benefit from the innovation? What would the equity health impact be? Does the evaluation afford the team the opportunity to push the methodological boundaries of health service research? Is there a potential for the innovation to generate any other positive or negative externalities that the Nuffield Trust should consider?).

Once a project has been identified and reviewed by the funder (to advise the RSET team as to whether the project is within the scope of the research grant), the project will enter a scoping phase. A ‘topic specification form’ is filled in at this point. This includes a description of the motivation for undertaking the research, the research questions, the proposed work and the timescales. This is reviewed by the Nuffield Trust alongside its RSET partners and the funder, and a decision is taken by the RSET team as to whether the project should proceed to a full evaluation.

Where the decision is made to proceed to evaluation, a detailed research protocol is developed by the RSET team (the Nuffield Trust and its UCL partners). This protocol develops on the evaluation research questions and outlines in detail the design and methods proposed to answer the research questions including the team's plans for data processing. The protocol will include references to the evaluation team’s approach to PPI, ethical issues, data and information management, risk management and dissemination of findings. The protocols will be reviewed by independent experts in healthcare organisation and management, and also by project stakeholders, NIHR and patient representatives. The aim of this stage of review will be to advise the RSET team of any shortcomings in the proposed work, or of additional research questions that the RSET might find valuable to address.

In developing the evaluation protocols the Nuffield Trust lead will ensure that any proposed use of HES data conforms to the agreed scope above, with appropriate input from our Data Protection Officer (DPO). The Nuffield Trust project lead will ensure that:

• Projects have clearly defined objectives and analysis plans;

• In each case, the use of the HES data is necessary and proportionate to the purpose of the project – this will include consideration of the necessity for use of each individual HES dataset; the number of years of data; the sizes of any cohorts or control cohorts derived from the data, and the inclusion and exclusion criteria (such as presence of specific diagnostic or procedure codes);

• Appropriate safeguards are in place to protect confidentiality; minimise risks of re-identification and use of excessive data beyond necessity

The individual or individuals within Nuffield Trust who will carry out the project will be bound by an analysis plan detailing what data is permitted for use in the project and how it shall be processed. The Nuffield Trust aim to start each project (or sub-project workstream, as appropriate) with an assessment of the data necessary for analysis, and the anticipated analytical steps.

The Nuffield Trust and UCL are joint data controllers for the data under this Data Sharing Agreement. As described above, the funder, the Advisory Board and the PPI panel offer advice on what service evaluation projects might be undertaken. Additionally, the funder may have a role in determining that a given project is not within scope of the research grant. However, decisions in respect of use of the data under this Agreement are taken jointly by the Nuffield Trust and UCL without obligation to the funder or any other body. The Nuffield Trust and UCL will have joint responsibility for determining the purposes for processing the data under this Agreement to support the delivery of the projects and for determining the manner in which the data will be processed.

With reference to the Information Commissioner’s published guidance on data controllership, the Nuffield Trust and UCL can confirm that these two organisations are jointly responsible for directing the research project. Specifically they decide:

 to collect the data under this Agreement and the legal basis for doing so;

 which items of data to collect, i.e. the content of the data;

 the purposes the data are to be used for;

 which individuals to collect data about;

 whether to disclose the data – the data will not be disclosed to any third party;

 whether subject access and other individuals’ rights apply; and

 how long to retain the data

There are several projects at various stages of progress. Some which have either made use of HES, or are likely to, are listed here:

• An evaluation of NHSI interventions for trusts in Special Measures for Quality and for those on the ‘challenged providers’ list

• An evaluation of a youth violence prevention intervention at University College Hospital London

• An evaluation of the centralisation of specialised health care services

o An evaluation of innovations in outpatient service delivery

o An evaluation of the impact of remote monitoring services for Covid-19 patients

The nature of this NIHR programme is that further projects are not currently specified in this Agreement but will be identified at relatively short notice over the next several years. This Agreement permits the Nuffield Trust to use subsets of the data for the purposes of the projects specified above and for the purposes of the projects to be identified for the purpose of this NIHR programme which must be compliant with the description of such projects given above.

The Nuffield Trust will send NHS Digital details of each RSET project’s progress as part of an annual update.

Some of the innovations the Nuffield Trust will look at will inevitably need to be analysed with reference to long term historic data and so the Nuffield Trust will require inpatient, outpatient and A&E data for all of England from 2008/09 to 2022/23.

***Amendment Request - Increased Frequency Of Distribution Requested***

A change to the schedule of distribution is required to increase distribution to monthly for a period of 12 months.

The Nuffield Trust has a number of projects planned (covered under DARS-NIC-226261-M2T0Q-v2 which expires 30/06/22) to investigate the impact of Covid-19 on the use of health services which are dependent on or would benefit from access to more timely (ie monthly rather than quarterly) HES data. These include, for example, an analysis of outpatient attendances with a view to identify variations in activity by trust and specialty over the early pandemic period as part of the evaluation work for the NIHR-funded Rapid Service Evaluation Team.

Under this agreement, the Nuffield Trust is also scoping out analyses to monitor the impact of remote monitoring services for Covid-19 patients (supported by PHE who are providing aggregate data for the study, but are not involved in the study design or analysis); this may involve the need for timely analysis of hospital data as a second wave of Covid-19 progresses. Additionally, the trust is preparing to test the feasibility of using HES data to measure the impact of a youth violence reduction scheme in London and beyond; as the numbers of young people referred to the service were lower than anticipated due to Covid-19 restrictions, more timely data will help to provide the best statistical chance of measuring impact within the project’s rapid timescales.

Update 4/4/21

Justification for change in frequency of data distribution:

The Nuffield Trust are requesting to continue to receive monthly data, because this has been very beneficial in the last 12 months. Within evaluations it has been valuable to include the most recent data in analysis. Specifically, The Nuffield Trust are using monthly data in the evaluations of Covid oximetry at home, and the Covid virtual wards, mentioned above. For example, having monthly data enables The Nuffield Trust to check whether observations in the data reflect end of data effects, as a result of missing data on patients still in hospital, or underlying trends.

Note 26/4/61 ECDS Record Identifier (Identifiable) will be used as the only available unique identifier for each row in ECDS.

The Nuffield Trust and the wider RSET team are highly aware of the need to balance rapid formative feedback (to areas conducting innovations) with the need to produce summative findings (shared locally and nationally, including academic publications).

The RSET team will use innovative approaches to sharing findings, including formative feedback locally during scoping and the full evaluation, and active dissemination of findings more widely so that they can be taken up quickly in other parts of the health and care system.

The Trust will use their extensive communications facilities & networks for dissemination (including professionals in the fields of media relations, public affairs, digital communications and event management), working with the UCL communications team, National Institute for Health Research (NIHR) Dissemination Centre, and Health Services and Delivery Research (HS&DR) programme to maximise the impact of evaluation findings.

The RSET team’s communications/dissemination strategy will:

• Identify the audience: who needs to know?

• Set a timetable for dissemination outputs: weeks, not months

• Decide on appropriate media and form of communications

• Ensure written outputs are as simple as possible

• Provide context and other evidence relevant to each evaluation.

The RSET team will develop a communication strategy for the overall programme and each project in collaboration with NIHR,PPI & end users, including stakeholder dissemination list; social media strategy; bespoke events; conference presentations; open-access journal articles; sharing findings with trade press, such as the Health Service Journal and National Health Executive.

GP = General practitioners; STP = Sustainability and transformation partnerships; CEOs = Chief Executives.

All RSET projects have PPI input. Patients and the public have been involved in the following ways:

PPI representatives were involved in designing the COVID Oximetry @home questionnaires, as well as providing feedback on the early findings from the workforce and patient surveys.

A new PPI representative was recruited for the Innovations in Outpatient Services project to provide perspectives on their experiences of care and different outpatients services, and two PPI representatives commented on the prpject protocol.

RSET’s PPI strategy has been revised, in collaboration with PPI representatives. The revised strategy was distributed in January 2021. Since then, three new PPI members have been recruited.

The RSET team will plan outputs and events to stimulate formative learning over the course of evaluations. The RSET team will reach target audiences through a variety of tailored methods, from graphics and tweetchats on social media, to blogs, vlogs and blogshots, opinion articles, short films and visualisations to traditional media coverage and targeted two-way conversation events through, for example, Nuffield Trust’s extensive learning sets (involving GPs, STP leads, acute trust CEOs etc).

The RSET team have also set up a micro site on the Nuffield Trust website (https://www.nuffieldtrust.org.uk/project/rset-the-rapid-service-evaluation-team) devoted to the whole RSET programme. This will serve as a focus for reaching out to the NHS and other organisations to advertise the RSET programme and its findings.

The Trust will review and evaluate the impact of their communications activity and refine their approach for each new output.

- The Nuffield Trust use HES data for several Quality Watch indicators, for example hip replacements and cancelled operations. This part of the quality and equity work, and contributes to developing the evidence base and understanding of quality of care and inequalities in health.

- The Nuffield Trust have used HES data to track emergency admissions in integrated care 'pioneer' areas over time. This is part of the politics, legislation and governance programme, and the work is part of a large scale of evaluation which will contribute to learning and on effectiveness of integrated care.

- The Nuffield Trust continue to use HES data in the ongoing work on prisoner health and have presented interim findings from the current project to key stakeholders. This work is part of the quality and equity programme and aims to identify priorities for improving health services in prisons, and engage national stakeholders in addressing these.

- For the community services project the Nuffield Trust started by examining data quality issues and working with other stakeholders on criteria for which areas could be included in analysis, based on data quality, along with developing metrics for analysing trends in community provision. This is part of the quality and equity programme and aims to understand the impact of covid-19 on community services provisions.

- The analysis of quality of ethnicity coding, also part of the quality and equity work, has led to a report with recommendations for improvements in data quality and analysis, which are vital to properly understanding and addressing ethnic inequalities.

Completed Projects:

I. Special Measures for Quality (SMQ)

The team carried out a considerable amount of dissemination and engagement with key stakeholders (Department of Health and Social Care, Care Quality Commission, NHS Improvement/England). The latest output (since July 2020) are a rapid literature review published in the International Journal of Health Policy and Management (IJHPM) please see https://www.ijhpm.com/article_3943.html), and a Q&A blogpost on the impact of improvement interventions in NHS trusts (please see - https://www.nuffieldtrust.org.uk/news-item/special-measures-for-quality-a-q-a-on-the-impact-of-improvement-interventions-in-nhs-trusts). The team are currently working on further papers for submission in the next few months and continuing with dissemination remotely to relevant stakeholders.

II. Identification and prioritisation of research topics in adult social care and social work

The paper outlining adaptation of the James Lind Alliance (JLA) method for rapid identification was published in the Health Research Policy and Systems (HARPS) in March 2021 (please see - https://health-policy-systems.biomedcentral.com/articles/10.1186/s12961-021-00693-2).

III. Virtual Wards/remote home monitoring for COVID-19 patients (Phase 1)

The team presented findings from phase I to national leaders including Public Health England, NHS England, NHS Digital, and National Institute for Health Research and in webinars in all English regions. Further, the team presented the findings to over 400 people around the country planning and developing virtual wards, including the NHS@Home Pulse Oximetry Learning Network, COVID-19 Virtual Wards Community of Practice, Liverpool City Region/Northwest Coast, and London COVID Care at Home (Virtual Ward) Webinar.

All outputs will be aggregated with small numbers suppressed in line with the HES Analysis Guide.

The analysis of the Care Quality Commission's special measures for quality regime carried out under the RSET project has led to directly led the project team presenting findings to senior staff at the Department for Health and Social care and NHSEI's Joint strategic oversight group.

The other RSET studies which have used HES data are still in progress, and final yielded benefits are not yet available. However, feedback from NHS England (via the mid-point review for RSET) recognises the contribution the evaluation has already made to establishing the Covid oximetry at home programme, and the direct impact this has had on improving treatment options available to Covid patients.

The Nuffield Trust for Research and Policy Studies in Health Services (The Nuffield Trust) is an independent health research charity overseen by a board of Trustees including a number of senior NHS clinicians, managers and academics. The Nuffield Trust aims to improve the quality of health care that improves the health of people in the UK by providing evidence-based research and policy analysis and informing and generating debate. The Nuffield Trust undertakes work for the public good and within a research governance framework. These are the Nuffield Trust’s legitimate interests which provide a lawful basis for processing personal data under the General Data Protection Regulation (GDPR).

In everything the Nuffield Trust does, they strive to be:

• independent and free from vested interests;

• rigorous, robust and evidence-based in the work they undertake;

• relevant, supportive but also challenging when they need to be;

• open and engaging with all those they come into contact with;

• an organisation that makes a difference to the quality of policy-making and practice in the UK.

Mindful of the quote by the Chief Scientific Adviser for the Department of Health and Social Care that “Research is of no use unless it gets to the people who need to use it”, Nuffield Trust’s strategy to maximise impact is as follows:

• Investigate relevant research questions that are important to patients, commissioners, providers and other stakeholders;

• Co-produce research with patients, the public, professionals and managers to address these questions using rigorous and appropriate research methods applied to the best available data;

• Identify the implications of this research for current and future service delivery for different stakeholders;

• Present the evidence in an appropriate format for the range of target audiences;

• Be guided by input from evidence users, including patients and the public, in all of the above.

Challenges to health and care systems such as increases in life expectancy and multi-morbidities, developments in treatments and technologies, and wider economic pressures have resulted in a drive at national and local levels to develop and accelerate service innovations to benefit patients and the public. Currently, innovations do not spread as fast nor have the same degree of impact as in other sectors, particularly in terms of rapidly shaping frontline service delivery. There is therefore a need for new approaches to evaluation that provide robust evidence to meet these needs in a timely fashion. To address this need, the National Institute for Health Research (NIHR) Health Services and Delivery Research (HS&DR) programme have commissioned a Rapid Evaluation Research Team (RSET).

The Nuffield Trust is part of this collaborative team funded by NIHR over 5 years (2018/9 to 2022/23) to conduct rapid evaluations of health and care service innovations, in close partnership with those who deliver, manage and use these services.

The service innovations may be ‘combinatorial’ and encompass several aspects of innovation such as technological, financial and service model re-design. To be successfully introduced and adopted by the health system, they need to demonstrate added value, cost-effectiveness, benefit to patients over current practice, and feasibility. At the same time, innovations generally require flexible and varied financial structures, and support mechanisms, to ensure they are scaled up and spread beyond local settings, whilst new service delivery models may demand innovative governance and payment arrangements. For these reasons, a flexible approach to applying new evaluation methods is important, in order to accommodate changing innovation processes and pathways. Developing these methods is necessary to address important research questions within this programme but will also be beneficial for future research.

The Nuffield Trust’s partners in the Rapid Service Evaluation Team (RSET) study are researchers from the Department of Applied Health Research (DAHR) at University College London (UCL) - although the data accessed through this agreement will be managed by the Nuffield Trust, and will not be shared with UCL colleagues or any other third-parties (unless aggregated in accordance with standard disclosure rules). The RSET team have a remit to produce timely findings of national relevance and immediate use to decision makers.

The Nuffield Trust’s and UCL's approach to evaluation will combine the questions, ‘What works at what cost?’ with ‘How and why?’ using a mix of innovative quantitative and qualitative methods. The team will enable provision of formative, as well as summative, feedback to people implementing innovations in health and care services. The data sets accessed through this agreement will be crucial to answering questions about the impact of these schemes on health and care. This will happen through high quality, timely research undertaken with a variety of methods. Over the five-year period the RSET team will identify approximately ten innovations to evaluate. NIHR, NHS England, DHSC and the RSET team’s stakeholder advisory board (which includes clinicians, managers, and patient and public representatives) will have advisory input into identifying potential innovations. There will be a focus on rapid evaluation of these schemes with some projects aiming to complete evaluation within 6 to 9 months. Others may have a slightly longer timeframe, but the period from inception to analysis will necessarily be short, to allow for timely formative feedback to the areas undertaking innovations.

All evaluations will conform to the following scope:

- The subject of the evaluation will be a health or care provider (or group of health and/or care providers) which has/have adopted a new way of working which differs to those that have been widely adopted by the health system – i.e. an ‘innovation’. Health or core providers may be NHS Trusts, Local Authorities or, for example, a group of district nurses. The innovation may be use of new technology or a financial and/or service model redesign.

- The HES data would potentially be used to calculate relevant metrics that those implementing the technology would be seeking to affect – for example – short term readmissions to hospital for the targeted groups in those areas. HES data would not necessarily be used in all of the evaluations. It would only be used where it would add value to the evaluation.

- The evaluations will attempt to determine whether there were any detectable changes in the metrics over time that might be associated with the introduction of the innovation.

- This would be done using a variety of techniques such as comparing metrics for the health care provider(s) pre and post introduction of the innovation or comparing the metrics of the health care provider(s) which adopted the innovation with health care provider(s) where the innovation is not in use.

- Standard hospital activity costing methods would be used to calculate the impact in terms of costs or savings to commissioners and hospitals and to model the impact of the innovation if adopted in larger areas (e.g. nationally).

- Alongside this quantitative analysis, local qualitative information gathering and analysis would be carried out to understand how and why the innovations might have had their particular impact in those local health economies.

The RSET team aim to minimise any delays (such as the approval process for data access) as much as possible in order to maximise time spent on each evaluation, to maximise the efficient use of researcher time and to maximise cost effective use of the NIHR grant. Note that the funders NIHR explicitly set up this programme to promote much more rapid analysis of service innovations, that might more quickly benefit health care services nationally.

The Nuffield Trust will be responsible for ensuring that all uses of the data will be within the agreed scope above.

The selection of research projects to be undertaken by the RSET team will be guided using a partnership approach, involving close working with the funder (NIHR), but also with other stakeholders via the RSET team’s Advisory Board, Patient and Public Involvement (PPI) input and other networks. The topics will be selected by initially producing a long list of potential research projects based on recommendations from the funder, the Advisory Board, and the PPI panel. This will be supplemented with a range of activities by the RSET team to identify topics of current interest that it would be timely to investigate. These activities will include desk-based research of published outputs, analysis of social media, interaction with stakeholders at dissemination events, contacting networks, and by people alerting the RSET team to innovations through their interactive website.

A shortlist of potential research projects will be produced using pre-determined criteria to be agreed with the funder, the Advisory Board, and PPI panel. These criteria are likely to include considerations of timeliness, strategic importance, and feasibility. Topics identified by the funder will be prioritised.

Once a project has been identified and reviewed by the funder (to advise the RSET team as to whether the project is within the scope of the research grant), the project will enter a scoping phase. A ‘topic specification form’ is filled in at this point. This includes a description of the motivation for undertaking the research, the research questions, the proposed work and the timescales. This is reviewed by the Nuffield Trust alongside its RSET partners and the funder, and a decision is taken by the RSET team as to whether the project should proceed to a full evaluation.

Where the decision is made to proceed to evaluation, a detailed research protocol is developed by the RSET team (the Nuffield Trust and its UCL partners). This protocol develops on the evaluation research questions and outlines in detail the design and methods proposed to answer the research questions including the team's plans for data processing. The protocol will include references to the evaluation team’s approach to PPI, ethical issues, data and information management, risk management and dissemination of findings. The protocols will be reviewed by independent experts in healthcare organisation and management, and also by project stakeholders, NIHR and patient representatives. The aim of this stage of review will be to advise the RSET team of any shortcomings in the proposed work, or of additional research questions that the RSET might find valuable to address.

In developing the evaluation protocols the Nuffield Trust lead will ensure that any proposed use of HES data conforms to the agreed scope above, with appropriate input from our Data Protection Officer (DPO). The Nuffield Trust project lead will ensure that:

• Projects have clearly defined objectives and analysis plans;

• In each case, the use of the HES data is necessary and proportionate to the purpose of the project – this will include consideration of the necessity for use of each individual HES dataset; the number of years of data; the sizes of any cohorts or control cohorts derived from the data, and the inclusion and exclusion criteria (such as presence of specific diagnostic or procedure codes);

• Appropriate safeguards are in place to protect confidentiality; minimise risks of re-identification and use of excessive data beyond necessity

The individual or individuals within Nuffield Trust who will carry out the project will be bound by an analysis plan detailing what data is permitted for use in the project and how it shall be processed.

The Nuffield Trust and UCL are joint data controllers for the data under this Data Sharing Agreement. As described above, the funder, the Advisory Board and the PPI panel offer advice on what service evaluation projects might be undertaken. Additionally, the funder may have a role in determining that a given project is not within scope of the research grant. However, decisions in respect of use of the data under this Agreement are taken jointly by the Nuffield Trust and UCL without obligation to the funder or any other body. The Nuffield Trust and UCL will have joint responsibility for determining the purposes for processing the data under this Agreement to support the delivery of the projects and for determining the manner in which the data will be processed.

With reference to the Information Commissioner’s published guidance on data controllership, the Nuffield Trust and UCL can confirm that these two organisations are jointly responsible for directing the research project. Specifically they decide:

- to collect the data under this Agreement and the legal basis for doing so;

- which items of data to collect, i.e. the content of the data;

- the purposes the data are to be used for;

- which individuals to collect data about;

- whether to disclose the data – the data will not be disclosed to any third party;

- whether subject access and other individuals’ rights apply; and

- how long to retain the data

There are three projects currently at an early scoping phase. It has not yet been determined whether these projects will need to use HES data and these are provided as examples of innovations which will be investigated and which may potentially use HES data. These are as follows:

• An evaluation of NHSI interventions for trusts in Special Measures for Quality and for those on the ‘challenged providers’ list

• An evaluation of the introduction of the Buurztorg model (of self-managing community care nursing teams) into areas in the UK

• An evaluation of the centralisation of specialised health care services

The nature of this NIHR programme is that further projects are not currently specified in this Agreement but will be identified at relatively short notice over the next several years. This Agreement permits the Nuffield Trust to use subsets of the data for the purposes of the projects specified above and for the purposes of the projects to be identified for the purpose of this NIHR programme which must be compliant with the description of such projects given above.

The Nuffield Trust will send NHS Digital details of each RSET project’s progress as part of an annual update.

Some of the innovations the Nuffield Trust will look at will inevitably need to be analysed with reference to long term historic data and so the Nuffield Trust will require inpatient, outpatient and A&E data for all of England from 2008/09 to 2022/23.

***Amendment Request - Increased Frequency Of Distribution Requested***

A change to the schedule of distribution is required to increase distribution to monthly for a period of 12 months.

The Nuffield Trust has a number of projects planned (covered under DARS-NIC-226261-M2T0Q-v2 which expires 30/06/22) to investigate the impact of Covid-19 on the use of health services which are dependent on or would benefit from access to more timely (ie monthly rather than quarterly) HES data. These include, for example, an analysis of outpatient attendances with a view to identify variations in activity by trust and specialty over the early pandemic period as part of the evaluation work for the NIHR-funded Rapid Service Evaluation Team.

Under this agreement, the Nuffield Trust is also scoping out analyses to monitor the impact of remote monitoring services for Covid-19 patients (supported by PHE); this may involve the need for timely analysis of hospital data as a second wave of Covid-19 progresses. Additionally, the trust is preparing to test the feasibility of using HES data to measure the impact of a youth violence reduction scheme in London and beyond; as the numbers of young people referred to the service were lower than anticipated due to Covid-19 restrictions, more timely data will help to provide the best statistical chance of measuring impact within the project’s rapid timescales.

The Nuffield Trust and the wider RSET team are highly aware of the need to balance rapid formative feedback (to areas conducting innovations) with the need to produce summative findings (shared locally and nationally, including academic publications).

The RSET team will use innovative approaches to sharing findings, including formative feedback locally during scoping and the full evaluation, and active dissemination of findings more widely so that they can be taken up quickly in other parts of the health and care system.

The Trust will use their extensive communications facilities & networks for dissemination (including professionals in the fields of media relations, public affairs, digital communications and event management), working with the UCL communications team, National Institute for Health Research (NIHR) Dissemination Centre, and Health Services and Delivery Research (HS&DR) programme to maximise the impact of evaluation findings.

The RSET team’s communications/dissemination strategy will:

• Identify the audience: who needs to know?

• Set a timetable for dissemination outputs: weeks, not months

• Decide on appropriate media and form of communications

• Ensure written outputs are as simple as possible

• Provide context and other evidence relevant to each evaluation.

The RSET team will develop a communication strategy for the overall programme and each project in collaboration with NIHR,PPI & end users, including stakeholder dissemination list; social media strategy; bespoke events; conference presentations; open-access journal articles; sharing findings with trade press, such as the Health Service Journal and National Health Executive.

In addition, the RSET team will plan outputs and events to stimulate formative learning over the course of evaluations. The RSET team will reach target audiences through a variety of tailored methods, from graphics and tweetchats on social media, to blogs, vlogs and blogshots, opinion articles, short films and visualisations to traditional media coverage and targeted two-way conversation events through, for example, Nuffield Trust’s extensive learning sets (involving GPs, STP leads, acute trust CEOs etc).

The RSET team have also set up a micro site on the Nuffield Trust website (https://www.nuffieldtrust.org.uk/project/rset-the-rapid-service-evaluation-team) devoted to the whole RSET programme. This will serve as a focus for reaching out to the NHS and other organisations to advertise the RSET programme and its findings.

The Trust will review and evaluate the impact of their communications activity and refine their approach for each new output.

All outputs will be aggregated with small numbers suppressed in line with the HES Analysis Guide.

The analysis of the Care Quality Commission’s special measures for quality regime carried out under the RSET project has led to directly led the project team presenting findings to senior staff at the Department for Health and Social care and NHSEI’s Joint strategic oversight group.

The Nuffield Trust for Research and Policy Studies in Health Services (The Nuffield Trust) is an independent health research charity overseen by a board of Trustees including a number of senior NHS clinicians, managers and academics. The Nuffield Trust aims to improve the quality of health care that improves the health of people in the UK by providing evidence-based research and policy analysis and informing and generating debate. The Nuffield Trust undertakes work for the public good and within a research governance framework. These are the Nuffield Trust’s legitimate interests which provide a lawful basis for processing personal data under the General Data Protection Regulation (GDPR).

In everything the Nuffield Trust does, they strive to be:

• independent and free from vested interests;

• rigorous, robust and evidence-based in the work they undertake;

• relevant, supportive but also challenging when they need to be;

• open and engaging with all those they come into contact with;

• an organisation that makes a difference to the quality of policy-making and practice in the UK.

Mindful of the quote by the Chief Scientific Adviser for the Department of Health and Social Care that “Research is of no use unless it gets to the people who need to use it”, Nuffield Trust’s strategy to maximise impact is as follows:

• Investigate relevant research questions that are important to patients, commissioners, providers and other stakeholders;

• Co-produce research with patients, the public, professionals and managers to address these questions using rigorous and appropriate research methods applied to the best available data;

• Identify the implications of this research for current and future service delivery for different stakeholders;

• Present the evidence in an appropriate format for the range of target audiences;

• Be guided by input from evidence users, including patients and the public, in all of the above.

Challenges to health and care systems such as increases in life expectancy and multi-morbidities, developments in treatments and technologies, and wider economic pressures have resulted in a drive at national and local levels to develop and accelerate service innovations to benefit patients and the public. Currently, innovations do not spread as fast nor have the same degree of impact as in other sectors, particularly in terms of rapidly shaping frontline service delivery. There is therefore a need for new approaches to evaluation that provide robust evidence to meet these needs in a timely fashion. To address this need, the National Institute for Health Research (NIHR) Health Services and Delivery Research (HS&DR) programme have commissioned a Rapid Evaluation Research Team (RSET).

The Nuffield Trust is part of this collaborative team funded by NIHR over 5 years (2018/9 to 2022/23) to conduct rapid evaluations of health and care service innovations, in close partnership with those who deliver, manage and use these services.

The service innovations may be ‘combinatorial’ and encompass several aspects of innovation such as technological, financial and service model re-design. To be successfully introduced and adopted by the health system, they need to demonstrate added value, cost-effectiveness, benefit to patients over current practice, and feasibility. At the same time, innovations generally require flexible and varied financial structures, and support mechanisms, to ensure they are scaled up and spread beyond local settings, whilst new service delivery models may demand innovative governance and payment arrangements. For these reasons, a flexible approach to applying new evaluation methods is important, in order to accommodate changing innovation processes and pathways. Developing these methods is necessary to address important research questions within this programme but will also be beneficial for future research.

The Nuffield Trust’s partners in the Rapid Service Evaluation Team (RSET) study are researchers from the Department of Applied Health Research (DAHR) at University College London (UCL) - although the data accessed through this agreement will be managed by the Nuffield Trust, and will not be shared with UCL colleagues or any other third-parties (unless aggregated in accordance with standard disclosure rules). The RSET team have a remit to produce timely findings of national relevance and immediate use to decision makers.

The Nuffield Trust’s and UCL's approach to evaluation will combine the questions, ‘What works at what cost?’ with ‘How and why?’ using a mix of innovative quantitative and qualitative methods. The team will enable provision of formative, as well as summative, feedback to people implementing innovations in health and care services. The data sets accessed through this agreement will be crucial to answering questions about the impact of these schemes on health and care. This will happen through high quality, timely research undertaken with a variety of methods. Over the five-year period the RSET team will identify approximately ten innovations to evaluate. NIHR, NHS England, DHSC and the RSET team’s stakeholder advisory board (which includes clinicians, managers, and patient and public representatives) will have advisory input into identifying potential innovations. There will be a focus on rapid evaluation of these schemes with some projects aiming to complete evaluation within 6 to 9 months. Others may have a slightly longer timeframe, but the period from inception to analysis will necessarily be short, to allow for timely formative feedback to the areas undertaking innovations.

All evaluations will conform to the following scope:

- The subject of the evaluation will be a health or care provider (or group of health and/or care providers) which has/have adopted a new way of working which differs to those that have been widely adopted by the health system – i.e. an ‘innovation’. Health or core providers may be NHS Trusts, Local Authorities or, for example, a group of district nurses. The innovation may be use of new technology or a financial and/or service model redesign.

- The HES data would potentially be used to calculate relevant metrics that those implementing the technology would be seeking to affect – for example – short term readmissions to hospital for the targeted groups in those areas. HES data would not necessarily be used in all of the evaluations. It would only be used where it would add value to the evaluation.

- The evaluations will attempt to determine whether there were any detectable changes in the metrics over time that might be associated with the introduction of the innovation.

- This would be done using a variety of techniques such as comparing metrics for the health care provider(s) pre and post introduction of the innovation or comparing the metrics of the health care provider(s) which adopted the innovation with health care provider(s) where the innovation is not in use.

- Standard hospital activity costing methods would be used to calculate the impact in terms of costs or savings to commissioners and hospitals and to model the impact of the innovation if adopted in larger areas (e.g. nationally).

- Alongside this quantitative analysis, local qualitative information gathering and analysis would be carried out to understand how and why the innovations might have had their particular impact in those local health economies.

The RSET team aim to minimise any delays (such as the approval process for data access) as much as possible in order to maximise time spent on each evaluation, to maximise the efficient use of researcher time and to maximise cost effective use of the NIHR grant. Note that the funders NIHR explicitly set up this programme to promote much more rapid analysis of service innovations, that might more quickly benefit health care services nationally.

The Nuffield Trust will be responsible for ensuring that all uses of the data will be within the agreed scope above.

The selection of research projects to be undertaken by the RSET team will be guided using a partnership approach, involving close working with the funder (NIHR), but also with other stakeholders via the RSET team’s Advisory Board, Patient and Public Involvement (PPI) input and other networks. The topics will be selected by initially producing a long list of potential research projects based on recommendations from the funder, the Advisory Board, and the PPI panel. This will be supplemented with a range of activities by the RSET team to identify topics of current interest that it would be timely to investigate. These activities will include desk-based research of published outputs, analysis of social media, interaction with stakeholders at dissemination events, contacting networks, and by people alerting the RSET team to innovations through their interactive website.

A shortlist of potential research projects will be produced using pre-determined criteria to be agreed with the funder, the Advisory Board, and PPI panel. These criteria are likely to include considerations of timeliness, strategic importance, and feasibility. Topics identified by the funder will be prioritised.

Once a project has been identified and reviewed by the funder (to advise the RSET team as to whether the project is within the scope of the research grant), the project will enter a scoping phase. A ‘topic specification form’ is filled in at this point. This includes a description of the motivation for undertaking the research, the research questions, the proposed work and the timescales. This is reviewed by the Nuffield Trust alongside its RSET partners and the funder, and a decision is taken by the RSET team as to whether the project should proceed to a full evaluation.

Where the decision is made to proceed to evaluation, a detailed research protocol is developed by the RSET team (the Nuffield Trust and its UCL partners). This protocol develops on the evaluation research questions and outlines in detail the design and methods proposed to answer the research questions including the team's plans for data processing. The protocol will include references to the evaluation team’s approach to PPI, ethical issues, data and information management, risk management and dissemination of findings. The protocols will be reviewed by independent experts in healthcare organisation and management, and also by project stakeholders, NIHR and patient representatives. The aim of this stage of review will be to advise the RSET team of any shortcomings in the proposed work, or of additional research questions that the RSET might find valuable to address.

In developing the evaluation protocols the Nuffield Trust lead will ensure that any proposed use of HES data conforms to the agreed scope above, with appropriate input from our Data Protection Officer (DPO). The Nuffield Trust project lead will ensure that:

• Projects have clearly defined objectives and analysis plans;

• In each case, the use of the HES data is necessary and proportionate to the purpose of the project – this will include consideration of the necessity for use of each individual HES dataset; the number of years of data; the sizes of any cohorts or control cohorts derived from the data, and the inclusion and exclusion criteria (such as presence of specific diagnostic or procedure codes);

• Appropriate safeguards are in place to protect confidentiality; minimise risks of re-identification and use of excessive data beyond necessity

The individual or individuals within Nuffield Trust who will carry out the project will be bound by an analysis plan detailing what data is permitted for use in the project and how it shall be processed.

The Nuffield Trust and UCL are joint data controllers for the data under this Data Sharing Agreement. As described above, the funder, the Advisory Board and the PPI panel offer advice on what service evaluation projects might be undertaken. Additionally, the funder may have a role in determining that a given project is not within scope of the research grant. However, decisions in respect of use of the data under this Agreement are taken jointly by the Nuffield Trust and UCL without obligation to the funder or any other body. The Nuffield Trust and UCL will have joint responsibility for determining the purposes for processing the data under this Agreement to support the delivery of the projects and for determining the manner in which the data will be processed.

With reference to the Information Commissioner’s published guidance on data controllership, the Nuffield Trust and UCL can confirm that these two organisations are jointly responsible for directing the research project. Specifically they decide:

- to collect the data under this Agreement and the legal basis for doing so;

- which items of data to collect, i.e. the content of the data;

- the purposes the data are to be used for;

- which individuals to collect data about;

- whether to disclose the data – the data will not be disclosed to any third party;

- whether subject access and other individuals’ rights apply; and

- how long to retain the data

There are three projects currently at an early scoping phase. It has not yet been determined whether these projects will need to use HES data and these are provided as examples of innovations which will be investigated and which may potentially use HES data. These are as follows:

• An evaluation of NHSI interventions for trusts in Special Measures for Quality and for those on the ‘challenged providers’ list

• An evaluation of the introduction of the Buurztorg model (of self-managing community care nursing teams) into areas in the UK

• An evaluation of the centralisation of specialised health care services

The nature of this NIHR programme is that further projects are not currently specified in this Agreement but will be identified at relatively short notice over the next several years. This Agreement permits the Nuffield Trust to use subsets of the data for the purposes of the projects specified above and for the purposes of the projects to be identified for the purpose of this NIHR programme which must be compliant with the description of such projects given above.

The Nuffield Trust will send NHS Digital details of each RSET project’s progress as part of an annual update.

Some of the innovations the Nuffield Trust will look at will inevitably need to be analysed with reference to long term historic data and so the Nuffield Trust will require inpatient, outpatient and A&E data for all of England from 2008/09 to 2022/23.

The Nuffield Trust and the wider RSET team are highly aware of the need to balance rapid formative feedback (to areas conducting innovations) with the need to produce summative findings (shared locally and nationally, including academic publications).

The RSET team will use innovative approaches to sharing findings, including formative feedback locally during scoping and the full evaluation, and active dissemination of findings more widely so that they can be taken up quickly in other parts of the health and care system.

The Trust will use their extensive communications facilities & networks for dissemination (including professionals in the fields of media relations, public affairs, digital communications and event management), working with the UCL communications team, National Institute for Health Research (NIHR) Dissemination Centre, and Health Services and Delivery Research (HS&DR) programme to maximise the impact of evaluation findings.

The RSET team’s communications/dissemination strategy will:

• Identify the audience: who needs to know?

• Set a timetable for dissemination outputs: weeks, not months

• Decide on appropriate media and form of communications

• Ensure written outputs are as simple as possible

• Provide context and other evidence relevant to each evaluation.

The RSET team will develop a communication strategy for the overall programme and each project in collaboration with NIHR,PPI & end users, including stakeholder dissemination list; social media strategy; bespoke events; conference presentations; open-access journal articles; sharing findings with trade press, such as the Health Service Journal and National Health Executive.

In addition, the RSET team will plan outputs and events to stimulate formative learning over the course of evaluations. The RSET team will reach target audiences through a variety of tailored methods, from graphics and tweetchats on social media, to blogs, vlogs and blogshots, opinion articles, short films and visualisations to traditional media coverage and targeted two-way conversation events through, for example, Nuffield Trust’s extensive learning sets (involving GPs, STP leads, acute trust CEOs etc).

The RSET team have also set up a micro site on the Nuffield Trust website (https://www.nuffieldtrust.org.uk/project/rset-the-rapid-service-evaluation-team) devoted to the whole RSET programme. This will serve as a focus for reaching out to the NHS and other organisations to advertise the RSET programme and its findings.

The Trust will review and evaluate the impact of their communications activity and refine their approach for each new output.

All outputs will be aggregated with small numbers suppressed in line with the HES Analysis Guide.

The Nuffield Trust for Research and Policy Studies in Health Services (The Nuffield Trust) is an independent health research charity overseen by a board of Trustees including a number of senior NHS clinicians, managers and academics. The Nuffield Trust aims to improve the quality of health care that improves the health of people in the UK by providing evidence-based research and policy analysis and informing and generating debate. The Nuffield Trust undertakes work for the public good and within a research governance framework. These are the Nuffield Trust’s legitimate interests which provide a lawful basis for processing personal data under the General Data Protection Regulation (GDPR).

In everything the Nuffield Trust does, they strive to be:

• independent and free from vested interests;

• rigorous, robust and evidence-based in the work they undertake;

• relevant, supportive but also challenging when they need to be;

• open and engaging with all those they come into contact with;

• an organisation that makes a difference to the quality of policy-making and practice in the UK.

Mindful of the quote by the Chief Scientific Adviser for the Department of Health and Social Care that “Research is of no use unless it gets to the people who need to use it”, Nuffield Trust’s strategy to maximise impact is as follows:

• Investigate relevant research questions that are important to patients, commissioners, providers and other stakeholders;

• Co-produce research with patients, the public, professionals and managers to address these questions using rigorous and appropriate research methods applied to the best available data;

• Identify the implications of this research for current and future service delivery for different stakeholders;

• Present the evidence in an appropriate format for the range of target audiences;

• Be guided by input from evidence users, including patients and the public, in all of the above.

Challenges to health and care systems such as increases in life expectancy and multi-morbidities, developments in treatments and technologies, and wider economic pressures have resulted in a drive at national and local levels to develop and accelerate service innovations to benefit patients and the public. Currently, innovations do not spread as fast nor have the same degree of impact as in other sectors, particularly in terms of rapidly shaping frontline service delivery. There is therefore a need for new approaches to evaluation that provide robust evidence to meet these needs in a timely fashion. To address this need, the National Institute for Health Research (NIHR) Health Services and Delivery Research (HS&DR) programme have commissioned a Rapid Evaluation Research Team (RSET).

The Nuffield Trust is part of this collaborative team funded by NIHR over 5 years (2018/9 to 2022/23) to conduct rapid evaluations of health and care service innovations, in close partnership with those who deliver, manage and use these services.

The service innovations may be ‘combinatorial’ and encompass several aspects of innovation such as technological, financial and service model re-design. To be successfully introduced and adopted by the health system, they need to demonstrate added value, cost-effectiveness, benefit to patients over current practice, and feasibility. At the same time, innovations generally require flexible and varied financial structures, and support mechanisms, to ensure they are scaled up and spread beyond local settings, whilst new service delivery models may demand innovative governance and payment arrangements. For these reasons, a flexible approach to applying new evaluation methods is important, in order to accommodate changing innovation processes and pathways. Developing these methods is necessary to address important research questions within this programme but will also be beneficial for future research.

The Nuffield Trust’s partners in the Rapid Service Evaluation Team (RSET) study are researchers from the Department of Applied Health Research (DAHR) at University College London (UCL) - although the data accessed through this agreement will be managed by the Nuffield Trust, and will not be shared with UCL colleagues or any other third-parties (unless aggregated in accordance with standard disclosure rules). The RSET team have a remit to produce timely findings of national relevance and immediate use to decision makers.

The Nuffield Trust’s and UCL's approach to evaluation will combine the questions, ‘What works at what cost?’ with ‘How and why?’ using a mix of innovative quantitative and qualitative methods. The team will enable provision of formative, as well as summative, feedback to people implementing innovations in health and care services. The data sets accessed through this agreement will be crucial to answering questions about the impact of these schemes on health and care. This will happen through high quality, timely research undertaken with a variety of methods. Over the five-year period the RSET team will identify approximately ten innovations to evaluate. NIHR, NHS England, DHSC and the RSET team’s stakeholder advisory board (which includes clinicians, managers, and patient and public representatives) will have advisory input into identifying potential innovations. There will be a focus on rapid evaluation of these schemes with some projects aiming to complete evaluation within 6 to 9 months. Others may have a slightly longer timeframe, but the period from inception to analysis will necessarily be short, to allow for timely formative feedback to the areas undertaking innovations.

All evaluations will conform to the following scope:

- The subject of the evaluation will be a health or care provider (or group of health and/or care providers) which has/have adopted a new way of working which differs to those that have been widely adopted by the health system – i.e. an ‘innovation’. Health or core providers may be NHS Trusts, Local Authorities or, for example, a group of district nurses. The innovation may be use of new technology or a financial and/or service model redesign.

- The HES data would potentially be used to calculate relevant metrics that those implementing the technology would be seeking to affect – for example – short term readmissions to hospital for the targeted groups in those areas. HES data would not necessarily be used in all of the evaluations. It would only be used where it would add value to the evaluation.

- The evaluations will attempt to determine whether there were any detectable changes in the metrics over time that might be associated with the introduction of the innovation.

- This would be done using a variety of techniques such as comparing metrics for the health care provider(s) pre and post introduction of the innovation or comparing the metrics of the health care provider(s) which adopted the innovation with health care provider(s) where the innovation is not in use.

- Standard hospital activity costing methods would be used to calculate the impact in terms of costs or savings to commissioners and hospitals and to model the impact of the innovation if adopted in larger areas (e.g. nationally).

- Alongside this quantitative analysis, local qualitative information gathering and analysis would be carried out to understand how and why the innovations might have had their particular impact in those local health economies.

The RSET team aim to minimise any delays (such as the approval process for data access) as much as possible in order to maximise time spent on each evaluation, to maximise the efficient use of researcher time and to maximise cost effective use of the NIHR grant. Note that the funders NIHR explicitly set up this programme to promote much more rapid analysis of service innovations, that might more quickly benefit health care services nationally.

The Nuffield Trust will be responsible for ensuring that all uses of the data will be within the agreed scope above.

The selection of research projects to be undertaken by the RSET team will be guided using a partnership approach, involving close working with the funder (NIHR), but also with other stakeholders via the RSET team’s Advisory Board, Patient and Public Involvement (PPI) input and other networks. The topics will be selected by initially producing a long list of potential research projects based on recommendations from the funder, the Advisory Board, and the PPI panel. This will be supplemented with a range of activities by the RSET team to identify topics of current interest that it would be timely to investigate. These activities will include desk-based research of published outputs, analysis of social media, interaction with stakeholders at dissemination events, contacting networks, and by people alerting the RSET team to innovations through their interactive website.

A shortlist of potential research projects will be produced using pre-determined criteria to be agreed with the funder, the Advisory Board, and PPI panel. These criteria are likely to include considerations of timeliness, strategic importance, and feasibility. Topics identified by the funder will be prioritised.

Once a project has been identified and reviewed by the funder (to advise the RSET team as to whether the project is within the scope of the research grant), the project will enter a scoping phase. A ‘topic specification form’ is filled in at this point. This includes a description of the motivation for undertaking the research, the research questions, the proposed work and the timescales. This is reviewed by the Nuffield Trust alongside its RSET partners and the funder, and a decision is taken by the RSET team as to whether the project should proceed to a full evaluation.

Where the decision is made to proceed to evaluation, a detailed research protocol is developed by the RSET team (the Nuffield Trust and its UCL partners). This protocol develops on the evaluation research questions and outlines in detail the design and methods proposed to answer the research questions including the team's plans for data processing. The protocol will include references to the evaluation team’s approach to PPI, ethical issues, data and information management, risk management and dissemination of findings. The protocols will be reviewed by independent experts in healthcare organisation and management, and also by project stakeholders, NIHR and patient representatives. The aim of this stage of review will be to advise the RSET team of any shortcomings in the proposed work, or of additional research questions that the RSET might find valuable to address.

In developing the evaluation protocols the Nuffield Trust lead will ensure that any proposed use of HES data conforms to the agreed scope above, with appropriate input from our Data Protection Officer (DPO). The Nuffield Trust project lead will ensure that:

• Projects have clearly defined objectives and analysis plans;

• In each case, the use of the HES data is necessary and proportionate to the purpose of the project – this will include consideration of the necessity for use of each individual HES dataset; the number of years of data; the sizes of any cohorts or control cohorts derived from the data, and the inclusion and exclusion criteria (such as presence of specific diagnostic or procedure codes);

• Appropriate safeguards are in place to protect confidentiality; minimise risks of re-identification and use of excessive data beyond necessity

The individual or individuals within Nuffield Trust who will carry out the project will be bound by an analysis plan detailing what data is permitted for use in the project and how it shall be processed.

The Nuffield Trust and UCL are joint data controllers for the data under this Data Sharing Agreement. As described above, the funder, the Advisory Board and the PPI panel offer advice on what service evaluation projects might be undertaken. Additionally, the funder may have a role in determining that a given project is not within scope of the research grant. However, decisions in respect of use of the data under this Agreement are taken jointly by the Nuffield Trust and UCL without obligation to the funder or any other body. The Nuffield Trust and UCL will have joint responsibility for determining the purposes for processing the data under this Agreement to support the delivery of the projects and for determining the manner in which the data will be processed.

With reference to the Information Commissioner’s published guidance on data controllership, the Nuffield Trust and UCL can confirm that these two organisations are jointly responsible for directing the research project. Specifically they decide:

- to collect the data under this Agreement and the legal basis for doing so;

- which items of data to collect, i.e. the content of the data;

- the purposes the data are to be used for;

- which individuals to collect data about;

- whether to disclose the data – the data will not be disclosed to any third party;

- whether subject access and other individuals’ rights apply; and

- how long to retain the data

There are three projects currently at an early scoping phase. It has not yet been determined whether these projects will need to use HES data and these are provided as examples of innovations which will be investigated and which may potentially use HES data. These are as follows:

• An evaluation of NHSI interventions for trusts in Special Measures for Quality and for those on the ‘challenged providers’ list

• An evaluation of the introduction of the Buurztorg model (of self-managing community care nursing teams) into areas in the UK

• An evaluation of the centralisation of specialised health care services

The nature of this NIHR programme is that further projects are not currently specified in this Agreement but will be identified at relatively short notice over the next several years. This Agreement permits the Nuffield Trust to use subsets of the data for the purposes of the projects specified above and for the purposes of the projects to be identified for the purpose of this NIHR programme which must be compliant with the description of such projects given above.

The Nuffield Trust will send NHS Digital details of each RSET project’s progress as part of an annual update.

Some of the innovations the Nuffield Trust will look at will inevitably need to be analysed with reference to long term historic data and so the Nuffield Trust will require inpatient, outpatient and A&E data for all of England from 2008/09 to 2022/23.

The Nuffield Trust and the wider RSET team are highly aware of the need to balance rapid formative feedback (to areas conducting innovations) with the need to produce summative findings (shared locally and nationally, including academic publications).

The RSET team will use innovative approaches to sharing findings, including formative feedback locally during scoping and the full evaluation, and active dissemination of findings more widely so that they can be taken up quickly in other parts of the health and care system.

The Trust will use their extensive communications facilities & networks for dissemination (including professionals in the fields of media relations, public affairs, digital communications and event management), working with the UCL communications team, National Institute for Health Research (NIHR) Dissemination Centre, and Health Services and Delivery Research (HS&DR) programme to maximise the impact of evaluation findings.

The RSET team’s communications/dissemination strategy will:

• Identify the audience: who needs to know?

• Set a timetable for dissemination outputs: weeks, not months

• Decide on appropriate media and form of communications

• Ensure written outputs are as simple as possible

• Provide context and other evidence relevant to each evaluation.

The RSET team will develop a communication strategy for the overall programme and each project in collaboration with NIHR,PPI & end users, including stakeholder dissemination list; social media strategy; bespoke events; conference presentations; open-access journal articles; sharing findings with trade press, such as the Health Service Journal and National Health Executive.

In addition, the RSET team will plan outputs and events to stimulate formative learning over the course of evaluations. The RSET team will reach target audiences through a variety of tailored methods, from graphics and tweetchats on social media, to blogs, vlogs and blogshots, opinion articles, short films and visualisations to traditional media coverage and targeted two-way conversation events through, for example, Nuffield Trust’s extensive learning sets (involving GPs, STP leads, acute trust CEOs etc).

The RSET team have also set up a micro site on the Nuffield Trust website (https://www.nuffieldtrust.org.uk/project/rset-the-rapid-service-evaluation-team) devoted to the whole RSET programme. This will serve as a focus for reaching out to the NHS and other organisations to advertise the RSET programme and its findings.

The Trust will review and evaluate the impact of their communications activity and refine their approach for each new output.

All outputs will be aggregated with small numbers suppressed in line with the HES Analysis Guide.

Yielded Benefits is not a requirement for new applications.