How general practice team composition and climate relate to quality, effectiveness and human resource costs: a mixed methods study in England.

Background:

The British National Health Service (NHS) is a primary care led system with general practitioners (GPs) being the first point of contact for citizens with non-emergency health care needs. GPs have traditionally worked in practices, led by partners (or a sole partner), employing a team of staff (nurses, care assistants, receptionists, managers) and liaising with other community services. They coordinate care for local people who register with their practice. The sector is currently facing financial and other pressures that threaten the patient experience. Increases in the number of older people, conditions related to social or economic determinations of health , rising expectations of access and quality of medical care of the general public and transfer of some tasks previously undertaken in hospitals to primary care have added significantly to the general practice workload. Simultaneously, recruitment and retention problems have reduced the number of GPs per capita, and shortages of primary and community nurses have exacerbated staffing problems. The number of qualifying doctors choosing general practice has gradually declined over the last decade, whilst increasing numbers of GPs have left practice, with many opting to work abroad.

Concerns about recruitment and retention have coincided with a period of rapid change in the organisation of general practice. Over time, practices have become larger and incorporated a wider range of staff. In September 2016, the BMA reported 7,613 GP practices in England, a decline of 8% since 2006.

Recently, new organisational forms (e.g. ‘super-practices’, federations, and integrated models of primary and community-based care), and different ownership and contractual models (e.g. Alternative Provider Medical Services) of general practice have developed. In this challenging and changing situation, research is required to produce evidence that will enable primary care commissioners and GP practice managers to make resource allocation decisions that will ensure the workforce is effectively and efficiently deployed, and high quality care is maintained. Whilst it is clear that practices are becoming increasingly multidisciplinary, with a wider range of staff involved in direct patient care representing more varied roles, identifying the optimal mix of professionals is complex. Historically workforce planning has been unidisciplinary, but promotion of workforce flexibilities for care delivery relies on a range of disciplines and requires a different approach to workforce planning.

Workforce is the largest single component of healthcare expenditure and the size and composition of the workforce affects performance and outcomes for patients. The ability of health care systems to provide safe, high-quality, effective, and patient-centred services depends on sufficient, well-motivated, and appropriately skilled personnel operating within service delivery models that optimise their performance.

Problems have been highlighted by the Health Foundation regarding national workforce policy in the English NHS concluding that “Workforce is a relatively neglected area of policy which is often pursued as an afterthought, with important clinical, operational and financial impacts on the front line”. However, a number of recent policy proposals (e.g. the NHS Five Year Forward View, and the GP Forward View) have specifically addressed general practice workforce issues. Moreover, developments driven locally by general practices, Clinical Commissioning Groups and community health service providers have led to changes in the practice organisation and structure. In addition, there have been a number of national reviews of the primary care workforce which have had an influence on policy and practice.

Evidence explaining why this research is needed now.

Aside from calling for increased investment and extended use of technology, recent workforce challenges in general practice have been approached in two ways: different ways of working (e.g. skill mix changes, task shifting, role substitution), and organisational changes. As a result, extended use of mid-level practitioners (advanced nurses, paramedics, pharmacists, physiotherapists) and the introduction of new roles (physician’s associates) is becoming more widespread. New collaborative forms of general practice and integrated models involving hospital-based specialists are also emerging (‘super practices’, networks and federations; and polyclinics and multispecialty providers, respectively).

Aim:

The overall aim of this study is to explore how team composition and climate affect quality of care, clinical outcomes (effectiveness) and human resource costs in England, in order to inform practice management and commissioning decisions. The workforce configurations in general practices are highly variable and there is a lack of evidence about what skill mixes and staff deployments generate the best outcomes for patients and savings for health care economies. In addition, evidence on how the micro-level team climate (trust, relationships, processes, etc) relates to quality is not strong.

Study objectives are:

1. Description of policy context; delivery models; practice level variability in skill mix and human resource costs for general practice

2. Exploration of the factors associated with practice performance in terms of quality of care, in particular, the role of skill mix and human resource costs

3. Exploration of impact of role substitution on practices costs and quality of care

4. Conduct patient level modelling of associations between skill mix and clinical effectiveness and implications for costs

5. Examination of how team working affects quality of care and effectiveness through focus groups with service users, a staff survey in a sample of GP practices and practice based case studies

To complete this research, the research team will obtain fully pseudonymised data from the Royal College of General Practice (RCGP) Research Surveillance Centre (RSC). In addition to the pseudonymised primary care medical records, the research team will need to link the RCGP RSC primary care data with secondary Hospital Episode Statistics (HES) Accident and Emergency records for the primary care data cohort.

These data are being requested by the University of Oxford in the performance of a task in the public interest Article 6(1)(e) (EU GDPR, "Lawfulness of processing") i.e. “processing is necessary for the performance of a task carried out in the public interest or in the exercise of official authority vested in the controller”; requested by the National Institute of Health Research to investigate work force and skill mix in GP primary care services, as part of their Health Services and Delivery Research (HS&DR) Programme which aims to produce rigorous, relevant evidence to improve the quality, accessibility and organisation of health services.

The processing required is in accordance with Article 9(2)(j) (EU GDPR, "Processing of special categories of personal data") with regards to the processing being necessary for archiving purposes in the public interest, scientific or historical research purposes or statistical purposes in accordance with Article 89(1) based on Union or Member State law.

Patients who are cared for in general practice surgeries will interact with a wide range of staff, including administrative staff, nurses, healthcare assistants, physiotherapists as well as doctors. How the staff work together has a significant effect on how well a patient is treated and how their medical conditions are managed. Patients who become unwell often attend hospital Accident and Emergency departments and may be admitted for care to hospital as in-patients.

In order to measure how well or poorly general practices are managing the health of their patients, data about how often patients attend hospital accident and emergency departments due to serious medical conditions and how many of a practice’s patients become hospital inpatients. General practice's achievement are measured in the Quality Outcome Framework (QOF) against a scorecard of evidence-based indicators. These indicators span four domains: clinical, organisational, patient experience and additional services. In addition the study will use Emergency hospitalisations for ambulatory care sensitive conditions (ACSC) as the measure of effectiveness, and markers for performance globally as well as in the NHS (Tian Y, Dixon A, Gao H. Emergency hospital admissions for ambulatory care sensitive conditions: identifying the potential for reductions. The Kings Fund, Data Briefing 2012 & World Health Organisation. Assessing health services delivery performance with hospitalizations for ambulatory care sensitive conditions, working document WHO Europe, April 2016).

NHS Digital can provide the Hospital Episode Statistics about Accident and Emergency and Admitted Patient Care (in patient) data, to allow the research team to measure both the clinical and financial impact of the way general practice staff work together to provide patient care.

The study is part of a national program of research funded by the National Institute of Health Research related to Health Services and Delivery Research (HS&DR) Programme and more specifically to the aspect of workforce and skill mix in GP services. The study is part of an ongoing digest of NIHR funded projects (https://www.journalslibrary.nihr.ac.uk/hsdr/#/) aimed to provide evidence to help implement the response to the 2015 Roland Commission’s vision to provide challenging and fulfilling careers for health professionals while delivering a high standard of care.

This study requires secondary (the Hospital Episode Statistics) data and will form the basis of the quantitative assessment of clinical efficacy of primary care and the related cost of clinical outcomes of variation of care, when combined with patients’ medical records from primary care.

500 general practices, comprising the Royal College of General Practice (RCGP) Research Surveillance Centre (RSC) research network will be analysed. This analysis will use a multilevel logistic regression model for the likelihood of hospital admission testing for cross-level interactions between practice characteristics and skill mix profile and emergency admission into secondary care. The registered patients from practices will form the level 1 data, comprising data from secondary care:

• Hospital Episode Statistics Admitted Patient Care

• Hospital Episode Statistics Accident and Emergency

and primary care data from the RCGP RSC database for the year 2019 (1/1/2019 to 31/12/2019).

These data will provide both clinical outcome data and facilitate the calculation of the cost of secondary care.

The data will be pseudonymised and the primary care and secondary care data will be linked and combined with practice level data:

Workforce:

- Total FTE care staff per head of practice population

- Ratio of care staff FTE to total practice FTE

- Ratio of GP FTE to total practice FTE

- Proportion of care staff FTE that are temporary (locum, bank)/ mid level (physician’s associate, advanced nurse)

- Staff turnover, vacancies

Practice Characteristics:

- List size

- Age/ sex distribution of practice population (e.g. % over 75)

- Morbidity (clinical registers

- Region, urban / rural

- Index of multiple deprivations

- Contract type, practice payments

- Type of practice (traditional /new)

Quality indicators:

QOF clinical summary score

CQC inspection rating

GPPS patient experience indicators

When analysed, these data will be used to establish the practice level factors which are associated with patient outcomes / indicators of clinical effectiveness, controlling for patient demographic and comorbidity status, and other practice characteristics which may confound the relationship.

Hospital Episode Statistics about Accident and Emergency and Admitted Patient Care will form the basis of outcome measures of clinical effectiveness : Emergency hospitalisations for ambulatory care sensitive conditions (ACSC) will be the measure of effectiveness used in the analysis. Hospitalisations for ACSCs present a significant burden upon healthcare systems and adjusted rates are used as markers for performance globally as well as in the NHS . ASCS have been described as those conditions where it is possible, to a large extent, to prevent acute exacerbations and reduce the need for hospitalizations through strong primary health care-based services delivery, and are indicators used within the NHS Outcomes Framework (http://content.digital.nhs.uk/nhsof ). Whilst the levels of hospital admissions for select ACSC appear to be decreasing or stabilising over time, there remains wide variation in hospitalisation rates. ACSC are a suitable proxy for primary care clinical effectiveness, and individual conditions will be selected from the Kings Fund categorisation:

- Vaccine preventable: influenza and pneumonia

- Acute (dehydration and gastroenteritis, pyelonephritis, perforated or bleeding ulcer, cellulitis, pelvic inflammatory disease, ENT infections, dental conditions, convulsions/ epilepsy, gangrene)

- Chronic (asthma, congenital heart failure, diabetes complications, COPD, angina, iron deficiency anaemia, hypertension.

To establish the relevant factors determining the current performance of primary care, the most recent stable information has been sought, namely data for 2019.

Regional variation of both healthcare provision and outcomes is well established and to match the geographical spread of the primary care data sources, England wide hospital data will be required in creating a nationally representative dataset.

In order to obtain robust statistical models, hierarchical analyses require patient level will necessary to control for patient characteristics.

Variables have been defined to utilise minimal potential identifiers. Study variables have been specified in formats that will reduce the risk of any inadvertent identification through combinations of variables. Data requested has been limited to that directly relevant to the main outcomes of interest. The study team are not requesting NHS number; this will be pseudonymised using a non-reversible hashing algorithm.

The University of Oxford and the Royal College of General Practitioners are joint data controllers for this study. The University of Oxford is the sole Data Processor. The RCGP RSC has its secure data and analytics hub at University of Oxford, who will manage data governance, encryption and access. The RCGP, has an interest specifically in this study and the use of the data collected due to the nature of the study and therefore has engaged as a joint controller undertaking the relevant activities as a controller along side Oxford. More generally the RCGP does not act as controller of the data used in the research undertaken with the collected data. The RCGP research surveillance centre provides its network general practices with posters which are displayed in the public areas of the practices’ premises (poster provided), patients are informed of the use of their data for both surveillance and research and are provided with a patient leaflet (leaflet provided) and provided with the link to the RCGP RSC transparency statement https://clininf.eu/index.php/transparency-statement/, they are also informed of their right to opt out of their data being used for research and planning (https://digital.nhs.uk/services/national-data-opt-out).

The University of Oxford has a contract with the RCGP to provide this surveillance, quality improvement and research platform. The University of Oxford is identified as a processor of personal data for the Royal College of General Practitioners (RCGP).

Wellbeing (formally Apollo Medical Software Solutions), an approved third-party provider, has formal service agreements and service specifications with RCGP RSC and with individual participating GP practices to conduct data collection and secure web transfer. Each unique patient within the RCGP RSC database is pseudonymised at source before data is extracted from individual practices using a computer generated patient ID created by Wellbeing Software Solutions. This pseudonymisation of records includes production of a hashed NHS number using pseudonymisation algorithm (SHA-512).

The service RCGP use is their secure extraction service:

Wellbeing SQL is a data extraction software that will extract any and all data from the GP practice in a consented, appropriate manner. It allows practices to share information in an anonymised format to third party organisations, who may amalgamate data from multiple practices and apply business intelligence or risk stratification to assist the individual practice, local CCG or PCT. There are regular letters that go out to all practices, a monthly newsletter, and a weekly email.

The National Institute of Health Research have funded this research as part of the Health Services and Delivery Research (HS&DR) Programme (NIHR Funding award ID 17/08/34).

for t

All organisations party to this agreement must comply with the Data Sharing Framework Contract requirements, including those regarding the use (and purposes of that use) by "Personnel" (as defined within the Data Sharing Framework Contract i.e.: employees, agents and contractors of the Data Recipient who may have access to that data).

The RCGP RSC is based at the University of Oxford. The University of Oxford has a contract with the RCGP to provide this surveillance, quality improvement and research platform. The University of Oxford is identified as a processor of personal data for the Royal College of General Practitioners (RCGP).

The hashing of identifiable data for the Clinical Informatics and outcomes Research Group at the University of Oxford is conducted by the Salt Service of the University of Oxford Central IT team, so that the holder of the pseudonymised data is separated from the service that holds the non-reversible hash key. This avoids pseudonymised data becoming identifiable data.

The SALT methodology rationale will be as follows

1) Wellbeing Software Solutions, an approved third-party provider, has formal service agreements and service specifications with RCGP Research Surveillance Centre and with individual participating GP practices to conduct data collection and secure web transfer.

2) Each unique patient within the RCGP RSC databank is pseudonymised at source before data is extracted from individual practices using a computer-generated patient identifier created by Wellbeing Software Solutions.

3) This pseudonymisation of records includes production of a hashed NHS number using pseudonymisation algorithm (SHA-512).

4) An encryption salt is held by a designated staff member of the University of Oxford Medical Science Division who is not a member of the ORCHID staff.

5) When a data linkage is required to the data extracted by Wellbeing Software Solutions in the RGCP RSC databank, the encryption salt holder sends the encryption salt to the data provider (in this case it is NHS Digital)

6) NHS Digital will hash personal identifiers (in the data requested by ORCHID) using a hashing algorithm. NHS Digital will use the same pseudonymisation algorithm (SHA-512)

7) To make this key unique, an encryption salt is added at the end of the NHS number (e.g. NHS number= 12345678 ; SALT (held by someone other than ORCHID staff) = bob. So, hashing would take place using the SHA2-512 algorithm by 12345678bob = return pseudonymised data.

The encryption salt is one-way.

8) The member of staff who holds the encryption SALT is not a member of the research team working on the data provided by the RGCP or NHS Digital.

9) Therefore the researchers do not have the means available to ‘un-hash’ the data provided.

NHS Digital will hash their NHS numbers using the same pseudonymisation algorithm (SHA-512). NHS Digital will undertake data linkage via the hashed NHS numbers in both sets of data. This process has been used for previous projects linking different sets of data, and the linkage has been successful. Records for each study participant containing information from HES, together with hashed NHS numbers will be sent to the University of Oxford.

There will be no subsequent flows of data from the University of Oxford.

In this agreement the University of Oxford will act as a joint data controller with the RCGP. University of Oxford will process the data.

University of Oxford have requested 41 fields of the dataset HES APC, including the codes present in

General practices within the RCGP RSC network have been involved in disease surveillance for over 50 years. Over this period practices have had feedback about their data quality and many practices have been computerised since the late 1990s, allowing long-term outcomes to be studied.

Wellbeing Software Solutions has formal service agreements and service specifications with RCGP RSC and with individual participating GP practices to conduct data collection and secure web transfer.

Each unique patient within the RCGP RSC database is pseudonymised at source before data is extracted from individual practices using a computer generated patient ID created by Wellbeing Software Solutions. This pseudonymistion of records includes production of a hashed NHS number using pseudonymisation algorithm (SHA-512).

Pseudonymised record-level HES data will be processed and stored at the University of Oxford. Patient level databases are held in the database server within the Research Group's secure network. The Research Group's dedicated secure network is sited behind a firewall within the University's network. It is a standalone, independent network, all in-bounded connections are block, but out-bounded connections are allowed. All staff members of the research group working within the team base work from secure workstations or secure laptops with encrypted drive. Only substantive employees of the University of Oxford will have access to the data and only for the purposes described in this document. The data will be used solely for the study titled "How general practice team composition and climate relate to quality, effectiveness and human resource costs: a mixed methods study in England".

The University of Oxford will send the hashed NHS numbers to NHS Digital. The following flow of hashed NHS numbers will be undertaken.

University of Oxford will identify the study patients for the cohort above from primary care records in the RCGP RSC practices and send the hashed NHS numbers of the cohort under study to NHS Digital to link to HES/ Civil registration data. No other GP data will be sent to NHS Digital.

The process of linkage is as follows:

• NHS Digital will hash their NHS numbers using the same pseudonymisation algorithm (SHA-512) as used by the RCGP.

• NHS Digital will undertake data linkage via the hashed NHS numbers in both sets of data. This process has been used for previous projects linking different sets of data, and the linkage has been successful

• NHS digital extract all HES records for which there are matched primary care records

• NHS digital will send the extract of HES records with the hashed NHS number to the University of Oxford

• University of Oxford will link the HES records together with GP data from the primary care records from RCGP RSC practices with the same hashed NHS numbers

Records for each study participant will when fully linked contain information from HES together with information from RCGP RSC primary care practices.

Each unique patient within the RCGP RSC database is pseudonymised at source before data is extracted from individual practices using a computer-generated patient ID. The University of Oxford holds no identifiable data and only hashed NHS number. Combining/ linking data from University of Oxford for this project will not lead to or increase the risk of pseudonymised data becoming identifiable data. Linkage of two non-confidential datasets does not create a confidential dataset.

Only pseudonymised data with direct patient identifiers removed will be used. The research team will not seek individual patient identifiers; where required, data linkage will be achieved through ‘hashing’ algorithms to generate non-identifiable, unique IDs from identifiable data; as a further protection, non-reversible, pseudonymised ID numbers held be database organisations will be converted to unique study IDs, the keys to which will not be accessible to the research team; and, when using these data small numbers in reporting will be suppressed and the presentation of data that can potentially be used to reveal identities will be avoided. Data extracts and aggregate analyses will be pseudonymised/anonymised as described.

All data processing is carried out by staff with contracts with the University of Oxford. All staff have received Information Governance training on an annual basis and have all passed the NHS Information Governance on-line test for the current year.

Access to the data will limited to researchers with substantive employee contracts with the University of Oxford. All researchers will be required to complete training and sign the relevant agreements to be able to access the data on the University of Oxford secure environment. No individual-level study data can leave this environment, and all aggregated results data is reviewed prior to export.

All data transferred from the secure, “safe haven” computing environment, undergoes a statistical control process, where the aggregated data are assessed in order that no patients can be identified by inference, such as reporting rare diseases or operations conducted at a specific time or location. Practice level identification is also avoided by ensuring the granularity of the analyses reported are at the highest level possible while providing meaningful scientific insights.

The Research Group has conducted a risk assessment of the physical security of the offices and servers where patient level data is kept. The Research Group of Department of Clinical and Experimental Medicine at the University of Oxford has worked with routinely collected healthcare data in a number of research and evaluation projects over the last 15 years. The Research Group works within the Research and Information Governance team at the University of Oxford.

No data is stored outside of the secure computer system hosted at the University of Oxford.

All outputs will be scrutinized by a lead senior academic with the Service User Panel and the Study Steering Group before the output is disseminated. The study has a set up a working group, the dissemination of findings team specifically for this purpose.

The study follows a mixed methods design. The findings from the quantitative analysis of the data from NHS Digital data - the hospital data (accident and emergency, in-patient data) and civil registration deaths, will be combined with the practice level database and patient level data from primary care in the RCGP RSC database and an economic regression qualitative using a synthesised using a convergent parallel mixed methods design (Cresswell JW, Plano Clark VL. Designing and Conducting Mixed Methods Research. Thousand Oaks, CA: Sage Publications, Inc., 2011).

To ensure that the outputs inform practice and thereby maximise benefit to patients and the NHS, the dissemination strategy will use a knowledge management framework (de Lusignan S, Pritchard K, Chan T. A knowledge-management model for clinical practice. Journal of Postgraduate Medicine 2002; 48(4): 297-303), creating information at macro (health system), meso (health region/ locality) and micro (individual provider/ practice) levels.

The knowledge translation literature indicates that new information is most effectively disseminated using multiple approaches and ideally face-to-face. In addition to maintaining a project website and giving written and online feedback to study participants, activities will include:

Reports – a study report (planned delivery month 33) will be delivered to the funder, the NIHR. This report will detail all the methods, results and conclusions, including patient and public involvement.

Patient and public involvement has been formalised by the establishment of a Service User Panel (SUP), a form of public patient involvement (PI) advisory group. It will comprise 10 members recruited from different types of practices (traditional and new models, in varied socio-economic-ethnic areas in Kent and Surrey) and will meet four times per year to provide the perspective of patients and the public on issues within the research. The advisory group will be asked to assist with preparing information sheets for participants, focus group topics, patient survey questions, statements for the implementation guideline development process and dissemination materials for lay audiences. The SUP will receive training for their role and full information about the project at the first meeting and will be involved in the knowledge transfer process. Members will be reimbursed for their attendance at meetings, and contributing to research activity, for reasonable travel expenses and time commitments at National Standards for Public Involvement rates.

The NIHR study report the format will conform to the guidance given by the NIHR.

Submissions to peer reviewed journals – research papers will be submitted to lead journals relating to health service and delivery research, health economics and primary care medicine. These research papers will be written and submitted within the year following the end of the study.

Presentations - ten interactive workshops across England on implementation of good practice recommendations developed. These workshops will involve Commissioners and NHS managers- GPs, GP consortia, and other primary care providers- Dept. of Health, NHS Digital, National Institute for Care Excellence (NICE), Care Quality Commission, Health Education England - Royal College General Practitioners (RCGP), Royal College Nursing (RCN), British Medical Association and its Local Medical Committees; other groups dependent on skill mix e.g. Faculty of Physicians Associates (FPA), Royal College of Physicians

Conferences

Patient/public guide (developed with input from the SUP – to help patients and public appraise the pros and cons of skill mix in primary care; targeted at practices PPI group members; lay members of CCGs/STPs; national patient groups/charities

Press releases and policy briefings disseminated through links with key organisations

Social media (LinkedIn® & Twitter®) with associated infographics at key milestones

Massive open online course (MOOC) Webinar, video (YouTube®), multimedia evidence summaries

All data reported in the study outputs will be at the aggregate level with small number suppressed in line with HES analysis guide for quantitative analyses.

Implementation recommendations:

Qualitative research, comprising findings from all aspects of the work will be brought together in a consensus forming process involving GPs, professionals, commissioners and service users in order to produce implementation recommendations that are relevant and workable throughout the NHS .

The consensus forming process will employ the Nominal Group Technique (McMillan, S. S., King, M., & Tully, M. P. 2016. How to use the nominal group and Delphi techniques. International journal of clinical pharmacy, 38(3), 655–662. https://doi.org/10.1007/s11096-016-0257-x) to synthesise findings and elicit consensus among experts on implementation recommendations. The method facilitates the generation of ideas in relation to problems, solutions, or both, and is based on the premise that accurate and reliable assessment is best achieved by consulting a panel of experts and accepting group consensus. Development sessions attended by the members of the research team, Service User Panel and Professionals and Commissioners Panel will establish key learning from the research and identify Knowledge Transfer Topics. Consensus-building workshops with commissioners, healthcare professionals in general practice and service-user representatives (experts) will to consider the Knowledge Transfer Topics and will develop recommendations and ‘priority action points’ that will support practice management and commissioning decisions related to the GP workforce composition and team functioning.

Stakeholders will be recruited to the to the Nominal Group Technique based workshops, so as to ensure that there is good geographical coverage and that different types of practices and a variety of socio-economic and ethnic areas are represented.

The recruitment use two processes

1) an open invitation to commissioners, GPs, other professionals in General Practice and service users) will be publicised via the project website, social media and targeted communications.

2) partnerships will be formed with national networks, such as the Clinical Research Networks, the Primary Care Collaboratives, RCGP and other influential groups, to support recruitment.

These two approaches are expected to be supplemented by the snowballing technique and will thus increase participation rates, with members of the team and individuals who have participated in the research during the two-year period circulating invitations to their contacts. A nominal financial recruitment incentive will be offered to each stakeholder taking part in the workshop, and participant travel expenses will be reimbursed.

a) Patient and carer focus groups

b) Survey of team members in a representative sample of general practices

c) Case studies in 12 general practices

The recommendations from the Nominal Group Technique will inform short term staffing decisions and longer term training plans at practice, regional and national levels.

They will be disseminated through multiple means including interactive workshops, policy briefings and presentations to the relevant audiences.

The workshops lead by a senior primary care researcher will initiate discussion on how to implement good practice recommendations with Commissioners and NHS managers (e.g. Clinical Commissioning Groups, Sustainability and Transformation Plan areas, NHS England), GPs, GP consortia, and other primary care providers, external statutory organisations (e.g. Dept. of Health, NHS Digital, National Institute for Care Excellence (NICE), Care Quality Commission, Health Education England), external non-statutory bodies: Royal College General Practitioners (RCGP), Royal College Nursing (RCN), British Medical Association and its Local Medical Committees; other groups dependent on skill mix e.g. Faculty of Physicians Associates (FPA), Royal College of Physicians.

Policy briefings and press releases will be disseminated to Commissioners and NHS managers (e.g. Clinical Commissioning Groups, Sustainability and Transformation Plan areas, NHS England), through links with key organizations: external statutory organisations (e.g. Dept. of Health, NHS Digital, National Institute for Care Excellence (NICE), Care Quality Commission, Health Education England), external non-statutory bodies: Royal College General Practitioners (RCGP), Royal College Nursing (RCN), British Medical Association and its Local Medical Committees; other groups dependent on skill mix e.g. Faculty of Physicians Associates (FPA), Royal College of Physicians, academia, especially primary care academia through RCGP, conferences and Society of Academic Primary Care (SAPC). The briefing documents will be high level summaries of the key findings of the study, written in a less technical style.

Publications are planned; to report the clinical efficiency and economic evaluation findings, in conjunction with the qualitative research findings - within a year of the end of the study in journals such as the Health Services and Delivery Research (ISSN: 2050-4357).

Statistical mixed effects models of clinical effectiveness and estimation of care costs (and savings) at the practice level and more widely at higher levels of the NHS will be published in the appropriate journals to share findings with the various audiences identified by the dissemination team.

Due to information governance restriction patient and staff level data will not be shared. However all findings will be shared, and publishing in journals offering open access will be sought, aggregated with small numbers suppressed in line with HES analysis guide.

All reporting and wider dissemination activities are scheduled to be completed by the end of September 2021.

Outputs will be produced that meet the needs of six key audiences:

• Commissioners and NHS managers (e.g. Clinical Commissioning Groups, Sustainability and Transformation Plan areas, NHS England). They will be involved in ten interactive workshops based on implementation of good practice recommendations. Press releases and policy briefing will be targeted at this group along with peer reviewed journal articles.

The benefits of the evidence the study will produce are closely aligned with the aims of the funder, NIHR HS&DR to produce evidence on the quality, accessibility and organisation of health services and how the NHS might improve delivery of service. Use of the NHS Digital data will enable the development of statistical models exploring the key determinates of health outcomes of primary care and associated costs, allowing for the inclusion of both individual patient characteristics and the characteristics of their specific GP surgery’s health care teams characteristics responsible for delivery their care.

• GPs, GP consortia, and other primary care providers. They will be involved in ten interactive workshops based on implementation of good practice recommendations and will also benefit from social media, webinar and peer reviewed journal articles. The study findings and their implications will be presented in the workshops and it is hoped that the interaction between the researchers and the GP community will mean that findings and implications of the research can be explored in a supportive and collaborative environment.

• Patients and the public. A Patient/public guide (developed with input from the patient advisory group) will be provided– to help patients and public appraise the pros and cons of skill mix in primary care. Specifically practices PPI group members; lay members of CCGs/STPs; national patient groups/charities will be made aware of this resource. Additionally, social media, webinar and peer reviewed journal articles will be available to further inform.

The benefits of public involvement is an intrinsic part of citizenship, public accountability and transparency and can lead to empowering people who use health and social care services, providing a route to influencing change and improvement in issues which concern people most.

• External statutory organisations (e.g. Dept. of Health, NHS Digital, National Institute for Care Excellence (NICE), Care Quality Commission, Health Education England). These bodies will be invited to attend ten interactive workshops based on implementation of good practice recommendations. Press releases and policy briefing will be targeted at this group and they will also benefit from social media, webinar and peer reviewed journal articles.

This audience, while varied in its make-up, have broad remits in terms of planning and providing health services and also the generation of process and outcome data and its synthesis. By presenting new models of the interaction of GP team composition and climate and outcomes of quality and effectiveness of care, costs, the aim is to refine the co-ordination of the work of these organisations.

• External non-statutory bodies: Royal College General Practitioners (RCGP), Royal College Nursing (RCN), British Medical Association and its Local Medical Committees; other groups dependent on skill mix e.g. Faculty of Physicians Associates (FPA), Royal College of Physicians. These bodies will be invited to attend ten interactive workshops based on implementation of good practice recommendations. Press releases and policy briefing will be targeted at this group and they will also benefit from social media, webinar and peer reviewed journal articles. It is essential that the bodies representing the GP workforce are appraised of the findings of the study, if policy change is to be effective. Engaging with the representatives of the workers delivering change and explaining the study findings and their implications, can only help to initiate change within the body of healthcare professionals required, in response to the new insights the study findings will provide. It is expected that findings will include the degree of variation within primary care, of modes of multi-disciplinary working and it is thus essential the professional bodies can contribute to any strategic planning at the earliest opportunity while be appraised of the complexity of the totality of the system. These workshops will ensure the new findings can be explained in an interactive fashion ensuring details can be explored in a supportive and collaborative environment.

• Academia, especially primary care academia through RCGP, conferences and Society of Academic Primary Care (SAPC) . Academic researchers and societies will be key targets for peer reviewed journal articles generated by the study team and they will also have available the widely disseminated press releases and policy briefings, social media and webinar. The dissemination of research findings and the subsequent discourse is a well established aspect of modern science and will be key to establishing the validity of the findings, promoted the evidence and to initiate further work in the field.

• Ten interactive workshops across England on implementation of good practice recommendations developed as part of the study. The audiences will comprise: commissioners and NHS managers (e.g. Clinical Commissioning Groups, Sustainability and Transformation Plan areas, NHS England), GPs, GP consortia, and other primary care providers , external statutory organisations (e.g. Dept. of Health, NHS Digital, National Institute for Care Excellence (NICE), Care Quality Commission, Health Education England), and external non-statutory bodies (e.g. Royal College General Practitioners, Royal College Nursing, British Medical Association and its local medical committees; other groups dependent on skill mix e.g. Faculty of Physicians Associates, Royal College of Physicians)

• Patient/public guide (developed with input from the SUP)– to help patients and public appraise the pros and cons of skill mix in primary care; targeted at practices PPI group members; lay members of CCGs/STPs; national patient groups/charities (skill mix to deliver quality)

• Press releases and policy briefings disseminated through links with key organisations : commissioners and NHS managers, external statutory organisations, external non-statutory bodies, academia

• Social media (Linked in & Twitter) with associated infographics at key milestones (All)

• Massive open online course (MOOC) webinar, video (Youtube), multimedia evidence summaries

• Publications, including full NIHR report, articles for professional and academic journals, conference presentations

The findings will produce evidence about what skill mix configurations work best in primary care, and what opportunities exist for substitution of tasks between different health practitioners in order to reduce costs whilst maintaining or improving outcomes. The study will generate quantitative economic models, indicating key characteristics which drive costs and quality. These models will be corroborated by the detailed qualitative parts of the overall, complex, concurrent, parallel, multistage, mixed methods design, with embedded survey and intensive case studies, including patient surveys. The detailed technical economic models will then be interpreted by a working group comprising recommendations and ‘priority action points’. The targeted recommendations and ‘priority action points’ will be more readily interpretable by GP partners, managers and commissioners and will enable them to make staffing decisions based on comprehensive evidence based policy that will ensure that the limited available human resources can be deployed in a way that maximises patient benefit. Identifying efficient workforce configurations will enable more patients to be treated effectively at the same or lower costs. This will benefit the population who are service users, through improved access to more timely care, and tax payers (funders of the NHS).The findings will produce evidence about what skill mix configurations work best in primary care, and what opportunities exist for substitution of tasks between different health practitioners in order to reduce costs whilst maintaining or improving outcomes. The study will generate quantitative economic models, indicating key characteristics which drive costs and quality. These models will be corroborated by the detailed qualitative parts of the overall, complex, concurrent, parallel, multistage, mixed methods design, with embedded survey and intensive case studies, including patient surveys. The detailed technical economic models will then be interpreted by a working group comprising recommendations and ‘priority action points’. The targeted recommendations and ‘priority action points’ will be more readily interpretable by GP partners, managers and commissioners and will enable them to make staffing decisions based on comprehensive evidence based policy that will ensure that the limited available human resources can be deployed in a way that maximises patient benefit. Identifying efficient workforce configurations will enable more patients to be treated effectively at the same efficiently allocated. Overall this will contribute to the smooth running of the NHS in the future, and its sustainability.

The research will also provide information on the relative efficiency of new models of primary care, and whether new staff roles, and new ways of using existing staff, are associated with improvements in patient outcomes or savings in costs. Findings will also indicate how team working and relationships relate to patient outcomes and experiences and staff wellbeing and job satisfaction, providing further guidance about how to foster productive team working environments.

The identification of the changes in how health care is delivered in primary care in terms of roles, will allow for planning of the recruitment of the health care workforce across all specialties and with sufficient staff to provide sufficient training to do the work that is needed in primary care.

In line with the overall stated goals of the HS&DR Programme, this research will produce evidence on the workforce factors associated with quality care provided by general practice. This evidence will form the basis of evolving models of care, informing policy as to how the NHS might improve delivery of services , by developing statistical models that identify key GP workforce characteristics associated with both clinical effectiveness and cost of care. These models will be evaluated at various stages of the study to validate the models’ interpretations in the context of the qualitative evidence, building on the strengths of the mixed methods approach for investigating complex processes in health care. Practice workforce characteristic will be complied from the freely available NHS Digital Primary Care Workforce Minimum Dataset and supplemented by data from the Quality Outcome Framework process and data from the both the Royal College General Practitioners Research Surveillance Centre database and Workforce costs for practices will be examined in relation to the total payments received (NHS Digital Annual Payments Review). The cost of care data required for economic analysis will be estimated using a top down approach with national unit costs (Curtis L, Burns A. Unit Costs of Health and Social Care 2016, Personal Social Services Research Unit, University of Kent, Canterbury. 2016) applied to the direct FTE cost of each staff role by practice. Workforce costs for practices will be examined in relation to the total payments received (NHS Digital Annual Payments Review).

The statistical models based on the primary and secondary (Hospital Episode Statistics Accident and Emergency) care data, will identify the key characteristics of general practices organization and workforce that drive quality of care, facilitating the development of specific policies and planning for the NHS.

There is a detailed dissemination strategy to delivery throughout the project, led by a member of the research team, who is both a senior academic with expertise in Primary Care epidemiology and an active clinician). The plan will ensure results are shared and have impact. Input will be provided by part of the research team engaging directly with GP primary care practitioners, commissioners and service users to develop implementation recommendations, the Service User Panel and Study Steering Group.

To ensure that the outputs inform practice and thereby maximise benefit to patients and the NHS, the dissemination strategy will use a knowledge management framework, creating information at macro (health system), meso (health region/ locality) and micro (individual provider/ practice) levels.

The study outputs will have direct benefit to commissioners and strategic policy makers. Commissioning, policy development and implementation are complex processes. The outputs of this study will have the potential to contribute to these processes, ultimately improving the workforce effectiveness at the practice level through staffing policy and training.

Both economic analysis and investigation of the factors contributing to clinical effectiveness are major aspects of the proposed study. Findings will be disseminated to all stakeholders, both service users, clinicians and administrators at the 10 workshops and policy briefings and conferences. As one of the specified target audiences includes NHS England, there is potential to inform future policy at this level, in line with the NIHR’s remit. Such changes in policy regarding the future conformation of the GP practice workforce has the potential to improvement care quality outcomes and reduce cost of treatment throughout primary care , at the national level .

The benefits will be accrued at several levels of the primary healthcare system; the funder, the NIHR will gain insight into how workforce dynamics in general practice relate to clinical outcomes and from these insights commissioners will have statistical models which can form the basis of a workforce planning toolkit, to be utilized in the nationwide planning of primary healthcare services, to the advantage of the nation as a whole.

The study team will measure clinical effectiveness using adjusted hospitalization rates for ambulatory care sensitive conditions, which are well established markers for performance within the NHS. While the study will deliver its findings within 36 months and the project does not provide for implementation evaluation, the use of routinely used metrics will facilitate the on-going monitoring of clinical performance as defined within the research framework.

The epidemiological analysis of the primary care and linked HES data is expected to be completed within 27 months. The analysis will provide insight into the factors associated with clinical effectiveness However as the study employs a mixed methods approach, the synthesis of the quantitative epidemiological analysis with the qualitative patient and healthcare professional findings will be finalised and disseminated from month 34 to 36.

Specifically, the findings will produce evidence about what skill mix configurations work best in primary care, and what opportunities exist for substitution of tasks between different health practitioners in order to reduce costs whilst maintaining or improving outcomes. This will enable GP partners, managers and commissioners to make staffing decisions that will ensure that the limited available human resources can be deployed in a way that maximises patient benefit. Identifying efficient workforce configurations will enable more patients to be treated effectively at the same or lower costs. This will benefit the population who are service users, through improved access to more timely care, and tax payers (funders of the NHS), because the NHS budget will be more efficiently allocated. Overall this will contribute to the smooth running of the NHS in the future, and its sustainability.

The research will also provide information on the relative efficiency of new models of primary care, and whether new staff roles, and new ways of using existing staff, are associated with improvements in patient outcomes or savings in costs. Findings will also indicate how team working and relationships relate to patient outcomes and experiences and staff wellbeing and job satisfaction, providing further guidance about how to foster productive team working environments.

Yielded Benefits is not a requirement for new applications.