An Evaluation of Alcohol Treatment Centres: Implications for Service Delivery, Patient Benefit and Harm Reduction

The University of Sheffield are sole data controller for the Alcohol Intoxication Management Services Study (AIMS) and also process the data for this study. No other organisation process the data for the purposes of this agreement. The evaluation of the diversion of alcohol related attendances is an National Institute of Health Research, Health Service and Research Delivery Programme funded research project to estimate the effectiveness, cost-effectiveness, efficiency and acceptability of alcohol intoxication management services (AIMS) in managing alcohol-related ED attendances. NIHR funding is on-going and is currently in place until end of September 2019.

AIMS, also referred to as Alcohol Treatment Centres, Alcohol Recovery Centres, Alcohol Welfare Centres and, in the media, “Drunk Tanks”) are designed to receive, treat and monitor intoxicated patients who would normally attend Emergency Departments (ED) and to lessen the burden that alcohol-misuse, an avoidable healthcare cost, places on unscheduled care. AIMS offer the potential to mitigate some of the pressures on ED at times when it is experiencing a sustained increase in demand. At peak times (e.g. Friday and Saturday nights) most admissions to ED are alcohol-related and they cause the ED clinical environment to suffer, as well as staff morale.

This study, which began in January 2016, is a mixed methods study and this request is for NHS Digital HES ED and inpatient data to evaluate the effectiveness and cost-effectiveness of AIMS. The study follows a natural experiment where six cities have been recruited which have already implemented AIMS throughout England and Wales and will compare them to six control cities where no AIMS is present (ten cities in England, two in Wales). AIMS cities will be matched with control cities for similar demographic characteristics using Home Office iQuanta.

There are three focuses to the evaluation:

i) What is the impact of Alcohol Intoxication Management Services (AIMS) on the work practices and professional identities of frontline staff in managing the intoxicated and other related work activities?

ii) To what extent does AIMS implementation affect key performance indicators in ambulance and health services?

iii) What are the costs of setting up and running an AIMS and what cost savings may be realised elsewhere?

The aim being to provide evidence that informs local and national decision makers on opportunity for a national roll-out across UK cities and provide information about what works through the study of effectiveness, efficiency, processes, barriers and opportunities.

The School of Health and Related Research (ScHARR) at The University of Sheffield are responsible for work stream (aims) 2 and 3. This request is for data to enable the study to answer aims ii) and iii) and the study is collecting data from ambulance services, AIMS service providers (NHS, charities, police services) in order to achieve the study aims. All data being requested is pseudonymised and will not be linked as in line with the HES analysis guide. (Note that on page 14 of the protocol it is stated that the study would explore the linkage of AIMS data to routine NHS data, the conclusion of this exploration is that it is not viable to link and therefore no linkage of datasets will be carried out).

The data requested is for work stream (aims) ii and iii only and will not be used in work stream i. Data is only accessed by staff of University of Sheffield.

In order to assess key performance indicators (aim ii) it is aimed that it will be use in an interrupted time series approach to look at the impact on ED attendances (primary outcome), total time spent in ED, time to treat and alcohol related inpatient attendances. The project will evaluate the effectiveness by comparing attendance rates in control and intervention cites.

The study will also present AIMS activity data in terms of a summary of patient characteristics, and AIMS models (who provides the AIMS service and the type of service e.g. mobile or permanent services). This study will also conduct an economic evaluation to determine the costs required to set up and run an AIMS and estimate the cost savings to other health services. The costs of setting up an AIMS will be requested directly from the service providers (NHS, charities, police services) however, the study also needs pseudonymised data on services and treatments provided in the ED and for hospital admissions, including length of stay, and services provided in order to fully evaluate the cost-effectiveness of AIMS and its impact on the NHS.

The research is carried out in the public interest as results will be used to determine improved care pathways within the Health and Social Care system in England and Wales.

The data controller determines that there are unlikely to be any moral or ethical issues arising from the dissemination of this data.

Data will be stored on a secure drive at the University of Sheffield on a (securely housed) networked virtual machine accessible only from within the campus network. Four people (all substantive employees of the University of Sheffield) will have access to the data, data-manager who will be responsible for cleaning the data, a statistician health economist, a health economist research assistant and a statistician research assistant. No record-level data will be shared outside of these substantive employees.

The data will be analyses to establish the effectiveness of AIMS services and for estimating the cost-effectiveness of AIMS (see specific outcomes for further details). If small numbers arise they will be suppressed in accordance with the NHS Digital HES Analysis Guide and will follow guidelines regarding sensitive conditions. There will be no sharing of record-level data with third parties. No attempt will be made to re-identify anyone from the data. The data will not be linked and will never be used for commercial purposes.

The primary analysis will explore whether the intervention impacts on the number of ED attendances and will use an interrupted time series analysis of this data is looking at trends over time before and after the implementation of this intervention. To conduct this analysis on data from each intervention city it is assumed that each AIMS is open twice a week and will have at least 104 observations over a minimum period of 365 days. Statistical tests including the Dickey-Fuller test, autocorrelation (ACF) and partial auto correlation will be used to establish seasonality, stationarity and differencing which are for establishing the presence of absence of patterns common to time-series data.

Other statistical tests will be carried out to establish the statistical model fit and model goodness of fit. Time series models will also include information on type of incident, age and diagnosis, investigations and treatments in order to examine the effect of AIMS on different patient groups.

Secondary outcomes include hospital admissions, ED key performance indicators (total time in the ED, time to treatment, time to initial assessment, those leaving the ED before being treated and re-attendances within 7 days). Survival analysis methods will be used to analyse time, such as time in the ED, time to treatment and time to assessment. The Kaplan-Meier test will be used to explore differences in total times in the ED, time to treatment and time to initial assessment and an appropriate statistical test will be used to examine differences between groups (for example if times are evenly distributed over time then a log-rank test would be used). If the data is of sufficient quality to examine the impact of time (total, assessment and treatment) on a number of variables then an appropriate semiparametric (Cox proportional hazard model) or parametric survival model will be fitted to the data. An appropriate regression model (ordinary least squares or generalised linear model) will be used to look at length of stay and proportions will be examined using the Chi-squared statistic and logistic regression analysis to allow for differences in case mix.

A comparison will also be made between AIMS cities and paired matched (using Home Office iQuanta) cities that currently don’t have AIMS. Poisson regression models will be used to examine the number of ED attendances between AIMS and control cities to allow for differences in case-mix.

Resource use includes items such as the ED attendance, ambulance journey, inpatient stay and length of stay. The study will account for differences in key resource variables which expect to be in-patient admissions and length of stay as these are known to be expensive, by including them in the time series analysis as variables. The cost of setting up and running an AIMS will be collected from each AIMS site, this will be collated at the aggregate (overall cost) level. The resource use related to the cost of ED services will be obtained from HES and ambulance dispatch data (this information will not be linked but a cost for each service obtained. Unit costs will be obtained from NHS reference costs for HES data. The mean costs of AIMS will be compared with the mean cost of usual care and results will be presented as mean incremental cost per ED admission avoided. The study will also look at the mean incremental cost per ambulance dispatch avoided. The economic analysis will follow guidance on cost-effectiveness analysis set out by the National Institute for Health and Care Excellence (NICE, 2013).

The statistical package STATA will be used for all analysis.

All organisations party to this agreement must comply with the Data Sharing Framework Contract requirements, including those regarding the use (and purposes of that use) by “Personnel” (as defined within the Data Sharing Framework Contract ie: employees, agents and contractors of the Data Recipient who may have access to that data).

The results of this study will be published in an NIHR HS&DR report, which is an open access publication. The final report has been sent to NIHR. The project also aims to publish the results in high quality peer review journals, the main outcomes work is likely to be submitted to a generic journal such as the BMJ or the Journal of Public Health. The health economic results will be submitted to a health economic journal series such as Medical Decision Making. A recommendation report will also be produced that will be made available to trusts, ambulance service, police services and charities that are interested in setting up AIMS services in their area.

Descriptive statistics for the cohort will be presented at summary level, for example mean with standard deviation for continuous variables and numbers with proportions for categorical data. Results will be presented in an aggregate form, for ED and inpatient admissions this would be number of admissions per day and graphical figures of this data will be presented in the same format. The time series analysis of this data is looking at trends over time before and after the implementation of this intervention. Survival analysis methods will be used to analyse time, such as time in the ED, time to treatment and time to assessment. Results and figures will again be presented at an aggregate level. A comparison will also be made between AIMS cities and matched (using Home Office iQuanta) cities that currently don’t have AIMS.

Resource use includes items such as the ED attendance, ambulance journey, inpatient stay and length of stay. The project will account for differences in key resource variables which expect to be in-patient admissions and length of stay as these are known to be expensive, by including them in the time series analysis as variables. The economic analysis will follow guidance on cost-effectiveness analysis set out by the National Institute for Health and Care Excellence (NICE, 2013). Results will be presented as total costs to the NHS and the costs of AIMS with be compared with costs of usual care as costs per ED admission avoided.

All outputs will contain data only in aggregated form (with small numbers suppressed in accordance with the HES Analysis Guide). No commercial requests will be worked on.

Management of the intoxicated in city centres is complex and involves partnerships between health, police and ambulance services. Further, AIMS services are typically commissioned by local governments, police, health care or other agencies in partnership. AIMS are being implemented or decommissioned by NHS Trusts services throughout England and Wales without any evidence on their effectiveness or cost effectiveness. Further different types of AIMS in operation including mobile units verses permanent units and it is important to understand what works and what does not work in order to make recommendations. Further, there is an increased demand on EDs with approximately 70% of attendances being alcohol related at peak times.

This can create bottlenecks in the NHS system which can impact on health care. AIMS have the potential to alleviate this bottleneck by reallocating resources which can thus improve unscheduled care. However, existing AIMS services have not yet been evaluated and it is important to establish whether they are effective and if they are whether all types of services are effective or only specific types. This analysis will evaluate the system and identify what works well and make recommendations about where improvements can be made for local and national decision makers in the NHS. Clearly if AIMS are not cost effective this will highlight a need for action.

The potential benefits of AIMS to people seen in the ED and AIMS units are difficult to measure because of the complex nature of the problem. For the intoxicated the goal of AIMS units are to ensure they are treated in a safe environment and aim I of the project will seek to measure this, which is outside the NHS Digital request. By potentially addressing the bottleneck in EDs other non-intoxicated patients can be seen more quickly and treated sooner and the analysis of time to treatment, time to assessment and total time in ED and looking at the throughput of patients through an ED will answer these questions.

While the primary interest is in the area of health, the study also recognises that learning could promote co-funding and thus decision makers across the three primary partners need to be involved in real-time. In this respect the study aims to develop diffusion and dissemination strategies that both capitalise on stable links with local and national stakeholders and that are also able to encourage engagement in the face of rapid change. As such three diffusion mechanisms will be exploited: (1) A Policy and Impact Group led by project investigators that will be responsible for dissemination; (2) a Study Steering Committee (for preliminary membership see below), that includes a broad range of practitioners and (3) the formation and ongoing development of a Learning Community for practitioners who want to be involved with the project but are unable to commit sufficient resources to become more fully involved.

The Policy and Impact Group have experience of engagement and are directly involved with a number of policy and practice groups and key stakeholders (Department of Health, Primary Care Trusts, and Local Authorities). The group will continue to engage with policy impact groups and ensure the results are disseminated to these groups (e.g. UK Home Office, DoH, Welsh Government, NHS, Association of Chief Police Officers (ACPO), College of Emergency Medicine, Local Alcohol Action Areas (LAAAs)). In addition, the group will write a practitioner oriented report the aim of which would be to provide a summary of findings in such a way that the report can be used for service development across the UK.

One of the remits of the Study Steering Committee (SSC) is to ensure outputs are relevant and timely, and that can provide advice and decision making capacity to the research team, including guidance on dissemination of outputs. The SSC will be initiated in accordance with the NIHR HS&DR guidelines “Research Governance Guidelines: Study Steering Committee (SSC).” The SSC will provide the primary mechanism through which the findings will reach key decision makers and this diffusion mechanism will develop in parallel with Work Stream I. The SSC will identify both diffusion partners through which learnings can be best promulgated (e.g. ACPO, Community Safety Partnerships, Regional Leads for Public Health, the Welsh Government, and the regional commissioning boards for Clinical Commissioning Groups) and seek to target decision makers in Scotland, England, Wales and Northern Ireland.

The Learning Community will operate in parallel with Work Package I and SSC, and will seek to identify local and national parties that would be interested in learnings from the project but are unable to contribute to it. To facilitate engagement an online resource will be developed that makes use of social media such that interested parties are able to keep abreast of project developments with little effort. An online regular update will be published on a quarterly basis, promulgate this through email and twitter and encourage feedback to the Project Team. The recent "Have a Word" Knowledge Transfer Partnership (www.vrg.cf.ac.uk/Files/20140107_KTP_finalreport.pdf) that successfully engaged practitioners through social media, branding and media to encourage engagement in clinical and other staff will be used as a template. Formal and informal diffusion mechanisms will encourage practitioners and decision-makers to both contribute their views on managing the intoxicated and sign up to receive regular updates on project activity. These materials will also be made available to mainstream media and place quarterly updates on-line and encourage more general feedback from the public.

The outputs will be shared in terms of a final report and five peer reviewed open access publications, one on the overall results and a further two related to work packages 2 and 3. Results of the analysis will be disseminated in an easy to digest and accessible online format that seeks to develop the co-production of guidance on best practice and will also be released as part of the study newsletter. In addition to the guidance on best practice further realistic actionable learning outcomes will be defined throughout the project in collaboration with the study management group, study steering committee and with feedback from the learning community.

A patient and public involvement ( PPI) group is actively involved in the project, owing to the complexity of the project there are PPI members from three different areas: a member of The Involving People Network, Wales sits on the project steering group, and members of The Sheffield Emergency Care Forum (SECF) who reviewed earlier drafts of the protocol and The Sheffield Addiction Recovery Research Panel (ShARRP) form a separate PPI advisory group. Whilst the group are primarily involved in work stream (aim) I which includes surveys they will form a crucial part in ensuring the results are disseminated appropriately.

There are currently no peer reviewed published outputs in relation to the data specified within this agreement and therefore no 'benefits'.

The final report was submitted to NIHR on 4th March 2019. It is anticipated that the period of peer review and comments may include further interrogation of the data on which analyses are based. It is therefore imperative to retain and have access to the data as for a period of a further year (until 20.3.20).