Precision in Provision: Predicting Treatment Outcome and Resource Use in Child Mental Health

This Data Sharing Agreement permits University College London (UCL) to access NHS Digital data for the purpose of the Precision in Provision: Predicting Treatment Outcome and Resource Use in Child Mental Health study. The data was supplied under a previous version of this Agreement. No additional data will be provided.

Under GDPR, the lawful basis on which processing of data from NHS Digital concerning the Precision in Provision: Predicting Treatment Outcome and Resource Use in Child Mental Health study is laid out by Articles 6(1)(e) and 9(2)(j) namely that processing is necessary for the performance of a task carried out in the exercise of official authority, such authority to make provision for research being vested in the University by virtue of the Royal Charter establishing the University dated 15 November 1977, and, that in respect of special category personal data, processing is necessary for research purposes carried out in accordance with Article 89(1) of the GDPR. University College London are a public authority and the outcomes of this research is for the benefit of public interest

The University College London (UCL) requires a pseudonymised extract from the MHSDS for use in the Precision in Provision project to conduct analysis into predicting treatment outcome and resource use in child and adolescent mental health in children and young people aged 2 to 25.

UCL instigated this project after recognising that there is a call for precision medicine whereby child mental health interventions are tailored to meet the specific needs of individual young people. The team at UCL applied for and secured funding from MQ: Transforming Mental Health to undertake this work. MQ represents mental health and quality of life, two things MQ believe everyone deserves. The data controller for this project is UCL who will also process the data. MQ is purely the funder.

The individuals who will process the data are substantive employees of UCL or hold honorary contracts with UCL. Honorary contract holders are subject to the terms and conditions of their substantive employer and are subject to the same disciplinary procedure as substantive employees of UCL.

There is a lack of evidence about which characteristics of a young person are associated with treatment outcome and resource use. This research aims to address this gap and thereby expand the use of data resources for mental health research, while at the same time develop the skills base in data linkage. To this aim, UCL will link and analyse data on young people accessing child mental health services from the Child Outcomes Research Consortium (CORC) Research programme and the MHSDS data being requested from NHS Digital. No identifiable data is being used to link the data sets rather linkage is being performed using the probabilistic method (data will be linked using overlapping variables).

In addition to exploring the best methods for data linkage, the two substantive research questions will be:

RQ1. What is the association between case-mix characteristics and effective treatment outcome in young people accessing child mental health services? Here, effective treatment outcome will be assessed using fields relating to a service user’s change in symptoms and functioning between the start and end of treatment (e.g., Strengths and Difficulties Questionnaire, Revised Child Anxiety and Depression Scale).

RQ2. What is the association between case-mix characteristics and efficient resource use in young people accessing child mental health services? Here, efficient resource use will be assessed using fields relating to the number of sessions attended, drop out, and time taken for case closure.

The corresponding expected outcomes are:

O1. Evidence showing that young people with certain case-mix characteristics are more or less likely to achieve an effective treatment outcome;

O2. Evidence showing that young people with certain case-mix characteristics are more or less likely to achieve an efficient resource use.

Case-mix characteristics refer to characteristics of service users including demographics (e.g., age, gender, ethnicity, socio-economic status, special educational needs) and needs-based groupings.

***********

A secondary purpose from this project is to support the feasibility and sustainability of data linkage and future data linkage work that is part of the government’s commitment to better data on child mental health. This is not a separate project and no other parties are involved. UCL will link the pseudonymised record level data from the MHSDS to the pseudonymised record level data from the Child Outcomes Research Consortium Research dataset. Examples of the linkage variables that will be used are gender, organisation identifier, team local identifier, team type and care contact date. The linkage will be carried out using a probabilistic method.

NHS Digital reminds all organisations party to this agreement of the need to comply with the Data Sharing Framework Contract requirements, including those regarding the use (and purposes of that use) by “Personnel” (as defined within the Data Sharing Framework Contract ie: employees, agents and contractors of the Data Recipient who may have access to that data).

For data from the Mental Health (MHSDS, MHLDDS, MHMDS) data sets, the following disclosure control rules must be applied:

• National-level figures only may be presented unrounded, without small number suppression

• Suppress all numbers between 0 and 5

• Round all other numbers to the nearest 5

• Percentages can be calculated based on unrounded values, but need to be rounded to the nearest integer in any outputs

• In addition for Learning Disability data in Mental Health (MHSDS, MHLDDS, MHMDS), the England-level data also must apply the suppression of all numbers between 0 and 5, and rounding of other numbers to the nearest 5.

University College London are the sole Data Controller who also process the data.

Data flow:

1. UCL provides to NHS Digital a list of variables and a list of service providers and organisations submitting to MHSDS who have also submitted to UCL as part of the CORC Research Programme since January 2016.

2. NHS Digital will provide via Secure Electronic File Transfer, a pseudonymised extract of MHSDS (package 1d - community) this will then be uploaded to the ‘Precision in Provision’ partitioned area of the UCL Data Safe Haven. This area is only accessible to named individuals within the project team, all of whom are substantive employees of UCL or hold an honorary contract with UCL. The MHSDS 1d community package provides data to support analysis of local level community activity (administrative data, clinical data and demographics).

3. Also in the ‘Precision in Provision’ partitioned area of the UCL Data Safe Haven will be an extract from the pseudonymised CORC Research Programme UCL dataset.

Once NHS Digital has shared the pseudonymised MHSDS data with UCL and UCL has uploaded it to the UCL Data Safe Haven, the data will then be prepared by standardising and cleaning each data set (pseudonymised NHS Digital data and the anonymised CORC Research programme dataset held by UCL) so they can be best prepared for linkage. Particular time and care will be taken in this step given that good file preparation is one of the most important factors for ensuring effective data linkage. Example activities will include transforming data sets from hierarchical to flat structures as required, converting overlapping variables to the same format (e.g. dates), mapping variables to a common set of codes where different code lists have been used (e.g. ethnicity), and recalculating age variables that are based on different reference dates.

The linkage approach will be driven by the established framework of probabilistic linkage. This step will involve determining the specifics of the approach most suited to linking the UCL data and MHSDS data set. It will involve confirming the overlapping variables to use for linking, picking blocking variables, estimating parameters for assigning matching weights to record pairs, setting thresholds for classifying record pairs as matches, non-matches or possible matches (for manual review), and deciding on rules for multiple iterative passes of linkage.

Record pairs falling between the thresholds will be manually reviewed for classification as matches or non-matches. Based on the findings from the manual reviewing, iterative refinements to the linkage approach may be made. After data linkage, manual checks will be carried out on small random samples of record pairs designated as matches.

All data processing and analysis will be conducted within the partitioned area of the UCL Data Safe Haven. This will include linking the pseudonymised record level data from the MHSDS to the pseudonymised record level UCL data using a probabilistic method. Examples of linkage variables that will be used are gender, organisation identifier, team local identifier, team type, care contact date. The CORC Research programme data set contains pseudonymised information of patient demographics, period of contact, events and questionnaire data.

Linking these datasets is crucial to maximise the strengths and overcome the limitations of each individual dataset; for example, the UCL dataset contains rich outcome information but limited demographic information, whereas the MHSDS contains rich demographic information but limited outcome information. The linked data will be used in analysis investigating association between case-mix characteristics, treatment outcome and resource use.

The data will be minimised by only including variables needed for linkage and analysis and filtering for only services that submit data to the data set held by UCL. The lists of service providers to filter by and variables to be included for linkage and analysis have been provided to NHS Digital.

These steps that have been put in place and the nature of the data mean that the possibility of re-identification is so remote as to render the data as pseudonymised. There will be no requirement/attempt to re-identify individuals

The data that does not link between the two sets of data will be considered for comparator analysis. The data will not be made available to any third parties except in the form of aggregated outputs with small numbers suppressed in line with the Mental Health data sets disclosure control rules.

For all outputs from this project, the following disclosure rules will be applied:

• Suppress all numbers between 0 and 5

• Round all other numbers to the nearest 5

• Percentages may be calculated based on unrounded values, but will be rounded to the nearest integer in any outputs

As of January 2016, the Mental Health and Learning Disabilities Data Set (MHLDDS) standard superseded and replaced the Child and Adolescent Mental Health Services (CAMHS) data set. For this reason, the date period of the data requested will be January 2016 up to the most recent available extract.

This project will produce reports, publications and presentations. These will contain advice to commissioners, practitioners and service users about implications of case-mix for clinical outcomes. The advice will be written commentary on the learning from the analysis, in terms of the understanding of factors correlating with child mental health outcomes relevant to the original research question. All outputs included in reports, summaries and papers will contain only data that is aggregated with small numbers suppressed in line with the Mental Health data sets disclosure control rules.

UCL will work with young people, carers and therapists on the best ways of using the information about the associations between case-mix characteristics and effective treatment outcomes and efficient resource use in clinical discussions with young people and families. These groups will be accessed by UCL existing networks including charities such as the Anna Freud Centre and Young Minds and through learning collaborations of practitioners and commissioners such as the Child Outcomes Research Consortium. These networks are well established and reach over six thousand practitioners and tens of thousands of children and young people. The guidance will be disseminated by email and web presence using existing newsletters and presentation events.

Clinicians will be encouraged to take note of the guidance via their regular clinical supervision. This will be supported through providing training, guidance and helping people to audit their practice. The organisation also works closely with colleagues in NHS England who are developing payment systems for child mental health services who will be able to draw on the learning from this project and can use the analysis outcomes to help determine case-mix groupings for payment purposes. These in turn will then underpin clinician behaviour.

UCL also has close working links with child mental health policy leads in NHS England and Department of Health who will be able to utilise the learning from this project to contribute and influence policy development in this area.

Academic articles on the associations between case-mix characteristics and treatment outcome and resource use have been submitted to European Child & Adolescent Psychiatry and Administration and Policy in Mental Health and Mental Health Services Research. These are leading journals of interest to researchers and professionals in the sector. A blog and video summary will also be produced for the general public. This will be available by early 2022.

A summary of findings will be disseminated through UCL networks, the Child Outcomes Research Consortium’s (CORC) channels and UCL will liaise with MQ for dissemination via their networks. UCL and CORC websites will provide links to open access papers and offer free downloads of accessible summaries of findings. All publications and presentations will be promoted on twitter via the CORC account (over 1500 followers). Preliminary findings were presented at the CORC Members' Forum in November 2020.

The measurable benefits of this work are a more effective and efficient service provision for young people with mental health difficulties and their families. By learning and sharing the learning about the different outcomes achieved by different young people with different sorts of problems and how this links with other factors in their lives, clinicians can target help more effectively and also ensure people get what they need in the most timely way possible. These findings will also help commissioners agree more realistic targets for different groups with service providers allowing more efficient allocation of resources. The direct benefit to young people and their carers and families is that they will be able to have more refined information about what sort of help might help them given their particular circumstances and this should enable them to make more informed decisions about their care. These benefits will be facilitated by providing training, guidance and helping people audit practice in relation to this.

NHS England who are developing payment systems for child mental health services will be able to draw on the learning from this project as UCL works closely with them and they can use the outcomes to help determine the most appropriate case-mix groupings for payment purposes. In addition, the organisation has close working links with child mental health policy leads in NHS England and Department of Health, as well as also being part of the work for both the mental health policy research unit and child policy research unit. Through all of these routes, the findings and learning from this project will influence and contribute to policy development in this area.

By understanding more about the factors associated with different outcomes for different children and young people, service providers and policy makers can target their work more effectively allowing for cost efficiencies. It is expected that these would be realised within two years of project end.

Findings from this project will also work toward the government’s commitment to better data on child mental health, and it will also support the sustainability of data linkage and future data linkage – for example, across education, health and social care. This will be achieved by sharing learning about the probabilistic data linkage approach being trialled here which offers the opportunity to link data sets without inclusion of personal identifiers. As noted above, UCL has extensive links with government departments including Department of Health and NHS England. In addition, UCL are part of two policy research units (child and mental health) which work closely with civil servants to advise on research evidence to inform policy and have a particular commitment to advancing data linkage and use of secondary data to inform policy. The funders for this study, MQ, are committed to using this and other funded projects to learn more about best ways forward for data linkage and to promote this through their national campaigns and future funding initiatives.

The benefits of this research are expected to be realised by autumn 2022.

The first objective of the research project was to build evidence about how to tailor services to meet the individual needs of young people by identifying predictors of amounts child and adolescent mental health service use.

UCL achieved this objective by conducting a secondary analysis of the Mental Health Services Data Set, years 2016-17 and 2017-18. The final sample included 27,979 episodes of care from 71 services with 2-10,855 episodes per service.

UCL found that there were high levels of heterogeneity in the number of care contacts. Certain characteristics predicted differential patterns of service use. In terms of presenting difficulties, young people with psychosis, substance use, or eating disorder were particularly more likely to have higher levels of service use.

The second objective of the research project was to examine the predictors of treatment outcome or improvement in mental health difficulties in young people accessing child and adolescent mental health services. To do so, in the first instance, UCL aimed to examine characteristics that accounted for variation in levels of mental health difficulties at baseline.

UCL achieved this objective by conducting a secondary analysis of the Mental Health Services Data Set, years 2016-17 and 2017-18. UCL conducted multilevel regressions on 8,176 episodes of care from 26 services with complete information on mental health difficulties at baseline and on 3,267 episodes from 25 services also with complete information on mental health difficulties at follow up.

UCL found that young people with higher levels of mental health difficulties at baseline also had higher levels of deterioration in mental health difficulties at follow up. Girls had higher levels of mental health difficulties at baseline, and showed less improvement at follow up, than boys. Compared to young people referred through primary care, young people referred through social care/ youth justice had lower levels of mental health difficulties at baseline and higher levels of improvement in mental health difficulties at follow up. Finally, young people with social anxiety, panic disorder, low mood, or self-harm had higher levels of mental health difficulties at baseline and showed less improvement in mental health difficulties at follow up compared to young people without these presenting problems.

The conclusions from the research project are that the findings of the present research can inform young people, service providers, and policy makers about tailoring amounts of service use to the individual needs of young people. Services seeing higher proportions of young people with higher levels of mental health difficulties at baseline, social anxiety, panic disorder, low mood, or self-harm may be expected to show lower levels of improvement in mental health difficulties at follow up.

The findings from this research have already been used with a region to help inform their Joint Strategic Needs Assessment as part of evidence on service use and treatment outcome to inform how the findings relate to the groups they support and their thinking and practice. Now both sets of analysis are completed, UCL will feed back findings to relevant mental health services through brief summaries shared with CORC network members and in member events.