Establishing trends and understanding inequalities and context in people’s mental health, 1993-2014

The National Centre for Social Research (NatCen) requires APMS survey data for use in analyses investigating trends and inequalities in mental health in England.

NatCen is a research institute specialising in health and social survey data analysis. NatCen applied for and secured funding via the Public Health Research Consortium to undertake this work. While that initial funding is no longer current, NatCen wishes to continue with analyses that fit within the remit of this application. As a general population sample, the survey population includes both male and female adults aged 16 and over (no upper age limit) living in private households in England, UK. The 2014 survey is the most recent available in the series.

The primary objective for processing APMS data is to use the best available information on general population patterning in mental disorder and self-harm behaviours, to examine trends and identify risk factors for mental illness in women (and men). This is to ensure government have an accurate understanding about population trends in need, and can prioritise and plan services accordingly.

The National Centre for Social Research require the retention and processing of the 2014 APMS survey in order to continue to undertake further analysis while addressing the same purpose. The APMS data remains the most recent available data of its type. The dataset is designed to be able to address many different questions. The National Centre for Social Research continue to address the core aims:

1) Establish the trends: What are the trends across a range of different mental disorders and related behaviours in the population; how does this break down between women and men, by age group, by other characteristics? Are the patterns emerging in different datasets and sources consistent?

2) Identify the drivers: There is a need to understand what is underpinning current trends. Do men and women have different levels of exposure to risk factors – such as violence, abuse, environmental stress, poverty and adversity – or do they respond differently to such adversities?

3) Establish trends in potential drivers: Known social and economic factors shaping mental health include violence and abuse, experience of childhood adversity, debt, housing conditions, psychosocial working environment, flooding and natural disaster, caring responsibilities, and other sources of stress and support. New contextual factors include social media and technologies. What are the trends in each of these factors?

4) Profile multiple adversities: Examine how different aspects of mental health and life circumstances cluster together in the population.

5) Work with the data in order to inform the development and reporting of the next survey in the series, APMS 2022.

These research questions are broad and continue to be policy priorities. The context as to why this objective is necessary to address now is outlined below:

Women are more likely to have a common mental disorder (CMD) than men (Castle and Abel, 2015). Women’s higher rates of depression and anxiety disorders have been evident consistently over time, across countries, and in different social and economic groups (Freeman and Freeman, 2013). In recent years, evidence has emerged suggesting that the gap in mental illness between young women and men may be growing (Campbell-Jack D et al 2016; McManus et al 2016). The gap in levels of low wellbeing also appears to be increasing between girls and boys (Children’s Society 2016; Lessof 2016).

APMS found a steep rise in self-reported self-harm and identified young women as a high-risk group for several mental disorders. And while externalising responses to stress remained higher in men, this gap may be shrinking. For example, while hazardous use of alcohol declined in young men, it was stable in young women (McManus et al 2016). Recent suicide statistics showed an increase among women in the past year, and a decline in men (ONS 2016).

Hospital Episode Statistics support there being a greater rise in self-harm related hospital admissions in girls and young women (NHS Digital 2016). The number of girls treated as inpatients after cutting themselves almost quadrupled between 2005 and 2015.

Consistent with this, increases in antidepressant prescribing have been greatest in girls and young women (Sarginson et al 2017). It is likely that a complex mix of factors explain the differences in mental health between men and women, including physiological and biological factors (Castle and Abel 2015). However, if differences in rates are changing over time, it likely that changes in social and economic context must also be key.

In 2015, the Chief Medical Officer published a report on women’s health (Davies 2015). This was produced before the latest APMS data were available and was not able to take account of the emerging trends in the Scottish Health Survey or child datasets. It included chapters on eating disorders and perinatal mental health, but otherwise did not specifically focus on women’s mental health. There is a need, therefore, for a consolidation of recent evidence on trends and patterns.

In terms of background to this work: NatCen produced the initial survey report for APMS 2014. They also have used the survey data to produce a brief report for Department of Health examining inequalities in mental health and treatment use among people with intellectual impairment, for which DARS approval was obtained. The objectives of those pieces of work are distinct from this study.

Only APMS data has been requested – no other datasets have been requested. APMS is only available as a full dataset, though minimisation has taken place centrally prior to the dataset being made available. In addition, the full dataset is required in order to cover all England (national analyses) and to be able to control for a range of factors in the analyses. A range of conditions will be examined in the analyses. The data is pseudonymised and the dataset has been minimised as standard, and disclosure control suppression applied. The dataset has been through NHS Digital's Disclosure Control Panel and approved.

2014 survey data is required to plan and deliver the next survey in the series. The survey is made up of two phases, with the second phase carried out with a subsample of phase one participants. In order to design the sampling strategy for the phase two data collection, NatCen must analyse how people responded at the phase one data collection. NatCen conduct this work alongside the NHS Digital surveys team, who NatCen work closely with. NatCen work with the data on Freedom of Information (FoI) requests that come to NHS Digital, and other further data requests submitted to NHS Digital, e.g., from Public Health England and Cabinet Office. NatCen also deal with queries from data users, e.g., if someone is unsure how to use a weighting variable or has a query about how to score the scales or whether there is an error in their syntax.

The National Centre for Social Research is the sole data controller and data processor in this agreement. No other organisations have any role in how the data is processed, what the purpose of the study is, or have any access to the data.

The scope of the analyses have been conceived by and will be entirely conducted by NatCen Social Research. NatCen are supported by a steering group of experts from a range of organisations, including central government (Department of Health, Public Health England), NHS England, third sector (Agenda), and other research institutions (such as University College London and Kings College London). The group will provide advice however, all final decisions about scope will be the preserve of NatCen. All analyses and processing of the data will be carried out by NatCen.

The legal basis for processing this data falls under Article 6 (1)(f) of the GDPR, giving the lawful basis for processing where “processing is necessary for the purposes of the legitimate interests pursued by the controller or by a third party except where such interests are overridden by the interests or fundamental rights and freedoms of the data subject which require protection of personal data, in particular where the data subject is a child.” NatCen has completed a Legitimate Interest Assessment and determined that they need to process the APMS survey dataset in order to investigate prevalence and inequalities in adult mental health and treatment and service use in England. The legal basis for processing this data also falls under Article 9 (2) (j) of the GDPR, that is, 'processing is necessary for archiving purposes in the public interest, scientific or historical research purposes or statistical purposes'. The Legitimate Interests Assessment confirms that there is significant interest to both the public, and the Health and Social Care system, in the processing of data as described in this Agreement.

Although the analyses are not required by law, they are of a clear benefit to the government, to health and social care services, and to the wider population. The data are the most recent, most robust, and most suitable for establishing level of mental health care need, whether this level is changing, and which groups have the greatest level of need. This information has the benefit of improving the planning of services in terms of anticipating demand and improving the targeting of services and in terms of identifying which groups in the population are most likely to need support. The analyses have limited privacy impact on the individual as disclosure risk is tightly managed with the pseudonymisation of the data prior to data archiving and the publication of results in statistical form. Information leaflets provided to survey participants explained that the survey data would be analysed and reports and papers produced, and therefore participants would have expected the data to be used in this way. The legitimate interests basis is appropriate for these secondary analyses given there is minimal impact on the individual and compelling justification for the processing.

The 2014 APMS dataset is held on behalf of NHS Digital by the UK Data Service (UKDS) (www.ukdataservice.ac.uk ) and UKDS are responsible for dissemination under direction by NHS Digital. NatCen previously received the whole dataset; as there is no facility to select individual variables. They were able to download the dataset from UKDS for the period specific within the DSA and had to securely destroy all local copies of the dataset when the DSA expired and notify DARS in line with standard procedures.

As NatCen downloaded the data securely from the UK Data Service archive, there were no flows of data from NHS Digital or to NHS Digital. NatCen will store this data securely for the agreed period and not share it with any third parties. NatCen has secure data storage systems in accordance with those specified in the NHS Digital Data Protection and Security Toolkit, including building security, PGP encryption, password protection, and secure network locations where only named and certified individuals can access data. Data are not stored on laptops, rather on central and securely held servers. There are no further data flows. The UKDS securely transferred the APMS dataset to NatCen. The data file contains no identifiers and the dataset is not linked and will not be further linked.

Data will only be accessed by individuals within NatCen who have authorisation to access the data for the purpose(s) described, all of whom are substantive employees of NatCen.

What will be done with the data:

- The data will not be linked with any other data.

- There will be no requirement nor attempt to reidentify individuals from the data.

- The data will not be made available to any third parties except in the form of aggregated outputs with small numbers suppressed in line with the HES Analysis Guide.

- The data will only be used for the purposes described in this agreement.

Data management and variable derivation will be done in the software, SPSS (Statistical Product and Service Solutions). Analyses will be conducted in SPSS, Stata and Latent Gold, using survey weights and controlling for complex survey design. All patterns reported on will be tested for statistical significance.

Simple descriptive analyses will be used to generate prevalence rates and produce cross-tabulations. These will be presented in tabulations (using percentages and means, and always showing base sizes), bar charts, and line graphs. Trend data will be based on consistently defined age-ranges for comparability across years. Multiple regression analyses will be carried out to examine association between risk factors and mental health outcomes when other factors are controlled for, and to identify the role of different potential mediators. Population attributable fractions will be calculated to compare risk factors with variable extent and variable strength of association. NatCen will examine different options for testing what risk factors may be driving any changes in rate or distribution of mental illness. Latent class analyses will be conducted to profile how different and multiple adversities across mental health and other life circumstances cluster together.

While the focus of the study is women’s mental health, these patterns are most meaningful when examined in relation to men’s mental health. Where available, analyses will be conducted on both men and women, and on the population as a whole, so that comparisons can be made.

An objective of the study is to pull together information on trends in mental health from a range of different sources, and so while APMS will be the primary resource, a number of other datasets will also be analysed. Another objective is to look at trends in the risk factors for the relevant subgroups (e.g., is the proportion of 16-24 year olds in debt increasing? What are the trends in partner violence?). For this, other datasets may need to be utilised, depending on what risk factors emerge from the initial analyses.

APMS: The national Adult Psychiatric Morbidity Survey (APMS) is England’s primary tool for monitoring the extent of mental illness in the population. Headline figures from the study indicate pronounced increases in self-harming behaviour among young women, but initial reporting only looked at associations with a few risks. The survey included screening for the presence of a range of different types of mental disorder, and information about many aspects of lived experience. As a general population survey, the sample is representative of people living in a household irrespective of whether they have a diagnosis or get treatment or services. While the survey doesn’t cover the prison population or people or are homeless, it does ask about previous experience of these things. Trends in symptoms will be examined, as well as trends in CMD as a whole. This will enable trends to be isolated for symptoms such as anxiety.

This 2014 version of the dataset available via DARS has been redacted on Disclosure Control Procedure advice to minimise the likelihood of individuals being able to identify anyone taking part in the survey.

All organisations party to this agreement must comply with the Data Sharing Framework Contract requirements, including those regarding the use (and purposes of that use) by Personnel (as defined within the Data Sharing Framework Contract i.e.: employees, agents and contractors of the Data Recipient who may have access to that data).

The programme of research activity at NatCen has already yielded a series of important publications and outputs. NatCen have kept NHS Digital aware of outputs and informed them when publications have been released. These have included academic papers in journals that reach a wide clinical and practice audience, like the Lancet Psychiatry (see https://www.thelancet.com/journals/lanpsy/article/PIIS2215-0366(19)30188-9/fulltext) and Social Psychiatry and Psychiatric Epidemiology (https://link.springer.com/article/10.1007/s00127-019-01797-5) and reports that have been coproduced with user-group and awareness organisations like Agenda (for example, see: https://weareagenda.org/often-overlooked/). This work has been presented extensively to other organisations, e.g., to Mind, the DHSC and DWP.

Comprehensive reports continue to be written, clearly and making extensive use of graphs and charts. Methodological details are contained in separate appendices or technical reports. The reports and papers address the main objectives, including presenting trends in women and men’s mental health over time; risk factors for poor mental health in men and women; trends in risk factors over time in men and women; and lived context for subgroups of men and women.

An article has been published in the British Journal of Psychiatry (see https://www.cambridge.org/core/journals/the-british-journal-of-psychiatry/article/revisiting-the-one-in-four-the-prevalence-of-psychiatric-disorder-in-the-population-of-england-20002014/F4FA80EB6EB9FBE20729F986853E7718) which has provided important context for interpreting prevalence and change over time.

Outputs that have been produced from the programme of work so far include:

McManus S and Agenda. (2020) Often overlooked: Young women, Poverty and Self-harm. Agenda.

Bebbington PE, McManus S, Coid JW, Garside R, Brugha T. The mental health of ex-prisoners: analysis of the 2014 English National Survey of Psychiatric Morbidity. Social Psychiatry and Psychiatric Epidemiology. 2021 Mar 22:1-1.

Wardle, H, John A, Dymond S, McManus S. Problem gambling and suicidality in England: secondary analysis of a representative cross-sectional survey. Public Health 184 (2020): 11-16.

Bebbington PE, McManus S. Revisiting the one in four: the prevalence of psychiatric disorder in the population of England 2000–2014.

McManus, S. (2020). Using repeated cross-sectional surveys to measure trends in rates of self-harm. SAGE Research Methods Cases. doi:10.4135/9781529733679

McManus S, Gunnell D. (2019) Trends in mental health, self-harm and suicide attempts in 16 to 24-year old students and non-students in England, 2000-2014. Social Psychiatry and Psychiatric Epidemiology.

McManus S, Bebbington P, Jenkins R, Morgan Z, Brown L, Collinson D, Brugha T. (2019) Data Resource Profile: Adult Psychiatric Morbidity Survey. International Journal of Epidemiology.

McManus S, Gunnell D, Cooper C, Bebbington PE, Howard LM, Brugha T, Jenkins R, Hassiotis A, Weich S, Appleby L. (2019) Prevalence of non-suicidal self-harm and service contact in England, 2000-14: repeated cross-sectional surveys of the general population. Lancet Psychiatry.

Graham H, White P, Cotton J, McManus S. (2019) Weather- and Flood- damaged homes and mental health: a population profile using England’s mental health survey. International Journal of Environmental Research and Public Health. 16(18), 3256.

McManus S. (2019) General population surveys - comparing student and non-student mental health. Smarten.

Wardle H, Dymond S, John A, McManus S. (2019) Problem gambling and suicidal thoughts, suicide attempts and non-suicidal self-harm in England: evidence from the Adult Psychiatric Morbidity Survey 2007. Gambling Commission.

McManus S, Lubian K, Bennett C, Turley C, Porter L, Gill V, Gunnell D, Weich S. (2019) Suicide and self-harm in Britain: researching risk and resilience using UK surveys. NatCen: London.

The following outputs will be produced:

- Journal papers for publication in open access peer-review journals (three papers published so far, more planned).

- Summary public and policy-facing briefings co-produced with third sector and other organisations (who will not have any access to the underlying data, only to the aggregated results), highlighting the most relevant and accessible findings. These will tie in with blogs and other social media promoted communications. Reports have been co-produced so far with organisations like Agenda and Mind, who do not have any access to underlying data, but only to aggregated results.

- Part of NatCen's contracted role is also to promote use of the survey series and its data – so NatCen organise outputs like webinars for the UK Data Service and blogs. This also extends to seminars and presentations, including to the All Party Parliamentary Group on Sex Equality, at academic conferences (preferably, the European Public Health Association annual conference). Presentations have been done at events organised by MQ and others.

Events at existing festival events and talks with general public and school participation. This, in practice, has proved difficult this year with COVID-19, but more public engagement is hoped for the coming year.

All outputs will be aggregated with small numbers suppressed in line with the HES analysis guide.

Mental illness and mental wellbeing have risen higher up the policy agenda in recent years; the main political parties all featured mental health and mental health services in their election manifestos. Despite this attention, women’s mental health has not until recently been the focus of policy attention, specific study, or specialist provision. An initial benefit of this work will be to highlight and quantify the scale of the issue and demonstrate whether it should be considered a priority for policy.

The government’s primary source of data on prevalence and trends in mental disorders in England – the Adult Psychiatric Morbidity Survey (APMS) – published initial results back in 2016 (McManus et al 2016). Not only did the report show that the longstanding higher rates of anxiety and depression in women remain, but also that this gap may have grown. That data is the best source for examining trends in detail but has not yet been used to do so.

In response to the APMS results, and in recognition of the lack of policy attention that women’s mental health has generally received, the then Parliamentary Under Secretary for State for public health set up a new Taskforce. A different MP has since taken over from this Secretary for State, and therefore this responsibility has been passed on to them. Its remit is to make recommendations for what the country should do to address gender inequalities in mental illness. This policy development would benefit from a major consolidation of recent research evidence from different sources on what is known about women’s mental health. This is especially needed, given that the prevalence and patterning of mental illness in the population appears to be changing, with the highest risk age-group in women shifting to those in their teens and early twenties.

Groups like Agenda and Mind have called for the establishment of gender-specific mental health service provision. A recent Freedom of Information request from Agenda found just one NHS mental health trust with a strategy for providing gender-specific services to women, and most trusts provided no relevant policies or strategies in relation to gender specific services (Agenda 2016). Furthermore, most responding trusts had no policy on ‘routine enquiry’ about abuse, contrary to NICE guidelines. The results from this study could be used to help inform both the work of the Women's Mental Health Taskforce and in the development of gender-aware mental health service provision.

The reason for processing APMS data is to use the best available information on general population patterning in mental disorder and self-harm behaviours in adults in England, to examine prevalence and identify factors associated with mental disorder and treatment and service use. This is to ensure government have an accurate understanding about population trends in need and can prioritise and plan services accordingly. And also to ensure that the public have an accurate understanding of adult mental health. Without such information resources are allocated, services designed and commissioned, and public behaviour and concern will be based on outdated and incorrect information.

The Adult Psychiatric Morbidity Survey dataset contains important information on the prevalence of mental disorders assessed using standardised and validated instruments in a general population sample of adults in England. The questionnaire collects information from on treatment and social, health and economic context that is crucial to understanding temporal trends, inequalities in prevalence, risk factors and the scale and nature of treatment gaps. Understanding these patterns is key to our research institutes missions of creating a better and healthier society through better information, in particular for: understanding a) the reach of existing services, b) the demand for additional services, c) factors and context that prevention programmes should target or address.

The project also seeks to bring benefits through enabling public engagement with the findings:

Third sector and policy involvement: Agenda – an alliance of 70 voluntary sector organisations with an interest in redesigning systems and services to better address the needs of women girls – will help in the development of both public-facing and policy-facing summary outputs.

Academic collaboration: an academic advisory group will provide insight from a range of disciplines and current thinking in those areas.

Public engagement: alongside conventional seminar/conference presentations, sessions at events aimed at the general public and schools will also be sought out, including sessions at the Cheltenham Science Festival, local TEDx events, and Wellcome Trust talks.

The programme of research has already yielded benefits.

A key benefit has been the attention drawn to the increasing rates of self-harming in the population. A paper was published in the Lancet Psychiatry that got significant media attention (print and radio) and the paper has been widely cited in both academic literature and in policy briefings.

The article published in the British Journal of Psychiatry has also been key to informing understanding of what mental illness 'is', which is often misunderstood.

The paper published on student mental health has informed the work of the UKRI-funded SMARTEN network, by highlighting the mental health trajectories of both students and non-students (so they can be compared, and the difficulties of the non-student population also seen) and the differences by gender.

NatCen's briefing, co-produced with Agenda and widely reported in The Guardian and elsewhere, has been important for highlighting the wider social determinants in relation to self-harm, and the importance of this alongside a health-service related response.