Article Text
PDF
Abstract
Background Previous evaluations of area-based initiatives have not compared intervention areas with the full range of areas from top to bottom of the social spectrum to evaluate their health inequalities impact.
Setting Deprived areas subject to the New Deal for Communities (NDC) intervention, local deprivation-matched comparator areas, and areas drawn from across the socioeconomic spectrum (representing high, medium and low deprivation) in England between 2002 and 2008.
Data Secondary analysis of biannual repeat cross-sectional surveys collected for the NDC National Evaluation Team and the Health Survey for England (HSE).
Methods Following data harmonisation, baseline and time trends in six health and social determinants of health outcomes were compared. Individual-level data were modelled using regression to adjust for age, sex, ethnic and socioeconomic differences among respondents.
Results Compared with respondents in HSE low deprivation areas, those in NDC intervention areas experienced a significantly steeper improvement in education, a trend towards a steeper improvement in self-rated health, and a significantly less steep reduction in smoking between 2002 and 2008. In HSE high deprivation areas, significantly less steep improvements in five out of six outcomes were seen compared with HSE low deprivation areas.
Conclusions Although unable to consider prior trends and previous initiatives, our findings provide cautious optimism that well-resourced and constructed area-based initiatives can reduce, or at least prevent the widening of, social inequalities for selected outcomes between the most and least deprived groups of areas.
- DEPRIVATION
- INEQUALITIES
- POLICY
- SOCIO-ECONOMIC
Statistics from Altmetric.com
Introduction
Despite well-documented differences in physical and psychological morbidity and health-related behaviours by socioeconomic circumstances, less is known about the success of initiatives to tackle these socioeconomic inequalities in health. Area-based initiatives have been proposed and implemented within the UK and internationally as one approach. The suggested advantages of an area-based approach are that their remit extends beyond traditional healthcare settings to improve upstream determinants of health, including housing, employment, education and training and aspects of the physical and social environment at multiple levels. They are targeted to direct resources to residential areas identified as having the greatest need. Interventions at the environmental (rather than individual) level may help reduce barriers to access and uptake. However, area-based targeting of resources may not be an efficient way of reaching the majority of disadvantaged people. Furthermore, many projects within the umbrella of an area-based initiative require individuals to access and participate and so, as we and others have previously demonstrated, more advantaged residents in a deprived area may benefit to a greater extent than less advantaged residents, at least in the early years of an intervention.1–3
Recent evaluations of area-based initiatives within England include Health Action Zones,4 the Single Regeneration Budget,5 and a national evaluation of the New Deal for Communities (NDC) programme.6 The NDC started in 1998 and was a substantial 10-year programme that targeted 39 of the most deprived areas in England. It was set up to improve a range of social outcomes on six domains: housing and the physical environment, crime, community, worklessness, health and education. Each NDC intervention area received £50 million over 10 years, totalling £2 billion over the 39 areas during the lifetime of the initiative. 7
There are three main approaches to tackling social inequalities in health ranging from (A) interventions aimed at improving the health of the poorest in society; (B) those aimed at narrowing the health gap between the poorest and more advantaged; to (C) reducing the health gradient across the socioeconomic spectrum.8 Each carries implications for evaluation, for example, for approach ‘A’, the comparison would be between equally poor areas with and without the intervention. For approach ‘B’, the comparison would be between the poorest areas receiving the intervention and more advantaged areas not receiving the intervention (commonly the comparators are either the most advantaged areas or the average for the country as a whole). Interventions taking approach ‘C’ would need to be evaluated by assessing the progress made by each section of the socioeconomic spectrum towards ‘levelling up’ to the health standard reached by the most advantaged group. Since only the most deprived areas were the target of the NDC intervention, however, the intervention could not be expected to influence other sections of the socioeconomic spectrum and cannot be evaluated here against approach ‘C’ criteria. Essentially, changes in the gradient due to the NDC would be analytically equivalent to evaluating ‘B’ above.
Although the NDC was set up to ‘reduce the gaps between some of the poorest neighbourhoods and the rest of the country’,6 that is, implying approach ‘B’, there has been limited evaluation of progress towards this objective. The national evaluation team, in common with evaluations of other area-based initiatives, has monitored and compared wide-ranging outcomes in areas matched for baseline multiple deprivation within the same broad geographic area to facilitate statistical comparison to estimate an ‘NDC effect’ (evaluation approach ‘A’). The national evaluation team also carried out more limited comparisons of outcomes in NDC areas against national benchmarks (evaluation approach ‘B’).6
The aim of the study reported here is to conduct a comprehensive health inequalities impact assessment by employing evaluation approaches A, B, and an appropriately modified version of C in which we monitor changes in areas drawn from across the socioeconomic spectrum, not just two extremes. We do not attempt to identify the impact of particular projects, since these varied substantially in scope, reach and outcome of interest in each area. Instead, we treat the NDC intervention as a holistic one in order to determine whether the programme as a whole contributed towards reducing health inequalities and their social determinants.
Methods
The intervention
Implementation of the NDC intervention varied locally and comprised over 6000 different projects in the 39 NDC areas (see ref 9 for details of the intervention). We use data from household surveys representative of households in England to supplement household survey data collected by the National Evaluation Team in NDC intervention and deprived local comparator areas.
NDC intervention and comparator area data
Household surveys were conducted by MORI in 2002, 2004, 2006 and 2008. In each wave, approximately 17 700 residents in NDC intervention areas were sampled. In 2002, one adult from each randomly sampled household was invited to participate. In subsequent waves, a panel component plus a randomly selected refreshment sample was recruited (the latter constituting 44% of the sample). Following the same random sampling procedure, approximately 3000 residents per wave were sampled in non-contiguous NDC comparator areas, matched on area deprivation score and local authority district.
Data for areas drawn from across the socioeconomic spectrum
The Health Survey for England (HSE) is an annual cross-sectional survey that is nationally representative of households in England. It adopts a multistage probability sampling design selecting a sample of postcode sectors from the Postcode Address File and households from each postcode sector. All adults (aged 16 years of age and over) were selected for interview in each household. Topics include general health, health-related behaviours, chronic disease, and demographic and socioeconomic data, with some changes to included modules each year.10 Data from the core samples of HSE in 2002, 2004, 2006 and 2008 were used to measure health and its social determinants in areas drawn from across the socioeconomic spectrum.
Measures
Area deprivation
The Index of Multiple Deprivation 2004 (IMD 2004) captures deprivation on seven domains (income, education, employment, health, crime, access to housing and services and the physical environment) at the super output area level. 11 With the exception of two areas which were in the second highest quintile, all NDC intervention and NDC comparator areas had IMD2004 scores in the highest (most deprived) quintile. We grouped HSE participants into those living in the highest two, middle, and lowest two quintiles of IMD2004. These groups of areas are referred to as HSE high deprivation, HSE medium deprivation and HSE low deprivation, respectively.
Health and social determinants of health
Six outcomes of interest could be acceptably harmonised across household surveys. Mental health was measured using the Mental Health Inventory-5 (MHI-5)12 in the NDC intervention and NDC comparator surveys. This is a five-item instrument with possible responses on a 5-point scale. Those who reported at least two symptoms ‘all of the time’ or ‘most of the time’ (versus ‘some of the time’ or less frequently in the past 4 weeks) were defined as having poor mental health. Two items capturing positive mental health were reverse coded. The 12-item General Health Questionnaire (GHQ-12)13 captured symptoms of mental distress in the HSE. Those scoring five or more out of a possible 12 were classified as having poor mental health. Self-rated health over the previous 12 months was captured on a three-point scale (‘good’, ‘fairly good’, ‘not good’) in the NDC intervention and NDC comparator surveys. Those reporting that their health was ‘not good’ were defined as having poor self-rated health. Self-rated health was captured on a 5-point scale in the HSE. Those reporting that their health was ‘bad’ or ‘very bad’ (rather than ‘very good’, ‘good’ or ‘fair’) were classified as having poor self-rated health. In all surveys, current smokers were distinguished from ex-smokers and never-smokers. Current employment status was captured using the same item and possible responses in all surveys and we classified participants into those who were and were not in paid employment. Housing tenure was dichotomised as ‘owning it outright’ or ‘buying with the help of a mortgage or loan’ (here described as owner occupancy) versus ‘renting’, ‘part renting’, or ‘living rent-free’. Participants in all surveys were defined as having low educational attainment if they had no formal qualifications. Ethnicity was self-identified using the response options provided in the 2001 England census and grouped as white, Asian, Black or other.
Statistical analysis
We hypothesised that improvements in health and the social determinants of health between 2002 and 2008 would be greater in NDC intervention areas than in areas drawn from across the socioeconomic spectrum. In order to test this, we estimated the difference per wave (ie, per 2 years) in the likelihood of each outcome, adjusting for baseline level and sociodemographic characteristics (age, sex and ethnicity) of participants using separate multilevel logistic regression models for each outcome. Individual residents (level 1) were clustered within small areas (level 2), the latter defined by postcode sector boundaries in the HSE data and by NDC boundaries in the MORI data. Postcode sectors and NDC neighbourhood geographies are different, but both contain resident populations of approximately 10 000 residents. The regression slope representing rate of change was estimated for each group of areas with HSE low-deprivation areas being taken as the reference group.
With employment status as the outcome of interest, we limited the analytical sample to men aged 65 years of age and under, and women aged 60 years of age and under. With educational attainment as the outcome of interest, we limited the sample to those aged 25 years of age and over. We repeated models after removing participants who had moved since 2002 and, since estimates were very similar, we report only the full sample models in the results section. All unadjusted prevalence estimates were weighted for the multistage survey design and non-response using weights provided by the survey depositors. Analyses were conducted using Stata V.12.0 SE and based on complete cases; missing item frequencies were low (2.3% for mental health, 0.9% for educational attainment and <0.5% for all other outcomes and covariates).
Results
The sociodemographic characteristics of participants in each wave are summarised in table 1. Those drawn from NDC intervention and NDC comparator areas were less likely to be of white ethnic origin and more likely to be of Asian or black ethnic origin than those in the HSE low, medium and high deprivation groups. Despite being matched by deprivation and local authority district, participants from NDC intervention areas were less likely to be of white ethnic origin than those in NDC comparator areas (p<0.001 in all four waves). Table 2 shows that residents in NDC intervention areas were also more likely than those in NDC comparator areas to not be in paid employment, to have no formal qualifications and to be renting their accommodation (p<0.05 for all outcomes in all four waves).
Weighted demographic profile of respondents by deprivation group and survey wave
Weighted prevalence of reporting poor health and social determinants of health by deprivation group and survey wave
The weighted, unadjusted prevalence of each outcome is presented in table 2. Substantially different crude rates of poor self-rated health in NDC intervention and NDC comparator areas compared with HSE high-deprivation areas (in which rates were higher) could indicate some lack of harmonisation in this measure. On the other hand, differences in the prevalence of not being in paid employment, the prevalence of having no formal qualifications, and the prevalence of living in rented accommodation between NDC intervention and NDC comparator areas versus HSE areas (in which rates were considerably lower) cannot be explained by differences in item wording.
Table 3 summarises the estimates from the multiple adjusted logistic regression models for each outcome. Compared with the HSE low-deprivation areas reference group, the likelihood of smoking was higher in all other groups of areas at baseline. This is indicated by positive intercept coefficients with 95% CIs that do not include zero. These coefficients can be converted by exponentiation to ORs and, for example, show that those in NDC intervention areas were 3.49 (95% CI 3.06 to 3.97) times as likely to be a smoker as those in HSE low-deprivation areas. The likelihood of smoking reduced between 2002 and 2008. Each 2-year advance in time was associated with a drop in the log likelihood of smoking of −0.056 (95%CI −0.077 to −0.035). In other words, each 2-year advance in time was associated with an OR of 0.95 (95%CI 0.93 to 0.97) of smoking. There was evidence of a difference in slope between the reference HSE low-deprivation areas and NDC intervention areas, NDC comparator areas and HSE high deprivation areas. The decline in the likelihood of smoking was less steep in these three groups of highly deprived areas than in the HSE low deprivation reference (indicated by positive slope coefficients). These associations are illustrated in figure 1A. There was also a decline in the likelihood of poor mental health in the HSE low-deprivation areas. There was no evidence of different rate of change in mental health between the reference areas and any other groups of areas (figure 1B).
Modelling health and social determinants of health outcomes by deprivation group over time, adjusted for sex, age and ethnicity using multilevel logistic regression analysis
(A) Likelihood of being a current smoker by deprivation group over time. (B) Likelihood of poor mental health by deprivation group over time. (C) Likelihood of poor self-rated health by deprivation group over time. (D) Likelihood of not being in paid work (working age residents only) by deprivation group over time. (E) Likelihood of renting accommodation by deprivation group over time. (F) Likelihood of no qualifications by deprivation group over time.
Compared with the HSE low-deprivation areas reference group, there was evidence of a higher likelihood of poor self-rated health in all other deprivation groups of areas at baseline. There was a non-significant trend towards decline in the likelihood of poor self-rated health in the reference group over time and statistically significantly less steep decline in the likelihood of poor self-rated health in HSE high and medium deprivation areas (table 3 and figure 1C). There was a suggestion of a slightly steeper decline in the likelihood of poor self-rated health in NDC intervention areas though this did not attain statistical significance (p=0.09).
At baseline, residents in all groups of areas were more likely to not be in paid work and more likely to be renting their accommodation compared with the HSE low deprivation reference. There was a decline in the likelihood of not being in paid employment in HSE low-deprivation areas and statistically significant differences in slopes for not being in paid employment and renting accommodation in HSE high-deprivation areas (figure 1D). In HSE low-deprivation areas, each 2-year advance in time was associated with an OR of 0.99 of renting accommodation. In HSE high-deprivation areas, each 2-year advance in time was associated with a log OR of 0.076 (calculated as −0.011+0.087, ie, the sum of the slope coefficients in the reference group and the HSE high deprivation group), or an OR of 1.08 of renting accommodation (figure 1E).
The likelihood of having no qualifications declined over time in the reference group, declined more steeply in NDC intervention and NDC comparator areas and declined less steeply in HSE high-deprivation areas (figure 1F). Whereas the log likelihood of having no qualifications decreased by 0.024 per 2 years in HSE low-deprivation areas, the HSE high-deprivation areas saw a rise of 0.006 (–0.024+0.030) per 2 years (p=0.04).
Findings were essentially unchanged when analyses were restricted to the subgroup of residents in NDC intervention areas who had lived at the same accommodation since 2002 or before (data available on request from the authors).
Discussion
Highly deprived areas, whether NDC intervention areas, NDC comparator areas, or HSE high-deprivation areas, had higher likelihood of disadvantage on all outcomes at baseline compared with the least deprived areas, as expected. There was a general trend of improvement on four of the six outcomes (the exceptions being self-rated health and housing tenure). Compared with HSE low-deprivation areas, residents in NDC intervention areas experienced a significantly steeper improvement in education, a trend towards a steeper improvement in self-rated health, and a significantly less steep reduction in smoking. Our findings, therefore, provide some evidence that the NDC intervention may have contributed to narrowing, or at least preventing the widening of, the gap between the most and least disadvantaged parts of England (approach ‘B’ as laid out in the introduction). By contrast, in HSE high-deprivation areas, significantly less steep improvements in five out of six outcomes were seen compared with HSE low-deprivation areas. Although, for brevity, we have not statistically compared changes in outcomes in the NDC intervention and HSE high-deprivation areas, these findings indicate that NDC intervention areas saw improvements in health and the social determinants of health that were not mirrored in similarly deprived areas (approach ‘A’). The findings also illustrate the changes in areas in the middle of the socioeconomic spectrum (a modified version of approach ‘C’), and generally indicate somewhat less dramatic improvements in outcomes than those seen in either the least disadvantaged areas or the NDC intervention areas. (Results using these alternative reference categories for formal statistical comparison are available from the authors.)
Strengths and limitations of the current study
The reliability of these results rests heavily on having achieved adequate harmonisation across surveys. The greatest challenge to harmonisation relates to the self-rated health and mental health outcomes. It is not clear whether this would affect the time trend as well as the overall prevalence of poor self-rated or mental health. In addition to differences in instruments and item wording, the surveys were led by different organisations and may have had different ways of handling non-response/non-contact at the household doorstep. However, all surveys were conducted by face-to-face interview and the items capturing covariates and social determinants of health outcomes were similarly worded. These analyses are based on repeat cross-sectional studies and are not assessing within-person change over time. This reduces the problem of attrition, but means that differences in the sample and residential mobility could contribute to time trends. Many NDCs undertook to increase owner-occupancy rates as a way of improving residents’ outcomes and diluting problems related to concentrated social housing. However, one consequence of this may be the displacement of original residents. To address this, we undertook sensitivity analysis excluding those who had recently moved and adjusted for a range of demographic characteristics in order to make the samples as similar as possible. It was not possible to identify NDC intervention or comparator areas from the Health Survey for England. It is possible that some individuals in the HSE high deprivation group could be located in NDC intervention areas, however, we estimate this overlap to be small at approximately 100 per wave.
Relationship to previous studies
The positive change in self-rated health in NDC intervention areas supports an earlier review that synthesised impact data from 10 area-based interventions and showed a 3.8% improvement in self-rated health across intervention areas.14 Previous evaluation of the NDC programme has demonstrated greater improvement on 18 out of 36 indicators compared with the national benchmark.6 We add to this by demonstrating some narrowing of the socioeconomic gap on some, though not all, outcomes of interest. Previous work suggested improvements in the education domain were the most difficult to achieve when evaluated against matched deprived comparators.6 Our analyses, evaluated against a less deprived reference, by contrast show a steeper reduction in the proportion of residents having no formal qualifications in the NDC intervention areas. These contrasting findings may be due to the use of different educational indicators in the two studies (we assessed the likelihood of having no qualifications while the national evaluation team used indicators of primary and secondary school progress (key stages 2–4)), and the considerably different levels of educational attainment at baseline (with the prevalence of having no qualifications in the NDC intervention areas being more than double that in the least deprived reference areas at baseline). It is interesting that NDC comparator areas showed similar improvements in the likelihood of not having any qualifications. There are several potential explanations. First, although matched by deprivation with the IMD2004, those living in the comparator areas did not appear as deprived as respondents from the NDC areas. The NDC programme commissioners stipulated that local authorities used the intervention as an opportunity to assist the most deprived communities. In some local authorities, there may not have been equally deprived areas to include as comparators. Second, the comparator areas may have been the recipients of other interventions that were taking place during this period, such as the Spearhead initiative15; however, we were unable to explore this further in this study. Third, although the selected comparator areas were non-contiguous to the NDC intervention sites, the effects of the NDC intervention may have spilled over into these communities, but we were unable to test for any contamination. It is likely that implementation of education and employment policies, for example, was not constrained within the very local NDC area but stretched beyond NDC boundaries.
Implications for practice and future research
Our analysis has focused on individual level outcomes, and taken an arbitrary baseline without being able to consider prior trends and previous initiatives rather than a complex systems approach, as has been advocated, and may well be appropriate for evaluation of this intervention.16 Despite these limitations, the findings indicate some differential improvement in educational attainment, one key social determinant of health, in NDC intervention areas compared with areas at the top of the socioeconomic spectrum. While several other outcomes considered here did not demonstrate significant narrowing of the gap between NDC intervention areas and the least deprived end of the socioeconomic spectrum of areas, it is important to note that socioeconomic inequalities in the UK as a whole widened between 2002 and 2008. There was evidence of a widening gap on five of the six social determinants in HSE high versus HSE low-deprivation areas, whereas, only the gap in smoking widened in NDC intervention versus HSE low-deprivation areas. In other words, the NDC intervention may have helped to prevent a further widening of the gap, if not an actual narrowing, in some of these outcomes, which is still an important achievement.
Previously, area-based initiatives have been criticised for being too ambitious for the available resources and existing evidence for improving social and health outcomes and narrowing inequalities.17 Our findings when combined with the earlier evaluations of the New Deal for Communities initiative, however, provide cautious optimism that well-resourced and constructed area-based initiatives can provide modest improvements in health outcomes and social determinants of health, and perhaps more importantly, reduce, or at least prevent the widening of, social inequalities for selected outcomes between the most and least deprived groups of areas. These changes, when spread over a population may have far-reaching effects.
As far as we are aware, this is the first evaluation to consider the social inequalities in health impact of an area-based initiative. We have noted several challenges in identifying suitable comparative data, but contend that this is a useful approach to give an alternative perspective on the effectiveness of complex area-based interventions. On-going work is describing the nature of the NDC intervention in more detail in order to identify the particular features that are most strongly related to reducing or containing social inequalities in health and the determinants of health.
What is already known on this subject
-
Evaluations of the health impact of complex area-based interventions typically assess health improvement against a deprived comparator, or the national average. Previous studies have not compared with areas drawn from across the socioeconomic spectrum.
-
The New Deal for Communities (NDC) was a substantial programme which provided around £2 billion to 39 deprived areas in England between 1998 and 2011. Recent evaluation of the NDC found modest improvements in aggregated place-based outcomes, but limited change in people-based outcomes. It did not assess the health inequalities impact.
What this study adds
-
Compared with residents in low-level deprivation areas, those in NDC intervention areas saw a significantly greater reduction in the likelihood of low educational attainment and a trend towards a greater reduction in the likelihood of poor self-rated health.
-
By contrast, residents in deprived areas not undergoing the intervention saw a significantly greater increase in the likelihood of five out of six indicators of poor health and its social determinants (smoking, poor self-rated health, worklessness, renting one's home, and low educational attainment).
-
Well-resourced area-based initiatives may prevent the widening of social inequalities for selected outcomes between the most and least deprived groups of areas.
Acknowledgments
The authors would like to thank the NDC National Evaluation Team, the Centre for Regional Economic and Social Research (CRESR) at Sheffield Hallam University and, in particular, Paul Lawless and Christina Beatty, and Christine McGuire for their support. We would like to acknowledge funding support from the Department of Health. This is an independent report of research commissioned and funded by the Policy Research Programme in the Department of Health (Title: Evaluating the impact of New Deal for Communities on Health Inequalities: Phase 2. Reference No: PR-IP-0509-0180063). The views expressed are those of the authors and not necessarily those of the Department of Health.
Footnotes
-
Contributors JP, MW, JN, MS and HB designed the study. HB, PW and MS collated the data and conducted data analysis; MS and HB drafted the paper; all authors contributed to interpreting results, revised the draft and approved the final version.
-
Funding This research was funded by the Department of Health Policy Research Programme (PR-IP-0509-0180063).
-
Competing interests None.
-
Ethics approval Ethical approval for the survey was obtained from the London Multicentre Research Ethics Committee.
-
Provenance and peer review Not commissioned; externally peer reviewed.
-
Data sharing statement The data used in this study are available through the UK Data Archive (http://data-archive.ac.uk/).