The research evidence on which we act, or fail to act, is frequently imperfect. Even the most compelling research stories, such as that describing the well known Hawthorn effect, impose an academic interpretation on an “untidy reality” (p ).¹ Where we have important social problems, amenable to remedy, it is important not only to find out whether any solution is effective, but also to know how, under what conditions, and in what contexts a given solution may be made to work to maximum effect.

Children in social class V* are 16 times more likely to die in a fire than their counterparts in social class I.² This is partly attributable to the fact that socioeconomically disadvantaged children live in households at high risk of fire for both individual reasons (for example, cigarette smoking) and structural reasons (for example, a low level of smoke detector installation).³

Fires discovered by smoke alarms cause less damage, are discovered more rapidly, and are associated with lower casualty rates,⁴ yet in an analysis of 2002–03 British Crime Survey data, around a quarter of the 6000 strong sample did not have a working detector in their home.⁵ Six per cent had a non-functioning detector and 18% did not own a smoke detector at all.⁵

There is a relatively well developed body of research on the socioeconomic and demographic characteristics of people who live in households where there is no working smoke detector. In a British setting, non-ownership of a functioning smoke detector is associated with living in a property in poor physical shape, in a household with a smoker, or in a financially unstable household.⁵ In one study, smoke detectors were found to be absent in 20% of households with children aged under 15 years—despite children being at particular risk of fire related injury or death.^3,6

Research elsewhere, including New Zealand and Canada, has also shown an association between smoke detector use and socioeconomic status.⁷ The US has the highest incidence of burn hospitalisation in the developed world with evidence of stark differentials in smoke detector use by socioeconomic, ethnicity, and other variables.⁸

Work on the reasons for the non-installation of functioning smoke detectors includes a survey conducted in Cleveland (England), which showed that, among respondents who said that they would probably/definitely not buy a detector (n = 361) 21% did not feel at risk, 19% had “… never got round to it”, 18% said that they were “… careful about fire safety”, 13% said that there were no smokers in the household, and 7% said that they could not afford one.⁹

A prominent theme in the research on smoke detectors is their deactivation. False alarms (typically activated by the smoke associated with cooking) are sufficiently common to warrant the removal of batteries,³ and there may be up to 16 nuisance alarms for every “true” alarm.¹⁰

Despite the effects of nuisance alarms, smoke detector use overall is associated with public health benefits. In this paper, we report findings from an exploratory study funded by the Health Development Agency.¹¹ The aim was to map the evidence on the implementation of interventions to reduce accidental injuries, and to explore ways in which this evidence can be synthesised in a manner accessible to policymakers and practitioners. The work focused on two types of interventions: residential smoke detector promotion programmes and community based injury reduction programmes.

The focus of this paper is the implementation of initiatives promoting the installation and use of smoke detectors.

METHODS

The work comprised three stages. Firstly, we obtained papers reporting on the primary studies in a Cochrane systematic review (SR) on the effectiveness of residential smoke detectors.¹² Secondly, the authors of these papers were asked if they had additional data on implementation. We also sought papers describing smoke detector promotion initiatives not reported in the SR. We were particularly interested in initiatives that had used a qualitative design as part of the evaluation, on the grounds that these were more likely to have addressed implementation.¹³

The papers retrieved were read for reference to data on programme implementation. From the systematic review, the approach to researchers, and the search of the wider literature, we identified 24 evaluative studies of smoke detector promotion initiatives and retrieved 37 papers and/or other documents reporting on these. Twenty six reported on studies focusing primarily on smoke detector programmes. The remaining 11 studies of broader community based injury reduction programmes are reported elsewhere and are not included here.

In the third stage, the material we identified was filtered to: (1) assess the relevance of the work to the review; (2) assess the methodological quality of the studies, with a particular focus on studies using qualitative methods; and (3) attempt a synthesis of the findings from these studies on implementation processes. We developed a set of criteria to distinguish between research providing a “thicker” description of implementation processes compared with those providing a “thinner” description. These criteria, and the process by which they were developed, are described elsewhere.¹¹

Below we illustrate the kinds of evidence on implementation contained in these papers, and some of its limitations.

RESULTS: EVIDENCE ON IMPLEMENTATION

In general, papers that contained a “thicker” or fuller description of implementation processes have features which relate both to the nature of the intervention, and to the social context in which the intervention is implemented. Examples of “thicker” descriptions will be considered under the following four headings: (1) a relatively detailed description of the intervention and its suitability for the targeted population; (2) a recognition of the steps between the design of an intervention and its implementation in an everyday setting; (3) a description of the specific factors that affect implementation; (4) an exploration of the reasons for anomalous results.

1. The description of the intervention and its suitability for the targeted population is a consistent feature of many “thicker” descriptions of implementation processes. In the case of educational messages, for example, authors point to the need for a strong, central theme if the intervention is to be effective.¹⁴

Describing a safety training program given to new heads of households, McConnell et al emphasise that: “… every aspect of the intervention was oriented towards, and tailored precisely for, the targeted population and the environment in which they live” (p ).¹⁵ A video used in the program featured real tenants and was set in typical Housing Authority property.

Jenkins et al describe the cultural appropriateness of their intervention, an educational booklet randomly assigned to an intervention group consisting of caregivers of children discharged from a Canadian burns unit who were disproportionately of North American Indian ethnicity. The interviewer who delivered the intervention was familiar with their culture, and the booklet was designed with those communities in mind.¹⁶

2. “Thicker” descriptions of implementation processes are frequently characterised by detailed attention to the intervention as well as recognition of the gap between the design of an intervention and its implementation. Carlson Gielen and colleagues, for example, maintain that one of the goals of their project was to “… develop interventions that could be implemented in real-world settings”. The process of collecting data was not always straightforward and: “… there were many occasions when (the checklist) could not be completed”, due to problems with “… fractious infants, and other parent distractions …” (pp 38–39).¹⁷

3. In “thicker” descriptions there were more references to specific factors affecting implementation.^18–20 One frequently mentioned factor was the active involvement of key local figures. McLoughlin and colleagues maintain that the effective implementation of their program required: “… the creation of a network of local people who would make presentations in a variety of settings …” Community leaders were asked to help with the campaign “… [they] were most active and cooperative when their political standing and job status were enhanced by participation in the project”(p 29).²¹

Similarly, Jones et al attribute the success of their programme in a poor, Black US community to good rapport between research staff and residents and effective partnership working.²²

In their description of the “Let’s Get Alarmed” initiative, DiGiuseppi et al observe that barriers to participation included residents’ “mistrust of local government initiatives” and “community mistrust of strangers at the door” (p ).²³ These observations provide clues about the factors that help or hinder the implementation of programs aimed at increasing functioning smoke detectors.

4. “Thicker” descriptions of implementation processes often include an exploration of the reasons for anomalous results and findings. Jones et al, for example, explain discrepancies between householders’ and observers’ reports of smoke detector ownership: “Respondents may have reported that there was a smoke alarm installed because they perceived that the interviewers were implying that these devices should be in every home. … Interviewers reported that several respondents neither knew the name nor the purpose of the device prior to the visit by the Get-Alarmed Campaign staff” (p ).²²

Finally, “thicker” descriptions of implementation often contain some consideration of the broader social context within which the trials take place.

Qualitative research evidence and implementation

Although we found that few trials used both qualitative and qualitative designs, we identified two rather different examples of the insights that mixed method studies can bring to implementation. These studies took place in New South Wales,^24,25 and London,²⁶ and questionnaires and focus groups were used to identify barriers and levers to the use of smoke detectors.

The results from Australia suggested that barriers to purchasing smoke detectors were: (1) a lack of awareness and (2) living in rental properties where the landlord was believed to be unsympathetic to the need for detectors. Findings from focus groups suggested that respondents overestimated the cost of smoke detectors and did not see fire as a sufficiently serious risk. In the UK study, there was considerable awareness of the need. Cost was not an issue because detectors were offered free of charge, however the nuisance factor of smoke alarms was seen to outweigh the more immediate safety issues.

A paper reporting on the Australian study provides insights for practitioners implementing smoke detector campaigns.²⁴ In this study, television advertising was cut because of “issues relating to the tender for the sale of smoke alarms” (p ).²⁴ Bilingual campaign workers were able to “negotiate discounted rates for … print advertisements” (p ).²⁴ Pamphlets promoting the use of smoke detectors were more likely to be left behind after community festivals than during sales or after community talks.

SUMMARY AND IMPLICATIONS

This paper reports the findings from an exploratory study of the type of evidence on implementation processes found in studies of smoke detector promotion initiatives. We assigned papers to either “thicker” or “thinner” descriptions of implementation processes. The former were characterised by: a relatively detailed description of the intervention; recognition of the discrepancy between the design of an intervention and its implementation; some consideration of the broader social context within which the trial takes place; a description of the specific factors that affect implementation; and an exploration of the reasons for anomalous results.

Despite our search for further details about implementation processes, we consider that the implementation data we retrieved were insufficiently strong to provide a good evidence base for practitioners and policy makers.

There are systemic deficiencies in the literature in reporting context, methods, and details of implementation. This may be due in part to the nature and mission of scientific journals, and what “counts” as science in a publication.

Although we have used smoke detectors as a case study, implementation issues clearly have a relevance to all intervention studies, trials, and systematic reviews. Dissemination of study findings is only the first step to getting them used in (or where appropriate, withdrawn from) practice. Information on implementation is the next step.

This work has implications for researchers, systematic reviewers interested in synthesis, funders, and journal editors:

• Researchers with an interest in evidence based public health could be encouraged to consider implementation issues in research design.

• Funders could be encouraged to consider programs of intervention studies using mixed methods which will enable researchers to consider effectiveness and implementation.

• Some journal editors have already increased the quality, not only of research reporting, but also of research conduct through guidelines on presentation and content. There is scope for further work on the encouragement of good research reporting on implementation issues. This might include web based technical appendices, summaries for service planners, and sections of journals (or whole journals) devoted to implementation issues. Some of this would involve a shift in attitude to what gets published. There are disincentives to scientists reporting frankly on “what goes wrong”, despite the fact that this is a core part of our understanding of what might go right in future.

The above are not entirely straightforward—partly because they touch on the nature of science at a time when many academics are judged by the number and quality of articles in peer reviewed journals rather than the extent to which they reach out to research users.

Although academics increasingly tender for evaluation projects, there can be a tension between program evaluation and “research”, with the former almost exclusively addressing process while the latter is more inclined to concentrate on outcomes. These tensions affect the design, funding, reporting, and publication of injury (and other public health) research. Funders who ask for “user” involvement in applications have made an important step. The next step is to insure that “user” insights into barriers and levers to implementation, as well as other contextual information, go beyond the statements made when funding is approved and are collected carefully and well reported.