Since the Framework Convention on Tobacco Control (FCTC) of the World Health Organization (WHO) became binding international law in February 2005, it has already generated a large number of policy initiatives, which public health researchers have an obligation to ensure are as effective as possible. Towards this end, a working group formed by 26 scientists from nine countries and the WHO, under the joint chairmanship of Ron Borland (Australia) and Mike Cummings (USA), met at the International Agency for Research on Cancer (IARC) in Lyon, France, in March this year to outline a common conceptual framework for the evaluation of FCTC tobacco control policies, to guide assessments about what policies work and why they work.

The result of their work, to be published next year as the 12th handbook in the IARC Handbooks of Cancer Prevention series, will provide a theory driven framework that IARC plans to apply in the evaluation of specific tobacco control policy interventions.

As policies and the disseminated programmes that result from policy decisions can affect large numbers of people, in some cases the entire population, it is important to determine whether they achieve their objectives, and do so in a cost effective way with any incidental effects ideally having net benefits.

Four broad questions were considered, each with several more specific related questions, to guide the review of the scientific literature on the methods and measures of tobacco policy evaluation. The broad questions cover how the effects of a policy are determined; the core constructs for understanding how and why a given policy works; the potential moderator variables to consider when evaluating a given policy; and the data sources that might be useful for evaluation.

The working group proposed a common conceptual framework to guide future FCTC policy evaluation, specifying two levels of mediating variables: those specific to the policy, and those that are part of more general pathways to the outcomes of interest. It also accepted that various other factors (moderators) might affect the size of the effect, and recognised the possibility of effects incidental to those an intervention is designed to produce. The group concluded that tobacco use was the appropriate end point for most policy evaluations because both prevention and cessation of tobacco use reduce tobacco related deaths and illnesses. The group elaborated the model most completely for tobacco use outcomes, but it was also applied to policies affecting product harmfulness.

For policies designed to affect tobacco use, policy specific mediators involve such elements such as awareness, policy specific knowledge and reactions to specific elements of the intervention. For example, new graphic warning labels are predicted to increase the salience and visibility of warnings, and perhaps forgoing of occasional cigarettes. General mediators are constructs taken from behavioural science theory that we know mediate the effects of such things as knowledge and awareness on behaviour, in this case tobacco use. They include attitudes, normative beliefs and intentions.

Moderator variables include those factors that change the magnitude of the effects of an intervention without necessarily being changed by the intervention. They often include sociodemographic characteristics (for example, age, gender, socioeconomic status, cultural background) and psychological factors that are either assumed to be stable or which the intervention is not designed to change (for example, level of dependence).

The handbook will include logic models outlining relevant constructs for evaluating the effectiveness of policies on tobacco taxation, smoke-free environments, tobacco product regulations, limits on tobacco marketing communications, product labelling, anti-tobacco public communication campaigns and tobacco use cessation interventions. Additionally, it will provide examples of measures used to assess key constructs, with special attention to measurement issues with survey methods.

It will also include descriptions of sources of data on tobacco control policies, tobacco production and trade, and repositories of youth and adult surveillance surveys. These sources of information are particularly important for making comparisons between countries, and in some cases can be used to demonstrate the impact of policies, although not the mechanisms by which they occur. Lastly, the volume will present the terminology to judge the quality of the evidence considered in the evaluation of tobacco control policies and to be used in future IARC handbook publications.

The FCTC has propelled tobacco control into a new era as countries all over the world incorporate its policies and recommendations into their own laws. As tobacco control policies are formulated and implemented, it is important that they undergo rigorous evaluation. In the same way that evidence based medicine has been built from rigorous evaluation of treatment options, evidence based public health must build on a database of rigorous evaluations of public health policies. Such knowledge will allow implementation of the most powerful policy interventions, and will do so in ways that will maximise their effectiveness.