You are here

Article Text

Article menu
Review
Pictorial cigarette pack warnings: a meta-analysis of experimental studies
  1. Seth M Noar1,2,
  2. Marissa G Hall3,
  3. Diane B Francis1,
  4. Kurt M Ribisl2,3,
  5. Jessica K Pepper2,3,
  6. Noel T Brewer2,3
  1. 1School of Journalism and Mass Communication, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
  2. 2Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
  3. 3Department of Health Behavior, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
  1. Correspondence to Dr Seth M Noar, School of Journalism and Mass Communication, 382 Carroll Hall (CB 3365), University of North Carolina at Chapel Hill, Chapel Hill, NC 27599-3365, USA; noar{at}email.unc.edu

Abstract

Objective To inform international research and policy, we conducted a meta-analysis of the experimental literature on pictorial cigarette pack warnings.

Data sources We systematically searched 7 computerised databases in April 2013 using several search terms. We also searched reference lists of relevant articles.

Study selection We included studies that used an experimental protocol to test cigarette pack warnings and reported data on both pictorial and text-only conditions. 37 studies with data on 48 independent samples (N=33 613) met criteria.

Data extraction and synthesis Two independent coders coded all study characteristics. Effect sizes were computed from data extracted from study reports and were combined using random effects meta-analytic procedures.

Results Pictorial warnings were more effective than text-only warnings for 12 of 17 effectiveness outcomes (all p<0.05). Relative to text-only warnings, pictorial warnings (1) attracted and held attention better; (2) garnered stronger cognitive and emotional reactions; (3) elicited more negative pack attitudes and negative smoking attitudes and (4) more effectively increased intentions to not start smoking and to quit smoking. Participants also perceived pictorial warnings as being more effective than text-only warnings across all 8 perceived effectiveness outcomes.

Conclusions The evidence from this international body of literature supports pictorial cigarette pack warnings as more effective than text-only warnings. Gaps in the literature include a lack of assessment of smoking behaviour and a dearth of theory-based research on how warnings exert their effects.

  • Packaging and Labelling
  • Public policy
  • Global health

This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/

https://doi.org/10.1136/tobaccocontrol-2014-051978

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Tobacco use is the leading cause of preventable death and disease in the world, causing nearly six million deaths each year.1 While tobacco product packaging is a key part of marketing efforts to make tobacco use appealing,2 ,3 regulators can use that same packaging to communicate the health risks of tobacco products to consumers.4 A pack-a-day smoker potentially sees a cigarette pack an estimated 7300 times per year (20 views/day×365 days/year). Messages on these packs would generate exposure far outweighing exposure from other antitobacco communications, such as mass media campaigns.5

    The combination of high exposure, nearly universal reach, and very low cost has made pictorial warnings on cigarette packs a core tobacco control strategy globally. The WHO Framework Convention on Tobacco Control (FCTC) calls for the implementation of large warnings on tobacco products.6 The treaty's Article 11 specifies that health warnings may include pictures, and subsequent guidelines for implementation state that pictorial warnings are ‘far more effective’ than text-only messages.6 By 2015, implementation of pictorial warning policies had occurred in 77 countries and jurisdictions that are home to nearly 50% of the world's population.7

    As pictorial cigarette pack warnings have proliferated globally, so has research on their impact.8 ,9 Observational studies suggest increased cessation behaviour after the introduction of pictorial warnings,10 ,11 and such studies typically have high external validity. However, isolating the effects of pictorial warnings on smoking behaviour in such studies has proven difficult because governments often introduce the warnings alongside other tobacco control policies.8 ,12 By contrast, experiments can offer strong evidence of the causal impact of pictorial warnings, isolating the effects of warnings on key outcomes. For this reason, experiments are an important tool for studying the effects of pictorial warnings.

    Previous research on pictorial cigarette pack warnings

    A large and growing empirical literature has documented the effects of pictorial cigarette pack warnings. Some evidence suggests that pictures and imagery may be more effective than text-only messages at communicating health risks.13 ,14 Compared with text-only warnings, pictorial warnings have been associated with stronger beliefs about the harms of smoking and higher motivation to quit smoking.10 ,15–21 However, while some studies find that smokers and non-smokers rate pictorial warnings as more effective than text-only warnings,22–26 other studies have reported conflicting findings.27–29 For instance, studies have found that graphic, pictorial warnings result in poorer recall than less graphic or non-graphic warnings,28 do not increase youth's expectations to be non-smokers a year later,29 have no effect on beliefs about cancer or addiction among non-smoking adolescent boys,26 and are effective in lowering smoking intentions for Canadians but not for Americans.27

    Reviews of the literature on pictorial cigarette pack warnings have taken a variety of approaches. A narrative review by Hammond8 suggested that cigarette pack warnings can be effective in promoting smoking cessation, especially when warnings are large, full-colour, and use graphic images. While useful and an important contribution for understanding pictorial warnings, this review did not provide a systematic, quantitative synthesis of pictorial warning effects. A systematic review by Monarrez-Espino et al30 examined 21 mostly observational studies of the impact of pictorial warnings on reduced smoking, quit attempts and smoking cessation. Monarrez-Espino et al found that most of these studies were of poor methodological quality; for this reason, their findings on the impact of pictorial warnings on smoking behaviour were inconclusive. Importantly, this review did not examine many factors that are likely pre-requisites to changes in behaviour, such as attention to warnings, cognitive and emotional reactions to warnings, and changes in beliefs about smoking.

    While these recent reviews have summarised portions of the cigarette pack warnings literature,8 ,9 ,30 no meta-analysis has synthesised the experimental literature on pictorial cigarette pack warnings. To inform international research and public policy, we conducted a meta-analysis of experiments examining the impact of pictorial cigarette pack warnings. Our research question was: across the body of experimental studies, what are the effects of pictorial cigarette pack warnings compared with text warnings?

    Method

    Search strategy

    We used a comprehensive search strategy to locate studies relevant to this meta-analysis. The search strategy involved three steps. First, we searched PsycINFO, PubMed, EMBASE, Web of Science, Communication & Mass Media Complete, Business Source Complete, and CINAHL computerised databases in April 2013. We used the following Boolean terms: (cigarette* OR tobacco) AND (warning* OR label* OR pictorial OR graphic OR messag* OR text*). Second, we examined the reference sections of five narrative reviews of cigarette pack warnings.8 ,9 ,31–33 Third, we examined the reference lists of the final set of articles included in our review. We included all reports that came up in our searches—peer-reviewed journal articles, book chapters, and grey literature (eg, dissertations, publicly available reports)—as long as the full text was available.

    To be included, a study had to use an experimental protocol that tested warnings intended for cigarette packs. Studies had to report data on both a pictorial warning condition and a text-only condition. The experimental design could be between subjects (individuals were randomised to different warning label manipulation conditions—eg, text vs pictorial) or within subjects (individuals viewed multiple warning label manipulations). We excluded studies of non-cigarette tobacco products, public service announcements or multicomponent interventions, and warnings embedded in cigarette advertising. We excluded observational studies that asked individuals to report on warnings that they had seen on their own prior to the study. Finally, articles reporting the studies had to be available in English.

    Figure 1 depicts the search process. The initial database search yielded 14 139 total references, and searching through the other methods yielded 424 references. After removing duplicates, there were 8486 references. Two reviewers independently examined all study titles for relevance, reducing the number to 497, and then reviewed abstracts, further reducing the number to 98. During this process, we excluded articles only if both reviewers independently determined the article as irrelevant. Then, the two reviewers independently examined the full text of the 98 articles and tracked reasons for study exclusion. If the two reviewers made a different determination about the classification of a particular article, they consulted with a third referee to resolve the discrepancy and make a final determination. This process resulted in a total of 35 articles reporting on 37 studies. Since some studies reported results separately for different subgroups, we analysed effect sizes for each independent sample. Thus, the meta-analysis synthesised effects of 48 independent samples.

    Figure 1
    Figure 1

    PRISMA flow diagram showing the study screening process.

    Article coding

    Coding study characteristics

    Two independent coders coded all articles on several features, including participant characteristics such as gender, age, race/ethnicity and country of origin, and study characteristics such as within-subject/between-subject design and use of theory. The coders also coded warning characteristics: warning type (pictorial, text), nature of pictorial labels (image only, image with text), whether pictorial text and control text matched, number of different labels viewed, number of times viewing each label, number of exposure sessions, exposure medium (warning only, warning on two-dimensional pack, warning on three-dimensional pack), exposure channel (digital, printed or paper, cigarette pack), exposure control (researcher-controlled exposure, participant-controlled exposure), and label order (random, non-random).

    The coders and the first author met to discuss each article after it was coded to compare the two coders’ work. All discrepancies between coders were resolved through discussion between the two coders and the first author. We calculated intercoder reliability for each characteristic. Most categories had perfect agreement, and the mean per cent agreement was 96%. Cohen's κ34 had a mean of 0.94.

    Coding dependent variables

    We developed a list of more than 30 dependent variables assessed in the studies based on an initial review of the literature. We then grouped these outcome variables into theory-based construct categories. Table 1 lists the constructs that at least two studies assessed, along with our definition of the construct, an example item from a study in the meta-analysis, and examples of the authors’ original terminology. We grouped all constructs into five categories. The first group (attention and recall) assessed participants’ attention to warnings and ability to recognise or recall the warnings. The second group (warning reactions) assessed participants’ cognitive, emotional and physiological reactions to warnings. The third group (attitudes and beliefs) assessed participants’ smoking or cigarette pack-related attitudes and beliefs. The fourth group (intentions) assessed participants’ intentions or willingness to act. Finally, the fifth group (perceived effectiveness) assessed participants’ perceptions of the effectiveness of warning messages.

    View this table:
    Table 1

    Outcomes assessed in experimental pictorial warning studies

    We organised these five groups of constructs into a message impact framework (figure 2), which is based on communication and psychological theory46–52 and previous tobacco warnings theory and research.29 ,53–55 The framework suggests that the characteristics of a warning affect the extent to which the warning will be noticed and later recalled, and that attention to (and recall of) the warning influences warning reactions. Warning reactions are thought to, in turn, affect attitudes/beliefs, which later influence intentions and ultimately behaviour. Given that cigarette pack warnings are often in public view, they may spark interpersonal communication and social interactions.49 These social interactions, such as talking about the warnings with friends and family members, may affect individuals’ attitudes, beliefs, and reactions to the warnings.

    Figure 2
    Figure 2

    Message impact framework applied to research on cigarette pack warnings.

    Perceived effectiveness is also pictured in figure 2, and such ratings are commonly used in formative work to develop and assess messages.56 ,57 However, currently, there is no evidence to suggest that these ratings play a direct role in warning message effects (ie, that participants must perceive a message to be effective in order for it to be so). Thus, perceived effectiveness is not pictured as an integral part of this framework.

    Effect-size extraction and calculation

    We characterised the effect size of the benefit of pictorial over text warnings by using the standardised mean difference statistic d (ie, the difference in treatment and control means divided by the pooled SD).58 Because d can be upwardly biased when based on small sample sizes,59 we applied the recommended statistical correction for this bias.58 We calculated effect sizes from data reported in the article (eg, means and SDs; frequencies) using standard formulas.58 For within-subject designs, using statistics such as t and F for effect-size computation can bias effect-size estimates.60 However, using raw statistics such as means and SDs does not yield this bias.60 ,61 Thus, we applied conventional formulas58 and computed all within-subject effect sizes from raw (vs inferential) statistics. If the article did not provide data necessary for effect-size computation, we requested the necessary data from authors.

    We computed effect sizes for outcomes that were (1) identified as a meaningful construct from the communication or psychological literature and (2) assessed in two or more studies. When studies reported multiple pictorial warning or text-only conditions, we averaged these (text or pictorial) conditions together when computing effects. When studies reported more than one measure of the same variable (eg, two measures of negative smoking attitudes), we averaged them together. In order to keep effect sizes consistent and interpretable, we gave a positive sign (+) to effect sizes in which the pictorial warning condition performed better (ie, yielded a finding conducive to behavioural change) than the text-only condition, and a negative sign (−) to effect sizes in which the pictorial warning condition performed worse than the text-only condition.

    Meta-analytic approach

    Analyses weighted effect sizes by their inverse variance and combined them using random effects meta-analytic procedures.58 We calculated the Q statistic and I2 to examine whether heterogeneity existed among the effect sizes. Most dependent variables had too few studies to perform moderator analyses. As some form of perceived effectiveness for motivation to not smoke was commonly assessed, we created a composite variable to use in moderator analyses. This composite variable consisted of all relevant perceived effectiveness motivation variables (ie, the first 5 constructs listed in table 4), and assessed the extent to which participants perceived pictorial warnings as motivating smokers or non-smokers to avoid smoking cigarettes. For the seven studies that measured a motivation to avoid cigarette use construct in multiple ways, we averaged together the effect sizes for all relevant outcomes. We performed moderator analyses on this variable using mixed-effects analyses, which allowed for the possibility of differing variances across subgroups.58 We calculated effect sizes for hypothesised categorical moderators along with their 95% CIs, and we statistically compared those effect sizes using the Qb statistic. We also examined correlations between continuous moderator variables and effect size. We conducted all analyses using Comprehensive Meta-Analysis software V.2.2.046 and SPSS V.21.

    Results

    Study characteristics

    The 37 studies were conducted in 16 different countries, with the most conducted in the USA (43%), followed by Canada (11%) and Germany (11%) (see online supplementary file).23–25 ,27–29 ,35–45 ,62–79 While studies were published as early as 2000, most studies (68%) were published between 2009 and 2013. Fifty per cent of study samples included both smokers and non-smokers, 47% were smokers only, and one study was of non-smokers only. Most studies (65%) included both young adults and adults but not adolescents. Eleven studies (29%) included adolescents in their sample, although only four studies (11%) focused solely on adolescents. Study sample sizes ranged from 25 to 4890 (median=197), and the cumulative sample size across all studies was 33 613. Nineteen of 37 studies (51%) mentioned a theory as informing the study.

    Studies varied considerably in how many different warnings they showed to participants (mean number of pictorial warnings=6.39, SD=10.86; mean number of text warnings=5.24, SD=10.91). However, in most studies, participants viewed a particular warning only once (86%), and they participated in only one viewing session (97%; table 2). In all but one study, participants were assessed only immediately after viewing the warning labels. The most commonly used exposure medium for warnings (57%) was a two-dimensional pack displayed on a computer with the participant controlling the duration of the exposure to the warning (ie, how long they viewed the warning before advancing further in the survey). Most pictorial warnings (89%) included both images and text, though some (8%) consisted of images only. In many cases (43%), the text in the pictorial warning matched the text presented in the comparison condition, though in several cases the text differed (43%).

    View this table:
    Table 2

    Characteristics of warning manipulations in studies in the meta-analysis

    Studies assessed more than 30 unique constructs (see online supplementary file). Each individual study assessed between one and eight constructs (M=2.75, SD=1.96). We identified 25 constructs that appeared in at least two studies, and these constructs are the focus of the meta-analysis (table 1).

    Effectiveness of pictorial warnings

    Pictorial warnings exhibited statistically significant effects relative to text warnings for 13 of 17 effectiveness outcomes (most at p<0.001; see table 3), with 12 of 17 effects favouring pictorial warnings. Compared with text-only warnings, pictorial warnings showed an advantage for two of four attention constructs (figure 3), with pictorial warnings scoring higher on both attention attracting (d=0.79) and attention duration (d=1.74). We observed no effects on response time or recall/recognition of warning text.

    View this table:
    Table 3

    Effectiveness of pictorial warnings: mean weighted effect sizes (d) and heterogeneity statistics

    Figure 3
    Figure 3

    Forest plot displaying effect sizes and 95% CIs for attention outcomes.

    For warning reactions, pictorial warnings showed an advantage for five of six constructs (figure 4). Relative to text warnings, pictorial warnings elicited more cognitive elaboration (d=1.70), negative affective reactions (d=0.54), credibility (d=0.15), lower smoking cravings (d=0.08), and aversiveness (d=0.58). However, pictorial warnings also elicited greater psychological reactance (d=−0.50).

    Figure 4
    Figure 4

    Forest plot displaying effect sizes and 95% CIs for warning reactions.

    Pictorial warnings showed an advantage on two of four attitude and belief constructs (figure 5), with effects on both negative pack/brand attitudes (d=0.79) and negative smoking attitudes (d=0.55) relative to text warnings. No effects were observed on perceived likelihood of harm (d=0.02) or self-efficacy to quit (d=0.01). Moreover, pictorial warnings showed an advantage on all three intentions constructs (figure 6), with effects on lower willingness to pay (d=0.26), intention to not start smoking (d=1.82), and intention to quit smoking (d=0.54).

    Figure 5
    Figure 5

    Forest plot displaying effect sizes and 95% CIs for attitudes/beliefs.

    Figure 6
    Figure 6

    Forest plot displaying effect sizes and 95% CIs for intentions.

    Homogeneity analyses indicated that 9 of 17 effect sizes were heterogeneous, with many outcomes exhibiting extremely high heterogeneity: 6 of these 9 outcomes had an I2 of greater than 90 (table 3).

    Perceived effectiveness of pictorial warnings

    Pictorial warnings exhibited statistically significant effects relative to text warnings for all eight perceived effectiveness outcomes (see table 4 and figure 7). Pictorial warnings were more likely to be rated as effective in motivating not starting smoking (d=1.03), motivating reducing smoking (d=0.41), motivating themselves (d=0.79) or others (d=1.09) to quit smoking, and motivating (smokers or non-smokers) to not smoke (d=0.24). Participants also perceived pictorial warnings as deterrents to giving cigarettes as a gift (d=1.64), as generally effective (d=1.00), and as effective for themselves and others (d=0.52).

    View this table:
    Table 4

    Perceived effectiveness of pictorial warnings: mean weighted effect sizes (d) and heterogeneity statistics

    Figure 7
    Figure 7

    Forest plot displaying effect sizes and 95% CIs for perceived effectiveness.

    Moderator analyses

    The weighted mean effect size for the composite variable motivation to avoid cigarette use was statistically significant (p<0.001) at d=0.95 (CI 0.56 to 1.34, k=15, cumulative n=13 023). This effect was statistically heterogeneous, Q=1310, p<0.001, I2=99. Moderation analyses found that studies using a within-subject design (k=7) differed from those using a between-subject design (k=8; Qb=7.50, p<0.01; table 5). Studies using within-subject designs (d=1.37) had larger effect sizes than those using between-subject designs (d=0.51). Statistical comparisons of samples of smokers (k=9) to non-smokers and mixed samples (k=6) did not reach statistical significance (p=0.07). The trend, however, suggested that non-smokers and mixed samples (d=1.39), rated warnings as being more effective than did smokers (d=0.65). Analyses of exposure medium (two-dimensional vs three-dimensional pack), exposure channel (digital vs paper or pack), and country of sample (USA vs other countries) found no differences. Effect sizes were also not significantly correlated with gender composition, r (14)=−0.02 (p=0.98) or age, r(14)=−0.32 (p=0.49).

    View this table:
    Table 5

    Moderators of perceived motivation to avoid cigarette use

    Discussion

    The purpose of this meta-analysis was to expand our understanding of the impact of pictorial cigarette pack warnings on smoking-related outcomes. Across an international body of experimental studies, we found effects favouring pictorial warnings for 12 of 17 effectiveness outcomes. Compared with text warnings, pictorial warnings (1) attracted and held attention better; (2) garnered stronger cognitive and emotional reactions; (3) elicited more negative pack attitudes and negative smoking attitudes; and (4) more effectively increased intentions to not start smoking and to quit smoking. These findings suggest that pictorial warnings are superior to text warnings at multiple stages of our message impact framework (figure 8) and may move people towards quitting smoking. While a recent systematic review did not find evidence that pictorial warnings were effective,30 that review examined only smoking behaviour and included mostly observational studies. The experimental studies we examined here showed promising evidence of effects.

    Figure 8
    Figure 8

    Effects of pictorial warnings on cigarette packs (summary of findings).

    In our meta-analysis, it is especially noteworthy that pictorial warnings changed negative smoking attitudes and quit intentions, as these variables are associated with quitting behaviour. For example, a previous meta-analysis of eight longitudinal studies found that negative smoking attitudes and quit intentions predicted subsequent quit attempts.80 Our review also demonstrated that pictorial warnings increased cognitive elaboration more than text-only warnings. Cognitive elaboration may play a particularly important role in the quitting process. A recent longitudinal study found that increased attention to cigarette pack warnings led to greater cognitive elaboration, which ultimately predicted quit attempts (via mediation through worry and quit intentions).81

    In our review, pictorial warnings were also superior to text warnings on all eight perceived effectiveness outcomes. Smokers and non-smokers rated pictorial warnings as more effective than text warnings at motivating not starting, reducing and quitting smoking. These findings are noteworthy, as research has suggested that messages with higher perceived effectiveness ratings may be more effective than those with lower ratings.82–84 Taken together, these findings on effectiveness (eg, increased quit intentions) and perceived effectiveness (eg, increased perception that warnings motivate quitting) offer strong evidence to support pictorial cigarette pack warnings as more effective than text-only warnings.

    Mediators of pictorial warning effects

    Our meta-analysis provides support for the notion that pictorial warnings elicit changes in an array of psychosocial constructs that are plausible mediators of the warning-behaviour link. Future studies should identify constructs that mediate pictorial warnings’ effects on smoking behaviour. The potential mediator most proximal to behaviour is intentions, one of the strongest predictors of behaviour according to both theory85 and empirical research.80

    We saw the effects for many beliefs and attitudes that are plausible mediators according to theories of health behaviour.50 ,85 ,86 In our meta-analysis, pictorial warnings elicited greater fear-oriented reactions than text warnings, as intended. This is consistent with previous research and theory on fear appeals, which has found that such appeals increase fear as a mechanism for attitude, intention and behavioural change.87 However, fear appeal theories, such as the extended parallel process model (EPPM),52 also suggest that two key constructs help explain how people respond to fear appeals—perceived threat and efficacy. On these two key constructs, we found no effects of pictorial warnings. It was surprising that only five studies (with 8 effect sizes) in the meta-analysis assessed perceived likelihood of harm, a component of perceived threat, when much theorising situates this as a central construct in fear appeals and risk communication.52 ,88 ,89 While our meta-analysis did not find an effect of pictorial warnings on perceived likelihood of harm, the reason is unclear. It may be due to inadequate perceived likelihood measures or a failure to change risk beliefs because of an inadequate dose of warning exposure. It is also important to note that the lone study that assessed perceived severity of harm, another component of perceived threat, found a large effect.36 More careful studies of the impact of pictorial warnings on perceived likelihood and severity are required before we can make stronger conclusions regarding the role of risk beliefs in warning effectiveness.

    Moreover, considering the potential importance of self-efficacy in predicting how people respond to health messages, pictorial warnings may be more effective if they increase self-efficacy to quit smoking. However, only two studies in this meta-analysis measured self-efficacy, and none manipulated it experimentally. Several countries, including Brazil, Australia and New Zealand, require that pictorial warnings provide information about cessation services, which may be a promising strategy for increasing smokers’ self-efficacy to quit.90 ,91 Future pictorial warning studies should examine the role of self-efficacy in predicting changes in intentions and behaviour, and the interaction of efficacy and threat, testing hypotheses from the EPPM.52

    Previous fear appeal research and theory also suggest that fear-oriented communications can elicit reactance.87 While we found that pictorial warnings elicited greater reactance than text warnings, the studies in our meta-analysis focused on the emotional aspect of reactance, and largely ignored the cognitive elements. Research characterises reactance as a construct comprised of both emotion (eg, anger, irritation) and cognition (eg, defensive processing, denial).92 Future studies on pictorial warnings should advance a more comprehensive measurement approach and should examine whether reactance leads to adverse outcomes, such as lower quit intentions or greater smoking behaviour.

    Finally, our meta-analysis found pictorial warnings increased aversiveness (ie, the warnings being ‘difficult to look at’). How aversiveness plays into effects of such warnings is unclear. For example, actions such as looking away from, covering up, or avoiding the warnings may reduce the warnings’ effects; alternatively, these behaviours may actually be markers of the warnings’ effectiveness. One observational study revealed that Canadian smokers who avoided pictorial warnings were equally likely to think about the warnings or engage in cessation behaviour than those who did not attempt to avoid the warnings.93 Further research is needed to understand how aversiveness affects pictorial warning effectiveness.

    Theoretical and measurement issues

    Despite the existence of several models to guide warnings research,53–55 our meta-analysis revealed a lack of consensus as to what outcomes experimental studies should assess. Outcomes varied widely, with little consistency across studies. The framework presented in this article (figure 2), along with our empirical findings (figure 8), may help bring theoretical clarity to the literature. In particular, we recommend that researchers pay particular attention to issues of construct validity, taking care to explicitly describe what they measure and how they measure it, and ensuring that the name accurately matches what the measure is assessing. Researchers should also carefully consider what types of constructs and measures are most appropriate for their study given the stage of warnings research in a given country, and the goals of the particular study.

    Study design

    Our meta-analysis included both experiments that manipulated pictorial warnings between subjects (participants viewed only text or only pictorial warnings) and within subjects (participants viewed both warning types). The between-subject studies had markedly smaller effect sizes for perceived effectiveness for motivation to avoid cigarette use than within-subject studies. Why was this the case? One explanation may be a reference point effect.94 That is, when one evaluates a text warning followed by a pictorial warning, the pictorial warning seems that much more powerful. In that case, the participant may rate the pictorial warning higher than would have been the case otherwise.95 Another possibility is that seeing multiple warnings makes it easier to focus on the presence or absence of graphics, the central attribute that differs. This is similar to Hsee's evaluability hypothesis that explains what happens when an important attribute is hard to evaluate independently.96 In the real world, participants are likely to see only text or pictorial warnings on cigarette packs, and so it may be that between-subject studies provide a more accurate estimate of effect size of that difference. However, comparisons among different pictorial warnings may best be done using within-subject studies, as people will likely see many such warnings on packs.

    A final study design issue is that no study in this meta-analysis tested warnings by placing them on smokers’ cigarette packs. Instead, participants had only brief exposure to warnings, often on a computer screen. By contrast, the large body of observational literature examines smokers who have had multiple exposures to warnings on their cigarette packs.8 Such studies are invaluable, as they can potentially demonstrate population-level effects that result from warning policy changes. However, internal validity threats make strong causal conclusions from such studies difficult, especially on outcomes such as smoking behaviour.30 Therefore, we recommend that future experimental studies place warnings on smokers’ cigarette packs, follow participants over time, and assess smoking behaviour as an outcome.97–99

    Gaps and future directions

    This review identified several areas for future research. First, our meta-analysis identified only a single experimental study that assessed behaviour.69 Future experimental pictorial warning studies should place warnings on smokers’ cigarette packs, measure smoking behaviour over longer periods of time, and include meaningfully intensive exposures.97 Second, given that smoking is a social behaviour,100 a better understanding of how social interactions influence warning effectiveness is needed. While some studies have assessed discussions about warnings,19 ,93 ,101 ,102 studies often use this variable in a composite scale representing depth of processing, making it impossible to tease out the influence of social interactions. Third and finally, an area that remains understudied is the effect of pictorial warnings on reducing smoking initiation. While warnings may be seen as messages designed only for smokers, some warnings could be specifically designed for youth and non-smokers.19 Future research should examine the differential impact of pictorial warnings on smokers and non-smokers, with careful attention to the potential for pictorial warnings to discourage smoking initiation among youth.

    Conclusion

    Our study was the first to estimate the effects of pictorial cigarette pack warnings through a meta-analysis of experimental studies. This investigation demonstrated that pictorial warnings were more effective than text warnings on the vast majority of outcomes studied, affecting several constructs, including intention to not start smoking and intention to quit smoking. Future research examining the effects of pictorial cigarette warnings should assess impact on smoking behaviour, including initiation and cessation. Studies should also adopt more explicit hypotheses derived from behavioural theory, use validated and standardised measures, include multiple follow-up assessments, and better advance a theoretical understanding of how warnings exert their effects.

    What this paper adds

    • Pictorial warnings on cigarette packs are a key international tobacco control policy. The current study presents the first meta-analysis of the experimental literature on pictorial cigarette pack warnings. This review found:

      • Pictorial warnings were more effective than text warnings on 20 of 25 outcomes examined in the meta-analysis;

      • Pictorial warnings were more effective (eg, increased quit intentions) and perceived to be more effective (eg, rated as likely to motivate smokers to quit) as compared with text warnings;

      • Pictorial warnings were more effective than text warnings in changing outcomes relevant to both non-smokers (eg, intentions to not start smoking) and smokers (eg, intentions to quit smoking).

    • Future experimental research should examine the impact of pictorial warnings on smoking behaviour. Future studies should also apply more behavioural theory, and test which theoretical variables mediate the effects of pictorial warnings. Such work would further strengthen the international evidence base for pictorial warnings and advance our ability to better understand the ‘active ingredients’ that underlie such warnings, informing more effective tobacco control policies.

    Acknowledgments

    The authors wish to thank the following people for providing additional data for this meta-analysis: Sonia A Duffy, Ahmed I Fathelrahman, Geoffrey T Fong, Sabine Glock, David Hammond, Ashish Nimbarte, James Nonnemaker, Ellen Peters, Daniel Romer, James F Thrasher and Ekant Veer.

    References (References in the meta-analysis are marked with an asterisk)

    1. 1.↵
      World Health Organization. WHO report on the global tobacco epidemic, 2013: enforcing bans on tobacco advertising, promotion and sponsorship. Geneva, Switzerland, 2013.
    2. 2.↵
      1. Wakefield M,
      2. Morley C,
      3. Horan J, et al
      . The cigarette pack as image: new evidence from tobacco industry documents. Tob Control 2002;11(Suppl 1):I73–80. doi:10.1136/tc.11.suppl_1.i73
      OpenUrlCrossRefPubMedWeb of Science
    3. 3.↵
      1. Moodie C,
      2. Hastings G
      . Tobacco packaging as promotion. Tob Control 2010;19:168–70. doi:10.1136/tc.2009.033449
      OpenUrlFREE Full Text
    4. 4.↵
      Centers for Disease Control and Prevention. Health warnings on tobacco products—worldwide, 2007. MMWR Morb Mortal Wkly Rep 2009;58:528–9.
      OpenUrlPubMed
    5. 5.↵
      1. Durkin S,
      2. Brennan E,
      3. Wakefield M
      . Mass media campaigns to promote smoking cessation among adults: an integrative review. Tob Control 2012;21:127–38. doi:10.1136/tobaccocontrol-2011-050345
      OpenUrlAbstract/FREE Full Text
    6. 6.↵
      World Health Organization. WHO framework convention on tobacco control. Geneva, Switzerland, 2003.
    7. 7.↵
      Canadian Cancer Society. Cigarette package health warnings: international status report. Canada, 2014.
    8. 8.↵
      1. Hammond D
      . Health warning messages on tobacco products: a review. Tob Control 2011;20:327–37. doi:10.1136/tc.2010.037630
      OpenUrlAbstract/FREE Full Text
    9. 9.↵
      1. Hammond D
      . Tobacco packaging and labeling policies under the US Tobacco Control Act: research needs and priorities. Nicotine Tob Res 2012;14:62–74. doi:10.1093/ntr/ntr182
      OpenUrlAbstract/FREE Full Text
    10. 10.↵
      1. Borland R,
      2. Yong H,
      3. Wilson N, et al
      . How reactions to cigarette packet health warnings influence quitting: findings from the ITC Four-Country survey. Addiction 2009;104:669–75. doi:10.1111/j.1360-0443.2009.02508.x
      OpenUrlCrossRefPubMedWeb of Science
    11. 11.↵
      1. Hammond D,
      2. Fong GT,
      3. McDonald PW, et al
      . Impact of the graphic Canadian warning labels on adult smoking behaviour. Tob Control 2003;12:391–5. doi:10.1136/tc.12.4.391
      OpenUrlAbstract/FREE Full Text
    12. 12.↵
      1. Hammond D,
      2. Wakefield M,
      3. Durkin S, et al
      . Tobacco packaging and mass media campaigns: research needs for Articles 11 and 12 of the WHO Framework Convention on Tobacco Control. Nicotine Tob Res 2013;15:817–31. doi:10.1093/ntr/nts202
      OpenUrlAbstract/FREE Full Text
    13. 13.↵
      1. Fong GT,
      2. Hammond D,
      3. Hitchman SC
      . The impact of pictures on the effectiveness of tobacco warnings. Bull World Health Organ 2009;87:640–3. doi:10.2471/BLT.09.069575
      OpenUrlCrossRefPubMedWeb of Science
    14. 14.↵
      1. Hammond D
      . Tobacco labelling and packaging toolkit: a guide to FCTC Article 11. Ontario, Canada: Tobacco Labelling Resource Centre, 2009.
    15. 15.↵
      1. Hammond D,
      2. Fong GT,
      3. Borland R, et al
      . Text and graphic warnings on cigarette packages: findings from the International Tobacco Control Four Country Study. Am J Prev Med 2007;32:202–9. doi:10.1016/j.amepre.2006.11.011
      OpenUrlCrossRefPubMedWeb of Science
    16. 16.↵
      1. Hammond D,
      2. Fong GT,
      3. McNeill A, et al
      . Effectiveness of cigarette warning labels in informing smokers about the risks of smoking: findings from the International Tobacco Control (ITC) Four Country Survey. Tob Control 2006;15(Suppl 3):iii19–25. doi:10.1136/tc.2005.012294
      OpenUrlAbstract/FREE Full Text
    17. 17.↵
      1. Hammond D,
      2. Reid JL,
      3. Driezen P, et al
      . Pictorial health warnings on cigarette packs in the United States: an experimental evaluation of the proposed FDA warnings. Nicotine Tob Res 2013;15:93–102. doi:10.1093/ntr/nts094
      OpenUrlAbstract/FREE Full Text
    18. 18.↵
      1. Thrasher JF,
      2. Hammond D,
      3. Fong GT, et al
      . Smokers’ reactions to cigarette package warnings with graphic imagery and with only text: a comparison between Mexico and Canada. Salud Publica Mex 2007;49(Suppl 2):S233–40. doi:10.1590/S0036-36342007000800013
      OpenUrlCrossRefPubMedWeb of Science
    19. 19.↵
      1. White V,
      2. Webster B,
      3. Wakefield M
      . Do graphic health warning labels have an impact on adolescents’ smoking-related beliefs and behaviours? Addiction 2008;103:1562–71. doi:10.1111/j.1360-0443.2008.02294.x
      OpenUrlCrossRefPubMedWeb of Science
    20. 20.↵
      1. Liefeld J
      , Canada Office of Tobacco Control. The relative importance of the size, content & pictures on cigarette package warning messages. Canada: Office of Tobacco Control, 1999.
    21. 21.↵
      1. Azagba S,
      2. Sharaf MF
      . The effect of graphic cigarette warning labels on smoking behavior: evidence from the Canadian experience. Nicotine Tob Res 2013;15:708–17. doi:10.1093/ntr/nts194
      OpenUrlAbstract/FREE Full Text
    22. 22.↵
      1. Cameron LD,
      2. Pepper JK,
      3. Brewer NT
      . Responses of young adults to graphic warning labels for cigarette packages. Tob Control 2015;24(e1):e14–22. doi:10.1136/tobaccocontrol-2012-050645
      OpenUrlAbstract/FREE Full Text
    23. *23.↵
      1. O'Hegarty M,
      2. Pederson LL,
      3. Nelson DE, et al
      . Reactions of young adult smokers to warning labels on cigarette packages. Am J Prev Med 2006;30:467–73. doi:10.1016/j.amepre.2006.01.018
      OpenUrlCrossRefPubMedWeb of Science
    24. *24.↵
      1. Bansal-Travers M,
      2. Hammond D,
      3. Smith P, et al
      . The impact of cigarette pack design, descriptors, and warning labels on risk perception in the US. Am J Prev Med 2011;40:674–82. doi:10.1016/j.amepre.2011.01.021
      OpenUrlCrossRefPubMed
    25. *25.↵
      1. Kees J,
      2. Burton S,
      3. Andrews JC, et al
      . Tests of graphic visuals and cigarette package warning combinations: implications for the framework convention on tobacco control. J Public Policy Marketing 2006;25:212–23. doi:10.1509/jppm.25.2.212
      OpenUrlCrossRef
    26. 26.↵
      1. Pepper JK,
      2. Cameron LD,
      3. Reiter PL, et al
      . Non-smoking male adolescents’ reactions to cigarette warnings. PLoS ONE 2013;8:e65533. doi:10.1371/journal.pone.0065533
      OpenUrlCrossRefPubMed
    27. *27.↵
      1. Sabbane LI,
      2. Lowrey TM,
      3. Chebat J-C
      . The effectiveness of cigarette warning label threats on nonsmoking adolescents. J Consum Aff 2009;43:332–45. doi:10.1111/j.1745-6606.2009.01142.x
      OpenUrlCrossRefWeb of Science
    28. *28.↵
      1. Kees J,
      2. Burton S,
      3. Andrews JC, et al
      . Understanding how graphic pictorial warnings work on cigarette packaging. J Public Policy Marketing 2010;29:265–76. doi:10.1509/jppm.29.2.265
      OpenUrlCrossRefWeb of Science
    29. *29.↵
      1. Nonnemaker J,
      2. Farrelly M,
      3. Kamyab K, et al
      . Experimental study of graphic cigarette warning labels: final results report. Prepared for Center for Tobacco Products, Food and Drug Administration, 2010.
    30. 30.↵
      1. Monarrez-Espino J,
      2. Liu B,
      3. Greiner F, et al
      . Systematic review of the effect of pictorial warnings on cigarette packages in smoking behavior. Am J Public Health 2014;104:e11–30. doi:10.2105/AJPH.2014.302129
      OpenUrl
    31. 31.↵
      Centers for Disease Control and Prevention. Cigarette package health warnings and interest in quitting smoking—14 countries, 2008–2010. MMWR Morb Mortal Wkly Rep 2011;60:645–51.
      OpenUrlPubMed
    32. 32.↵
      1. Davis R,
      2. Gilpin EA,
      3. Loken B, et al
      . The role of the media in promoting and reducing tobacco use. Bethesda, MD: USDHHS, NIH, National Cancer Institute. NCI Tobacco Control Monogr 2008;19:NIH Pub. No.07–6242.
    33. 33.↵
      National Cancer Institute. Cigarette warning labels, packaging & product labeling: current science & practice to identify research priorities. Washington, DC: National Institutes of Health, 2009.
    34. 34.↵
      1. Cohen J
      . A coefficient of agreement for nominal scales. Educ Psychol Meas 1960;20:37–46. doi:10.1177/001316446002000104
      OpenUrlCrossRefWeb of Science
    35. *35.↵
      1. Fathelrahman AI,
      2. Omar M,
      3. Awang R, et al
      . Impact of the new Malaysian cigarette pack warnings on smokers’ awareness of health risks and interest in quitting smoking. Int J Environ Res Public Health 2010;7:4089–99. doi:10.3390/ijerph7114089
      OpenUrlCrossRefPubMed
    36. *36.↵
      1. Schneider S,
      2. Gadinger M,
      3. Fischer A
      . Does the effect go up in smoke? A randomized controlled trial of pictorial warnings on cigarette packaging. Patient Educ Couns 2012;86:77–83. doi:10.1016/j.pec.2011.03.005
      OpenUrlCrossRefPubMed
    37. *37.↵
      1. Cantrell J,
      2. Vallone DM,
      3. Thrasher JF, et al
      . Impact of tobacco-related health warning labels across socioeconomic, race and ethnic groups: results from a randomized web-based experiment. PLoS ONE 2013;8:e52206. doi:10.1371/journal.pone.0052206
      OpenUrlCrossRefPubMed
    38. *38.↵
      1. Erceg-Hurn DM,
      2. Steed LG
      . Does exposure to cigarette health warnings elicit psychological reactance in smokers? J Appl Soc Psychol 2011;41:219–37. doi:10.1111/j.1559-1816.2010.00710.x
      OpenUrlCrossRef
    39. *39.↵
      1. Romer D,
      2. Peters E,
      3. Strasser AA, et al
      . Desire versus efficacy in smokers’ paradoxical reactions to pictorial health warnings for cigarettes. PLoS ONE 2013;8:e54937. doi:10.1371/journal.pone.0054937
      OpenUrlCrossRefPubMed
    40. *40.↵
      1. Golmier I,
      2. Chebat J-C,
      3. Gélinas-Chebat C
      . Can cigarette warnings counterbalance effects of smoking scenes in movies? Psychol Rep 2007;100:3–18. doi:10.2466/pr0.100.1.3-18
      OpenUrlAbstract/FREE Full Text
    41. *41.↵
      1. Glock S,
      2. Kneer J
      . Are deterrent pictures effective? The impact of warning labels on cognitive dissonance in smokers. Appl Psychol 2009;1:356–73.
      OpenUrl
    42. *42.↵
      1. Fong GT,
      2. Hammond D,
      3. Jiang Y, et al
      . Perceptions of tobacco health warnings in China compared with picture and text-only health warnings from other countries: an experimental study. Tob Control 2010;19(Suppl 2):I69–77. doi:10.1136/tc.2010.036483
      OpenUrlCrossRefPubMed
    43. *43.↵
      1. Jansen C,
      2. van der Berg M,
      3. Buurman C, et al
      . The scarier, the better? Effects of adding images to verbal warnings on cigarette packages. In: Carliner S, Verckens JP, de Waele C, eds. Information and document design: varieties on recent research. Amsterdam/Philadelphia: John Benjamins Publishing Company, 2006:124–47.
    44. *44.↵
      1. Hammond D,
      2. Thrasher J,
      3. Reid JL, et al
      . Perceived effectiveness of pictorial health warnings among Mexican youth and adults: a population-level intervention with potential to reduce tobacco-related inequities. Cancer Causes Control 2012;23:57–67. doi:10.1007/s10552-012-9902-4
      OpenUrlCrossRefPubMedWeb of Science
    45. *45.↵
      1. Qin Y,
      2. Wu M,
      3. Pan X, et al
      . Reactions of Chinese adults to warning labels on cigarette packages: a survey in Jiangsu Province. BMC Public Health 2011; 11:133. doi:10.1186/1471-2458-11-133
      OpenUrlCrossRefPubMed
    46. 46.↵
      1. McGuire WJ
      . Theoretical foundations of campaigns. In: Rice RE, Atkin C, eds. Public Communication Campaigns. 2nd edn. Newbury Park, CA: Sage, 1989:43–67.
    47. 47.↵
      1. Fishbein M,
      2. Ajzen I
      . Predicting and changing behavior: the reasoned action approach. New York, NY: Psychology Press, 2010.
    48. 48.↵
      1. Petty RE,
      2. Cacioppo JT
      . The elaboration likelihood model of persuasion. Adv Exp Soc Psychol 1986;19:123–205. doi:10.1016/S0065-2601(08)60214-2
      OpenUrlCrossRef
    49. 49.↵
      1. Southwell BG,
      2. Yzer MC
      . The roles of interpersonal communication in mass media campaigns. Commun Yearbook 2007;31:420–62.
      OpenUrl
    50. 50.↵
      1. Noar SM,
      2. Zimmerman RS
      . Health behavior theory and cumulative knowledge regarding health behaviors: are we moving in the right direction? Health Educ Res 2005;20:275–90. doi:10.1093/her/cyg113
      OpenUrlAbstract/FREE Full Text
    51. 51.↵
      1. Cane J,
      2. O'Connor D,
      3. Michie S
      . Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci 2012;7:37–53. doi:10.1186/1748-5908-7-37
      OpenUrlCrossRefPubMed
    52. 52.↵
      1. Witte K
      . Putting the fear back into fear appeals: the extended parallel process model. Commun Monogr 1992;59:329–49. doi:10.1080/03637759209376276
      OpenUrlCrossRefWeb of Science
    53. 53.↵
      International Agency for Research on Cancer. Measures to assess the effectiveness of restrictions on tobacco product labeling policies. Lyon, France: Methods for evaluating tobacco control policies, 2008.
    54. 54.↵
      1. Fong GT,
      2. Cummings KM,
      3. Borland R, et al
      . The conceptual framework of the International Tobacco Control (ITC) Policy Evaluation Project. Tob Control 2006;15(Suppl 3):iii3–11. doi:10.1136/tc.2005.014464
      OpenUrlAbstract/FREE Full Text
    55. 55.↵
      1. Strahan EJ,
      2. White K,
      3. Fong GT, et al
      . Enhancing the effectiveness of tobacco package warning labels: a social psychological perspective. Tob Control 2002;11:183–90. doi:10.1136/tc.11.3.183
      OpenUrlAbstract/FREE Full Text
    56. 56.↵
      1. Dillard JP,
      2. Ye S
      . The perceived effectiveness of persuasive messages: questions of structure, referent, and bias. J Health Commun 2008;13:149–68. doi:10.1080/10810730701854060
      OpenUrlCrossRefPubMedWeb of Science
    57. 57.↵
      1. Noar SM,
      2. Zimmerman RS,
      3. Palmgreen P, et al
      . Development and implementation of mass media campaigns to delay sexual initiation among African American and White youth. J Health Commun 2014;19:152–69. doi:10.1080/10810730.2013.811318
      OpenUrlCrossRefPubMed
    58. 58.↵
      1. Lipsey MW,
      2. Wilson DB
      . Practical meta-analysis. Thousand Oaks, CA: Sage Publications, 2001.
    59. 59.↵
      1. Hedges LV,
      2. Olkin I
      . Statistical methods for meta-analysis. Orlando, FL: Academic Press, 1985.
    60. 60.↵
      1. Dunlap WP,
      2. Cortina JM,
      3. Vaslow JB, et al
      . Meta-analysis of experiments with matched groups or repeated measures designs. Psychol Methods 1996;1:170–7. doi:10.1037/1082-989X.1.2.170
      OpenUrlCrossRefWeb of Science
    61. 61.↵
      1. Hofmann W,
      2. De Houwer J,
      3. Perugini M, et al
      . Evaluative conditioning in humans: a meta-analysis. Psychol Bull 2010;136:390–421. doi:10.1037/a0018916
      OpenUrlCrossRefPubMedWeb of Science
    62. *62.↵
      1. Duffy SA,
      2. Burton D
      . Cartoon characters as tobacco warning labels. Arch Pediatr Adolesc Med 2000;154:1230–6. doi:10.1001/archpedi.154.12.1230
      OpenUrlCrossRefPubMed
    63. *63.↵
      1. Glock S,
      2. Muller BC,
      3. Ritter S
      . Warning labels formulated as questions positively influence smoking-related risk perception. J Health Psychol 2013;18:252–62. doi:10.1177/1359105312439734
      OpenUrlAbstract/FREE Full Text
    64. *64.↵
      1. Gygax PM,
      2. Bosson M,
      3. Gay C, et al
      . Relevance of health warnings on cigarette packs: a psycholinguistic investigation. Health Commun 2010;25:397–409. doi:10.1080/10410236.2010.483334
      OpenUrlCrossRefPubMedWeb of Science
    65. *65.↵
      1. Hoek J,
      2. Maubach N,
      3. Gendall P
      . Effects of cigarette on-pack warning labels on smokers’ perceptions and behavior. In: Advances in Consumer Research—Asia-Pacific Conference Proceedings 2006:173–80.
    66. *66.↵
      1. Kempf DS,
      2. Harmon SK
      . Examining the effectiveness of proposed cigarette package warning labels with graphic images among US college students. Acad Marketing Stud J 2006;10:77–93.
      OpenUrl
    67. *67.↵
      1. Lin PN,
      2. Zimmermann MH,
      3. Bover Manderski MT, et al
      . Evaluation of graphic cigarette warning images on cravings to smoke. J Smok Cessat 2011;6:85–8. doi:10.1375/jsc.6.2.85
      OpenUrlCrossRef
    68. *68.↵
      1. Loeber S,
      2. Vollstadt-Klein S,
      3. Wilden S, et al
      . The effect of pictorial warnings on cigarette packages on attentional bias of smokers. Pharmacol Biochem Behav 2011;98:292–8. doi:10.1016/j.pbb.2011.01.010
      OpenUrlCrossRefPubMed
    69. *69.↵
      1. Malouff JM,
      2. Schutte NS,
      3. Rooke SE, et al
      . Effects on smokers of exposure to graphic warning images. Am J Addict 2012;21:555–7. doi:10.1111/j.1521-0391.2012.00284.x
      OpenUrlCrossRefPubMed
    70. *70.↵
      1. Nimbarte A,
      2. Aghazadeh F,
      3. Harvey C
      . Comparison of current US and Canadian cigarette pack warnings. Int Q Community Health Educ 2005;24:3–27. doi:10.2190/9PX0-NBG1-0ALA-G5YH
      OpenUrlCrossRefPubMed
    71. *71.↵
      1. Peters E,
      2. Romer D,
      3. Slovic P, et al
      . The impact and acceptability of Canadian-style cigarette warning labels among US smokers and nonsmokers. Nicotine Tob Res 2007;9:473–81. doi:10.1080/14622200701239639
      OpenUrlAbstract/FREE Full Text
    72. *72.↵
      1. Racela OC,
      2. Thoumrungroje A
      . The perceived ethicality and efficacy of fear appeals: the use of graphic warning labels in Thailand. J Int Bus Econ 2012;12:106–13.
      OpenUrl
    73. *73.↵
      1. Sabbane LI,
      2. Bellavance F,
      3. Chebat J-C
      . Recency versus repetition priming effects of cigarette warnings on nonsmoking teenagers: the moderating effects of cigarette-brand familiarity. J Appl Soc Psychol 2009;39:656–82. doi:10.1111/j.1559-1816.2009.00455.x
      OpenUrlCrossRefWeb of Science
    74. *74.↵
      1. Thrasher JF,
      2. Carpenter MJ,
      3. Andrews JO, et al
      . Cigarette warning label policy alternatives and smoking-related health disparities. Am J Prev Med 2012;43:590–600. doi:10.1016/j.amepre.2012.08.025
      OpenUrlCrossRefPubMed
    75. *75.↵
      1. Thrasher JF,
      2. Rousu MC,
      3. Anaya-Ocampo R, et al
      . Estimating the impact of different cigarette package warning label policies: the auction method. Addict Behav 2007;32:2916–25. doi:10.1016/j.addbeh.2007.05.018
      OpenUrlCrossRefPubMedWeb of Science
    76. *76.↵
      1. Thrasher JF,
      2. Rousu MC,
      3. Hammond D, et al
      . Estimating the impact of pictorial health warnings and “plain” cigarette packaging: evidence from experimental auctions among adult smokers in the United States. Health Policy 2011;102:41–8. doi:10.1016/j.healthpol.2011.06.003
      OpenUrlCrossRefPubMedWeb of Science
    77. *77.↵
      1. Vardavas CI,
      2. Connolly G,
      3. Karamanolis K, et al
      . Adolescents perceived effectiveness of the proposed European graphic tobacco warning labels. Eur J Public Health 2009;19:212–17. doi:10.1093/eurpub/ckp015
      OpenUrlAbstract/FREE Full Text
    78. *78.↵
      1. Veer E,
      2. Rank T
      . Warning! The following packet contains shocking images: the impact of mortality salience on the effectiveness of graphic cigarette warning labels. J Consum Behav 2012;11:225–33. doi:10.1002/cb.391
      OpenUrlCrossRef
    79. *79.↵
      1. Wade B,
      2. Merrill RM,
      3. Lindsay GB
      . Cigarette pack warning labels in Russia: how graphic should they be? Eur J Public Health 2011;21:366–72. doi:10.1093/eurpub/ckq098
      OpenUrlAbstract/FREE Full Text
    80. 80.↵
      1. Vangeli E,
      2. Stapleton J,
      3. Smit ES, et al
      . Predictors of attempts to stop smoking and their success in adult general population samples: a systematic review. Addiction 2011;106:2110–21. doi:10.1111/j.1360-0443.2011.03565.x
      OpenUrlCrossRefPubMedWeb of Science
    81. 81.↵
      1. Yong HH,
      2. Borland R,
      3. Thrasher JF, et al
      . Mediational pathways of the impact of cigarette warning labels on quit attempts. Health Psychol 2014;33:1410–20. doi:10.1037/hea0000056
      OpenUrlCrossRefPubMedWeb of Science
    82. 82.↵
      1. Dillard JP,
      2. Weber KM,
      3. Vail RG
      . The relationship between the perceived and actual effectiveness of persuasive messages: a meta-analysis with implications for formative campaign research. J Commun 2007;57:613–31. doi:10.1111/j.1460-2466.2007.00360.x
      OpenUrlCrossRefWeb of Science
    83. 83.↵
      1. Davis KC,
      2. Nonnemaker J,
      3. Duke J, et al
      . Perceived effectiveness of cessation advertisements: the importance of audience reactions and practical implications for media campaign planning. Health Commun 2012;28:461–72. doi:10.1080/10410236.2012.696535
      OpenUrlPubMed
    84. 84.↵
      1. Brennan E,
      2. Durkin SJ,
      3. Wakefield MA, et al
      . Assessing the effectiveness of antismoking television advertisements: do audience ratings of perceived effectiveness predict changes in quitting intentions and smoking behaviours? Tob Control 2014;23:412–18. doi:10.1136/tobaccocontrol-2012-050949
      OpenUrlAbstract/FREE Full Text
    85. 85.↵
      1. Ajzen I
      . The theory of planned behavior. Organ Behav Hum Decis Process 1991;50:179–211. doi:10.1016/0749-5978(91)90020-T
      OpenUrlCrossRefWeb of Science
    86. 86.↵
      1. Rosenstock IM
      . Historical origins of the health belief model. Health Educ Behav 1974;2:328–35.
      OpenUrlFREE Full Text
    87. 87.↵
      1. Witte K,
      2. Allen M
      . A meta-analysis of fear appeals: implications for effective public health campaigns. Health Educ Behav 2000;27:591–615. doi:10.1177/109019810002700506
      OpenUrlAbstract/FREE Full Text
    88. 88.↵
      1. Rogers RW
      . A protection motivation theory of fear appeals and attitude change. J Psychol 1975;91:93–114. doi:10.1080/00223980.1975.9915803
      OpenUrlCrossRefWeb of Science
    89. 89.↵
      1. Becker MH
      . The health belief model and personal health behavior. Health Educ Monogr 1974;2:324–508.
      OpenUrl
    90. 90.↵
      1. Li J,
      2. Grigg M
      . New Zealand: new graphic warnings encourage registrations with the quitline. Tob Control 2009;18:72. doi:10.1136/tc.2008.027649
      OpenUrlFREE Full Text
    91. 91.↵
      1. Miller CL,
      2. Hill DJ,
      3. Quester PG, et al
      . Impact on the Australian Quitline of new graphic cigarette pack warnings including the Quitline number. Tob Control 2009;18:235–7. doi:10.1136/tc.2008.028290
      OpenUrlAbstract/FREE Full Text
    92. 92.↵
      1. Dillard JP,
      2. Shen L
      . On the nature of reactance and its role in persuasive health communication. Commun Monogr 2005;72:144–68. doi:10.1080/03637750500111815
      OpenUrlCrossRefWeb of Science
    93. 93.↵
      1. Hammond D,
      2. Fong GT,
      3. McDonald PW, et al
      . Graphic Canadian cigarette warning labels and adverse outcomes: evidence from Canadian smokers. Am J Public Health 2004;94:1442–5. doi:10.2105/AJPH.94.8.1442
      OpenUrlCrossRefPubMedWeb of Science
    94. 94.↵
      1. Kahneman D,
      2. Tversky A
      . Prospect theory: an analysis of decision under risk. Econometrica 1979;47:263–91. doi:10.2307/1914185
      OpenUrlCrossRefWeb of Science
    95. 95.↵
      1. Brewer NT,
      2. Gilkey MB,
      3. Lillie SE, et al
      . Tables or bar graphs? Presenting test results in electronic medical records. Med Decis Making 2012;32: 545–53. doi:10.1177/0272989X12441395
      OpenUrlAbstract/FREE Full Text
    96. 96.↵
      1. Hsee CK
      . The evaluability hypothesis: an explanation for preference reversals between joint and separate evaluations of alternatives. Organ Behav Hum Decis Process 1996;67:247–57. doi:10.1006/obhd.1996.0077
      OpenUrlCrossRefWeb of Science
    97. 97.↵
      1. Brewer NT,
      2. Hall MG,
      3. Lee JGL, et al
      . Testing warning messages on smokers’ cigarette packages: a standardized protocol. Tob Control 2016;25: 1539.
      OpenUrlAbstract/FREE Full Text
    98. 98.↵
      1. McQueen A,
      2. Caburnay C,
      3. Kaphingst K, et al
      . What are the reactions of diverse US smokers when graphic warning labels are affixed to their cigarette packs? Memphis, TN: American Society of Preventive Oncology Conference, 2013.
    99. 99.↵
      1. Peters E,
      2. Romer D,
      3. Evans A
      . Reactive and thoughtful processing of graphic warnings: multiple roles for affect. Chapel Hill, NC: University of North Carolina, Lineberger Comprehensive Cancer Center, Center for Regulatory Research on Tobacco Communication, 2014.
    100. 100.↵
      1. Christakis NA,
      2. Fowler JH
      . The collective dynamics of smoking in a large social network. N Engl J Med 2008;358:2249–58. doi:10.1056/NEJMsa0706154
      OpenUrlCrossRefPubMedWeb of Science
    101. 101.↵
      1. Moodie C,
      2. MacKintosh AM,
      3. Hammond D
      . Adolescents’ response to text-only tobacco health warnings: results from the 2008 UK Youth Tobacco Policy Survey. Eur J Public Health 2010;20:463–9. doi:10.1093/eurpub/ckp199
      OpenUrlAbstract/FREE Full Text
    102. 102.↵
      1. Moodie C,
      2. Mackintosh AM,
      3. Hastings G
      . Adolescents’ response to pictorial warnings on the reverse panel of cigarette packs: a repeat cross-sectional study. Tob Control 2015;24(e1):e93–7. doi:10.1136/tobaccocontrol-2013-050999
      OpenUrlAbstract/FREE Full Text

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

      Files in this Data Supplement:

    Footnotes

    • Contributors SMN, KMR and NTB originated the study. SMN, MGH and DBF drafted the manuscript. MGH and DBF coded the studies. All authors provided critical feedback on drafts of the manuscript and approved the final manuscript.

    • Funding Research reported in this publication was supported by The National Cancer Institute and FDA Center for Tobacco Products (CTP) under Award Number P3OCAO1 6086-3832; 4CNC: Moving Evidence into Action, a collaborating site in the Cancer Prevention and Control Research Network (Grant U48/DP001944) from the Centers for Disease Control and Prevention and the National Cancer Institute; and a seed grant from the School of Journalism and Mass Communication at UNC Chapel Hill. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH or the Food and Drug Administration.

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.