You are here

Article Text

Article menu

Download PDFPDF

Research paper
Impact of e-cigarette health warnings on motivation to vape and smoke
  1. Noel T Brewer1,2,
  2. Michelle Jeong3,
  3. Marissa G Hall1,2,
  4. Sabeeh A Baig1,4,
  5. Jennifer R Mendel2,
  6. Allison J Lazard2,5,
  7. Seth M Noar2,5,
  8. Madeline R Kameny1,
  9. Kurt M Ribisl1,2
  1. 1 Department of Health Behavior, Gillings School of Global Public Health, University of North Carolina, Chapel Hill, North Carolina, USA
  2. 2 Lineberger Comprehensive Cancer Center, University of North Carolina, Chapel Hill, North Carolina, USA
  3. 3 Center for Tobacco Studies, Rutgers School of Public Health, New Brunswick, New Jersey, USA
  4. 4 Department of Sociomedical Science, Columbia University Mailman School of Public Health, New York, New York, USA
  5. 5 School of Media and Journalism, University of North Carolina, Chapel Hill, North Carolina, USA
  1. Correspondence to Dr Noel T Brewer, Department of Health Behavior, Gillings School of Global Public Health, University of North Carolina, Chapel Hill, North Carolina, USA; ntb{at}unc.edu

Abstract

Background A prevailing hypothesis is that health warnings for electronic cigarettes (e-cigarettes) could drive people away from vaping and towards smoking cigarettes. We consider an alternative hypothesis that e-cigarette warnings discourage both vaping and smoking.

Methods Participants were a national convenience sample of 2218 US adults who used e-cigarettes, cigarettes or both. In August 2018, we randomised participants to one of three warning types (control text about littering, text-only e-cigarette warning or pictorial e-cigarette warning). We further randomised participants viewing e-cigarette warnings to one of three topics (nicotine addiction, health hazards of use, or both health hazards and harms of use). The preregistered primary outcome was intentions to quit vaping among e-cigarette users. Secondary outcomes included interest in smoking and Tobacco Warnings Model constructs: attention, negative affect, anticipated social interactions and cognitive elaboration.

Results Text warnings elicited higher intentions to quit vaping than control among e-cigarette users (d=0.44, p<0.001), and pictorial warnings elicited still higher intentions to quit vaping than text (d=0.12, p<0.05). Text warnings elicited lower interest in smoking compared with control among smokers (p<0.05); warnings had no other effects on interest in smoking among smokers or non-smokers. Text warnings about health hazards elicited higher intentions to quit vaping than nicotine addiction warnings. E-cigarette warnings also increased Tobacco Warnings Model constructs.

Discussion E-cigarette health warnings may motivate users to quit vaping and discourage smoking. The most promising warnings include health hazards (other than nicotine addiction) and imagery. We found no support for the hypothesis that e-cigarette warnings could encourage smoking cigarettes.

  • electronic nicotine delivery devices
  • addiction
  • nicotine
  • packaging and labelling
  • prevention

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/tobaccocontrol-2018-054878

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Ever use of electronic nicotine delivery systems, or electronic cigarettes (e-cigarettes), has steadily increased,1 particularly among youth and young adults.2 3 Many e-cigarette users vape to quit smoking4 5 and believe using e-cigarettes is less harmful than smoking combustible cigarettes.4 6 However, growing evidence identifies health hazards and harms caused by vaping.7 E-cigarettes contain nicotine that is highly addictive8 and can harm adolescents’ neurological development.9 E-cigarettes also expose users to toxic chemicals (eg, formaldehyde) that can cause DNA damage10 and explosions that can cause burns.11 Although vaping by itself is likely less harmful than smoking, these risks are meaningful, especially when compared with not using any tobacco products.

    At least 14 countries have already required that e-cigarettes bear health warning labels,12 and many others are considering implementing policies to require e-cigarette warnings. For example, the US Food and Drug Administration (FDA) in 2018 required e-cigarette packages and advertisements to carry a single, text-only warning about nicotine addiction.13 However, the impact of a single warning will likely wear out over time,14–18 providing good reason to consider additional warning topics for future message rotation. Few studies19–22 have examined the impact of warnings about hazards (ie, causes of health consequences) and harms (ie, health consequences) of e-cigarettes. Furthermore, pictorial cigarette pack warnings are more effective in motivating quitting23 24 and increasing other psychosocial antecedents to quitting25 than text-only warnings. To our knowledge, no studies have compared the effects of pictorial with text-only warnings for e-cigarette devices.

    Because some people may misunderstand e-cigarette health warnings,26 unintended consequences of the warnings are a concern.27 E-cigarette warnings could result in misperceptions that e-cigarettes are as harmful as cigarettes, potentially driving people away from vaping and towards smoking cigarettes.27 However, we consider an alternative possibility that e-cigarette warnings could in fact discourage both vaping and smoking. We examined these questions in an experiment on e-cigarette warnings.

    Methods

    Participants

    In August 2018, we recruited a convenience sample of 2218 US adults, aged 18 or older, who currently smoked or vaped. Online convenience samples are a quick and low-cost way to study health behaviour and can yield highly generalisable findings for experiments.28 Participants were current e-cigarette users (defined as currently vaping every day or some days29), current smokers (defined as having smoked at least 100 cigarettes and now smoking every day or some days30 31) or dual users (defined as a current e-cigarette user and current smoker). Recruitment occurred through Amazon’s Prime Panels, a platform with access to over 20 million participants for behavioural research.32

    Procedures

    Stimuli development

    We developed e-cigarette warnings and control text using a multistep procedure (figure 1). First, we developed text-only health warnings. We adapted the current FDA e-cigarette warning about nicotine addiction: ‘WARNING: This product contains nicotine. Nicotine is an addictive chemical’.13 We removed the marker ‘warning’ to increase text size, thereby improving legibility, and adapted the statement into three variations of the nicotine addiction warning. For the other conditions, we identified three e-cigarette hazards (eg, harmful chemicals) and associated harms (eg, poisonous if swallowed) based on the latest scientific literature on hazards and harms associated with vaping.7 We selected e-cigarette hazards and harms with strong evidence, defined as a designation of ‘conclusive’ or ‘substantial’ evidence by the National Academies of Sciences, Engineering, and Medicine.7 We then integrated these hazards and harms into novel e-cigarette warnings based on our prior experience with developing tobacco risk communications.33–36

    Figure 1
    Figure 1

    E-cigarette warnings used in the experiment.

    Second, we created pictorial health warnings that combined each of the e-cigarette text warnings with a relevant photograph. A professional designer created these photographs using stock photos, stimuli developed in previous research studies and recent news stories about e-cigarette harms. Third, we developed text for the control condition. We adapted three control messages about not littering e-cigarettes from a previous trial.37 Finally, the designer developed control text, warning text and relevant pictorial material into visuals matched for layout, size and text colour.

    Experiment

    We randomised participants to a 2×3 factorial experiment with an additional control condition (ie, seven conditions, shown in figure 1). The first factor varied the warning type: text or pictorial. The second factor varied the warning topic: nicotine addiction (eg, ‘Nicotine is an addictive chemical’), hazard (eg, ‘E-cigarettes can explode’), or both hazard and harm (eg, ‘E-cigarettes can explode and cause burns’). The control condition presented text about not littering. In each condition, participants viewed three different warnings (or control text) in a random order and evaluated them. Participants received incentives in cash, gift cards or reward points from Prime Panels.

    Measures

    Participants rated each of the three e-cigarette warnings on attention (‘How much does this message grab your attention?’),38 believability (‘How believable is this message?’)39 and perceived message effectiveness (PME) (the three-item University of North Carolina (UNC) PME Scale: for example, ‘This message discourages me from wanting to vape’).40 After participants evaluated all three warnings, the survey assessed the primary outcome of intentions to quit vaping among current e-cigarette users (three items).41 The survey also assessed interest in e-cigarette use among current e-cigarette users (ie, planned to stop (coded as 1), decrease (2), continue (3) or increase (4) vaping); interest in cigarette use among current smokers (ie, planned to stop (coded as 1), decrease (2), continue (3) or increase (4) smoking); and interest in cigarette use among non-smokers (ie, planned to start smoking (no coded as 0, yes coded as 1).42 The survey also assessed behavioural antecedents from the UNC Tobacco Warnings Model (TWM), a model that proposes mechanisms by which warnings impact tobacco product use,36 including negative affective reactions to the messages (ie, fear, anxiety, disgust, sadness, guilt),38 43 44 cognitive elaboration (ie, thinking about health problems, information conveyed and quitting e-cigarettes)37 45 46 and anticipated social interactions about the messages.47 48 The survey assessed message reactance49 50 and anticipated avoidance of the messages.29 51 Finally, the survey assessed several risk beliefs for vaping and for smoking: affective risk perception (ie, scared), experiential risk perception52 (ie, concerned), perceived likelihood, perceived severity53 and perceived harm54 (eg, vaping is less or more harmful than smoking). All measures except for interest in vaping and smoking used 5-point response scales (coded as 1 (low) to 5 (high)). The survey also assessed standard demographics and tobacco product use. Survey measures appear in online supplementary table S1.

    Data analysis

    To determine whether randomisation was successful, we compared participant demographic characteristics across experimental conditions, for warning type (control, text, pictorial) and, among text-only warnings, topic (nicotine addiction, hazard, hazard and harm), using multinomial logistic regression. None of the 40 comparisons were statistically significant, confirming successful randomisation (all p>0.08).

    We examined the impact of e-cigarette warning type on intentions to quit e-cigarettes among current e-cigarette users using t-tests (ie, control vs text; text vs pictorial). We examined the impact of warning topic on e-cigarette quit intentions among current e-cigarette users using t-tests (ie, text-only warnings: nicotine addiction vs hazard; hazard vs hazard and harm).

    We used similar procedures to examine the impact of warning type and topic on interest in e-cigarette use (among e-cigarette users), interest in cigarette use (among smokers), behavioural antecedents from the TWM, anticipated avoidance, believability, message reactance, PME and risk beliefs. For attention, believability and PME, we used repeated-measures analysis of variance because these constructs were assessed for each of the three warnings. Data analyses were conducted in R (V.3.5.1), and statistical tests used a critical alpha of 0.05. Analyses followed our preregistration on AsPredicted.org (ID 13293).

    Results

    The mean age of participants was 43 years (table 1). Most were white (80%) and only 39% had a college degree. A minority of participants were gay, lesbian or bisexual (13%) or lived in poverty (30%). Most participants currently used both cigarettes and e-cigarettes (40%) or cigarettes only (40%), and the remainder used only e-cigarettes (20%).

    View this table:
    Table 1

    Participant characteristics (N=2218)

    Warning type

    Text e-cigarette warnings elicited reactions more likely to discourage vaping than control text, and pictorial warnings did better still (table 2). With respect to the primary outcome, text warnings elicited higher intentions to quit vaping than control text among e-cigarette users (mean=2.60, SD=1.27 vs mean=2.06, SD=1.17; p<0.001; figure 2). Pictorial warnings elicited still higher intentions to quit vaping than text warnings (mean=2.76, SD=1.76 vs mean=2.60, SD=1.27; p<0.05). These findings reflect a larger increase for text warnings (compared with control, d=0.44) than for pictorial warnings (compared with text-only e-cigarette warnings, d=0.12), a finding that carried through to the other outcomes. Exploratory analyses found similar impacts of warning type on intentions among key subgroups (ie, no moderation by respondent age, cigarette use or e-cigarette use).

    Figure 2
    Figure 2

    Impact of warnings on intentions to quit using e-cigarettes. Error bars show standard errors. 1=no intention; 4=high intention.

    View this table:
    Table 2

    Impact of type of e-cigarette warning

    With respect to TWM constructs, text warnings led to higher attention than control, but pictorial warnings did not elicit higher attention than text warnings. Text warnings led to more negative affect, anticipated social interactions and cognitive elaboration compared with control (all p<0.05; table 2). Pictorial warnings led to somewhat higher scores than text warnings for negative affect, anticipated social interactions and cognitive elaboration (all p<0.05). Text warnings elicited lower interest in e-cigarette use compared with control among e-cigarette users (p<0.05), but pictorial and text warnings did not differ. Text warnings elicited higher PME than control, and pictorial warnings elicited somewhat higher ratings than text warnings (both p<0.05). PME, believability and attention scores for the individual warning messages appear in online supplementary figure S1.

    For risk beliefs, the warnings led to few changes (oOnline supplementary table S2), text warnings led to higher affective risk and perceived harm for vaping compared with control (all p<0.05), and pictorial warnings led to higher levels on these variables compared with text (all p<0.05). Pictorial warnings led to higher perceived severity compared with text (p<0.05). No differences were found on other risk beliefs for vaping, and no differences were found on any risk beliefs for smoking except pictorial leading to higher affective risk compared with text (p<0.05).

    With respect to unintended effects, text warnings elicited lower interest in smoking compared with control (p<0.05) among smokers; text and pictorial warnings had similar effects. Among non-smokers, warning type was not associated with planning to start smoking cigarettes. Text warnings elicited greater anticipated message avoidance and message reactance compared with control (p<0.05), with slightly larger effects among pictorial than text warnings (p<0.05).

    Warning topic

    E-cigarette warnings about health hazards elicited reactions more likely to discourage vaping than nicotine addiction warnings on almost all outcomes, but adding a health harm to a hazard generally had no additional benefit (table 3). In the text warning arm, warnings about health hazards elicited higher intentions to quit vaping (mean=2.68, SD=1.25) than nicotine addiction warnings (mean=2.45, SD=1.21) among e-cigarette users (d=0.19, p<0.001). Warnings with both health hazards and harms elicited similar intentions as the warnings with hazards only (d=0.00, p=0.98). Exploratory analyses found similar impacts of warning topic on intentions among key subgroups.

    View this table:
    Table 3

    Impact of topic of text-only e-cigarette warning

    With respect to TWM constructs, text hazard warnings led to higher attention, negative affect, anticipated social interactions and cognitive elaboration, compared with the nicotine addiction warnings (all p<0.05). Text warnings led to lower believability (p<0.05). Text hazard warnings also led to higher PME (p<0.05). Hazard warnings did not differ on any outcomes from hazard plus harm warnings. Warning topic had no impact on any risk beliefs for vaping or smoking, except warnings about hazards leading to higher affective risk than warnings about nicotine addiction (p<0.05) (online supplementary table S3).

    With respect to unintended effects of text warnings, hazard warnings elicited more message reactance than nicotine addiction warnings (p<0.05). Warning topic had no other effects on message reactance, anticipated message avoidance or interest in cigarette use.

    Discussion

    Text warnings motivated adult e-cigarette users to want to quit vaping, with a small added benefit from including photographic images. Both text and pictorial warnings made smokers less interested in smoking compared with the control, suggesting that e-cigarette warnings could discourage both vaping and smoking. Warnings about nicotine addiction also discouraged vaping, providing evidence that the warning currently required on e-cigarette packages and advertisements in the USA may be effective. However, warnings with specific information about other hazards of vaping were even more discouraging of e-cigarette use than the nicotine addiction warnings.

    Our findings support initial use of text-only e-cigarette warnings, which had clear benefits in our study. E-cigarette health hazards are still highly novel for many people, and therefore text warnings highlighting these hazards may, at this time, be enough to effectively capture people’s attention and motivate behaviour change. Our findings show that pictorial warnings are somewhat more effective than text-only warnings, but the additional impact is small relative to text-only warnings at this time. Furthermore, if the history of pictorial cigarette pack warnings in the USA is any indication, pictorial content for e-cigarette warnings in the USA could prompt litigation by product manufacturers.55 Recent case law in the USA indicates that warnings must be both factual and uncontroversial, based on the First Amendment restrictions on compelled speech.56 US courts may readily agree text-only warnings about health harms and hazards are both factual and uncontroversial, allowing for more timely implementation. In contrast, pictorial e-cigarette warnings in the USA could be sidelined by the same kind of legal manoeuvring that has delayed implementation of pictorial cigarette warnings there. Therefore, it may be more strategic to implement additional e-cigarette text warnings in the USA as an immediate next step, while developing effective pictorial e-cigarette warnings for later implementation. Such a stepped approach would mirror the global history of cigarette pack warnings, which evolved from text to pictorial warnings,57 which may have greater impact over time. In other countries where governments have more leeway to implement effective warnings without the threat of industry litigation, pictorial e-cigarette warnings are a promising policy strategy.

    Our research shows that the current FDA nicotine addiction warning is likely to be effective. Extending warning topics to include other health hazards and harms is a clear next step to increase the impact of text-only warnings. As warnings’ effects tend to wear out over time,15–18 using multiple messages and rotating them could help to maximise the public health benefit of warnings. Our findings suggest that adding harms (eg, ‘poisonous if swallowed’) to hazards (eg, ‘e-liquids contain harmful chemicals’) produced no additional benefit, consistent with other studies showing that adding more information may not always increase the impact of risk communication.58 Another possibility is that people infer harms when reading hazard messages (eg, the hazard ‘e-cigarettes can explode’ may imply that the harm ‘causes burns’), with the result being that adding explicit harm information adds nothing new. However, we did not address what harm information would do on its own (eg, e-cigarette liquid can poison you). This is a potential area for future study.

    In our study, text and pictorial e-cigarette warnings elicited lower interest in smoking than the control among smokers. This promising finding suggests that e-cigarette warnings’ impact may generalise across several tobacco products, making them seem unappealing and discouraging their use, an important result given concern that e-cigarette warnings could drive people away from e-cigarettes and towards increased smoking. Future studies should examine whether e-cigarette warnings’ impact extends to reducing smoking and increasing quit attempts. In addition, future studies should examine the impact of e-cigarette warnings that directly compare the harms of smoking and vaping. In terms of other unintended effects, text and pictorial warnings generated some reactance to and avoidance of warnings, in line with prior research.59–62 However, reactance does not undermine the beneficial impact of otherwise effective warnings,49 63 64 and avoidance of warnings appears to be a marker of greater warning effectiveness.25 36 65 66

    The TWM36 offered substantial value in this new context of e-cigarette warnings. Consistent with the model, e-cigarette warnings increased negative affective reactions including fear, expected social interactions, thinking about harms and intentions to not use e-cigarettes. Previously, we demonstrated the model’s value for understanding how pictorial cigarette pack warnings36 and warnings on sugar-sweetened beverages67 motivate behaviour change. The model also helped to explain why text disclosures about toxic chemicals in cigarette smoke were largely ineffective in increasing intentions to quit smoking (ie, because they garnered little attention and generated few social interactions that failed to lead to elaboration of the messages).37 The model appears to be quite generalisable, at least across appetitive behaviours like tobacco use and beverage consumption. In contrast, risk beliefs appeared to play little to no role in the impact of e-cigarette warnings, which is consistent with experimental work on pictorial cigarette pack warnings finding no changes across extant risk beliefs.24 68 69

    The strengths of our study include use of an experimental design with successful randomisation, multiple messages, a large national sample of e-cigarette users and smokers, and highly consistent findings across outcomes. Study limitations include use of a convenience sample. Our and others’ past research suggests that online convenience samples provide stable and representative estimates of the impact of experimental manipulations,28 70 71 but confirming our findings in a probability sample would confirm their generalisability. Furthermore, we used measures that are validated for smokers and cigarette use, but these measures may need further validation in the context of e-cigarettes. Lastly, because our experiment did not have a behavioural outcome, future studies should examine the behavioural impact of e-cigarette warnings.

    Conclusions

    E-cigarette warnings could be an effective tool for encouraging e-cigarette users to quit vaping. Such warnings may have the additional benefit of reducing interest in smoking among smokers. Our study suggests that warnings about nicotine addiction could help to reduce vaping, but additional warnings about a broader set of e-cigarette hazards and health harms could be even more effective.

    What this paper adds

    • Cigarette warnings have been the subject of intensive study over several decades. In contrast, e-cigarette warnings have received limited evaluation about warning types, topics and impact. The popularity of vaping among young people has soared over recent years, demonstrating a need for such research.

    • In our study, text-only e-cigarette warnings led to higher intentions to quit vaping, and pictorial warnings pushed intentions to quit even higher.

    • Nicotine addiction warnings were effective, lending support to the new US Food and Drug Administration policy requiring this text. Warnings with other hazards of use were even more effective, suggesting a next step for warning regulations.

    • Among smokers, e-cigarette warnings led to less interest in smoking.

    References

    1. ↵
      2. Wang TW ,
      3. Asman K ,
      4. Gentzke AS , et al
      . Tobacco Product Use Among Adults - United States, 2017. MMWR Morb Mortal Wkly Rep 2018;67:1225–32.doi:10.15585/mmwr.mm6744a2
      OpenUrlPubMed
    2. ↵
      2. Cullen KA ,
      3. Ambrose BK ,
      4. Gentzke AS , et al
      . Notes from the Field: Use of Electronic Cigarettes and Any Tobacco Product Among Middle and High School Students - United States, 2011-2018. MMWR Morb Mortal Wkly Rep 2018;67:1276–7.doi:10.15585/mmwr.mm6745a5
      OpenUrlCrossRefPubMed
    3. ↵
      QuickStats: Percentage* of Adults Who Ever Used an E-cigarette and Percentage Who Currently Use E-cigarettes,§ by Age Group - National Health Interview Survey, United States, 2016 . MMWR Morb Mortal Wkly Rep 2017;66.doi:10.15585/mmwr.mm6633a6
    4. ↵
      2. Patel D ,
      3. Davis KC ,
      4. Cox S , et al
      . Reasons for current e-cigarette use among U.S. adults. Prev Med 2016;93:14–20.doi:10.1016/j.ypmed.2016.09.011
      OpenUrl
    5. ↵
      2. Bold KW ,
      3. Kong G ,
      4. Cavallo DA , et al
      . Reasons for trying E-cigarettes and risk of continued use. Pediatrics 2016;138. doi:doi:10.1542/peds.2016-0895. [Epub ahead of print: 08 Aug 2016].
    6. ↵
      2. Gorukanti A ,
      3. Delucchi K ,
      4. Ling P , et al
      . Adolescents' attitudes towards e-cigarette ingredients, safety, addictive properties, social norms, and regulation. Prev Med 2017;94:65–71.doi:10.1016/j.ypmed.2016.10.019
      OpenUrlPubMed
    7. ↵
      1. National Academies of Sciences Engineering and Medicine
      . Public health consequences of e-cigarettes. Washington, DC: The National Academies Press, 2018.
    8. ↵
      2. Liu G ,
      3. Wasserman E ,
      4. Kong L , et al
      . A comparison of nicotine dependence among exclusive e-cigarette and cigarette users in the path study. Prev Med 2017;104:86–91.doi:10.1016/j.ypmed.2017.04.001
      OpenUrl
    9. ↵
      2. England LJ ,
      3. Bunnell RE ,
      4. Pechacek TF , et al
      . Nicotine and the developing human: a neglected element in the electronic cigarette debate. Am J Prev Med 2015;49:286–-93.doi:10.1016/j.amepre.2015.01.015
      OpenUrlCrossRefPubMed
    10. ↵
      2. Canistro D ,
      3. Vivarelli F ,
      4. Cirillo S , et al
      . E-cigarettes induce toxicological effects that can raise the cancer risk. Sci Rep 2017;7.doi:10.1038/s41598-017-02317-8
    11. ↵
      2. Toy J ,
      3. Dong F ,
      4. Lee C , et al
      . Alarming increase in electronic nicotine delivery systems-related burn injuries: a serious unregulated public health issue. Am J Emerg Med 2017;35:1781–2.doi:10.1016/j.ajem.2017.05.029
      OpenUrl
    12. ↵
      2. Kennedy RD ,
      3. Awopegba A ,
      4. De León E , et al
      . Global approaches to regulating electronic cigarettes. Tob Control 2017;26:440–5.doi:10.1136/tobaccocontrol-2016-053179
      OpenUrlAbstract/FREE Full Text
    13. ↵
      1. Food and Drug Administration, HHS
      . Deeming tobacco products to be subject to the federal food, drug, and cosmetic Act, as amended by the family smoking prevention and tobacco control act; restrictions on the sale and distribution of tobacco products and required warning statements for tobacco products. final rule. Fed Regist 2016;81:28973–9106.
      OpenUrlPubMed
    14. ↵
      2. Li L ,
      3. Borland R ,
      4. Yong H , et al
      . Longer term impact of cigarette package warnings in Australia compared with the United Kingdom and Canada. Health Educ Res 2015;30:67–80.doi:10.1093/her/cyu074
      OpenUrlCrossRefPubMed
    15. ↵
      2. Hammond D ,
      3. Fong GT ,
      4. Borland R , et al
      . Text and graphic warnings on cigarette packages: findings from the International tobacco control four country study. Am J Prev Med 2007;32:202–9.doi:10.1016/j.amepre.2006.11.011
      OpenUrlCrossRefPubMedWeb of Science
    16. ↵
      2. Hitchman SC ,
      3. Driezen P ,
      4. Logel C , et al
      . Changes in effectiveness of cigarette health warnings over time in Canada and the United States, 2002-2011. Nicotine Tob Res 2014;16:536–43.doi:10.1093/ntr/ntt196
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      2. Thrasher JF ,
      3. Abad-Vivero EN ,
      4. Huang L , et al
      . Interpersonal communication about pictorial health warnings on cigarette packages: Policy-related influences and relationships with smoking cessation attempts. Soc Sci Med 2015;S0277-9536:00331–7.
      OpenUrl
    18. ↵
      2. Swayampakala K ,
      3. Thrasher JF ,
      4. Yong H-H , et al
      . Over-Time impacts of pictorial health warning labels and their differences across smoker subgroups: results from adult smokers in Canada and Australia. Nicotine Tob Res 2018;20:888–96.doi:10.1093/ntr/ntx134
      OpenUrl
    19. ↵
      2. Lee YO ,
      3. Shafer PR ,
      4. Eggers ME , et al
      . Effect of a voluntary e-cigarette warning label on risk perceptions. Tob Regul Sci 2016;2:82–93.doi:10.18001/TRS.2.1.9
      OpenUrl
    20. ↵
      2. Mays D ,
      3. Smith C ,
      4. Johnson AC , et al
      . An experimental study of the effects of electronic cigarette warnings on young adult nonsmokers' perceptions and behavioral intentions. Tob Induc Dis 2016;14.doi:10.1186/s12971-016-0083-x
    21. ↵
      2. Popova L ,
      3. Ling PM
      . Nonsmokers' responses to new warning labels on smokeless tobacco and electronic cigarettes: an experimental study. BMC Public Health 2014;14.doi:10.1186/1471-2458-14-997
    22. ↵
      2. Sanders-Jackson A ,
      3. Schleicher NC ,
      4. Fortmann SP , et al
      . Effect of warning statements in e-cigarette advertisements: an experiment with young adults in the United States. Addiction 2015;110:2015–24.doi:10.1111/add.12838
      OpenUrl
    23. ↵
      2. Brewer NT ,
      3. Hall MG ,
      4. Noar SM , et al
      . Effect of pictorial cigarette pack warnings on changes in smoking behavior: a randomized clinical trial. JAMA Intern Med 2016;176:905–12.doi:10.1001/jamainternmed.2016.2621
      OpenUrl
    24. ↵
      2. Noar SM ,
      3. Hall MG ,
      4. Francis DB , et al
      . Pictorial cigarette pack warnings: a meta-analysis of experimental studies. Tob Control 2016;25:341–54.doi:10.1136/tobaccocontrol-2014-051978
      OpenUrlAbstract/FREE Full Text
    25. ↵
      2. Hall MG ,
      3. Mendel JR ,
      4. Noar SM , et al
      . Why smokers avoid cigarette pack risk messages: two randomized clinical trials in the United States. Soc Sci Med 2018;213:165–72.doi:10.1016/j.socscimed.2018.07.049
      OpenUrl
    26. ↵
      2. Majeed BA ,
      3. Weaver SR ,
      4. Gregory KR , et al
      . Changing perceptions of harm of e-cigarettes among U.S. adults, 2012-2015. Am J Prev Med 2017;52:331–8.doi:10.1016/j.amepre.2016.08.039
      OpenUrlPubMed
    27. ↵
      2. Wackowski OA ,
      3. Hammond D ,
      4. O'Connor RJ , et al
      . Considerations and future research directions for e-cigarette Warnings-Findings from expert interviews. Int J Environ Res Public Health 2017;14. doi:doi:10.3390/ijerph14070781. [Epub ahead of print: 14 Jul 2017].
    28. ↵
      2. Jeong M ,
      3. Zhang D ,
      4. Morgan JC , et al
      . Similarities and differences in tobacco control research findings from convenience and probability samples. Ann Behav Med 2018;75.doi:10.1093/abm/kay059
    29. ↵
      1. Population Assessment of Tobacco and Health Study
      . PATH: population assessment of tobacco and health 2018. Available: http://www.pathstudyinfo.nih.gov/UI/HomeMobile.aspx [Accessed April 3 2018].
    30. ↵
      1. Centers for Disease Control and Prevention (CDC)
      . State-specific prevalence and trends in adult cigarette smoking--United States, 1998-2007. MMWR Morb Mortal Wkly Rep 2009;58:221–6.
      OpenUrlPubMed
    31. ↵
      2. Arrazola RA ,
      3. Singh T ,
      4. Corey CG , et al
      . Tobacco use among middle and high school students - United States, 2011-2014. MMWR Morb Mortal Wkly Rep 2015;64:381–5.
      OpenUrlPubMed
    32. ↵
      2. Crimmins EM ,
      3. Kim JK ,
      4. Alley DE , et al
      . Hispanic paradox in biological risk profiles. Am J Public Health 2007;97:1305–10.doi:10.2105/AJPH.2006.091892
      OpenUrlCrossRefPubMedWeb of Science
    33. ↵
      2. Noar SM ,
      3. Kelley DE ,
      4. Boynton MH , et al
      . Identifying principles for effective messages about chemicals in cigarette smoke. Prev Med 2018;106:31–7.doi:10.1016/j.ypmed.2017.09.005
      OpenUrl
    34. ↵
      2. Kelley DE ,
      3. Boynton MH ,
      4. Noar SM , et al
      . Effective message elements for disclosures about chemicals in cigarette smoke. Nicotine Tob Res 2018;20:1047–54.
      OpenUrl
    35. ↵
      2. Brewer NT ,
      3. Morgan JC ,
      4. Baig SA , et al
      . Public understanding of cigarette smoke constituents: three US surveys. Tob Control 2017;26:592–9.doi:10.1136/tobaccocontrol-2015-052897
      OpenUrlAbstract/FREE Full Text
    36. ↵
      2. Brewer NT ,
      3. Parada Jr H ,
      4. Hall MG , et al
      . Understanding why pictorial cigarette pack warnings increase quit attempts. Ann Behav Med 2018.
    37. ↵
      2. Brewer NT ,
      3. Jeong M ,
      4. Mendel JR , et al
      . Cigarette pack messages about toxic chemicals: a randomised clinical trial. Tob Control 2019;28:74–80.doi:10.1136/tobaccocontrol-2017-054112
      OpenUrlAbstract/FREE Full Text
    38. ↵
      2. Nonnemaker J ,
      3. Farrelly M ,
      4. Kamyab K , et al
      . Experimental study of graphic cigarette warning labels: final results report. Research Triangle Park, NC: RTI International, 2010.
    39. ↵
      2. Lazard AJ ,
      3. Kowitt SD ,
      4. Huang L-L , et al
      . Believability of cigarette warnings about addiction: national experiments of adolescents and adults. Nicotine Tob Res 2018;20:867–75.doi:10.1093/ntr/ntx185
      OpenUrl
    40. ↵
      2. Baig SA ,
      3. Noar SM ,
      4. Gottfredson NC , et al
      . UNC perceived message effectiveness: validation of a brief scale. Ann Behav Med 2018;23.doi:10.1093/abm/kay080
    41. ↵
      2. Klein WMP ,
      3. Zajac LE ,
      4. Monin MM
      . Worry as a moderator of the association between risk perceptions and quitting intentions in young adult and adult smokers. Ann Behav Med 2009;38:256–61.doi:10.1007/s12160-009-9143-2
      OpenUrlCrossRefPubMed
    42. ↵
      2. Pepper JK ,
      3. Byron MJ ,
      4. Ribisl KM , et al
      . How hearing about harmful chemicals affects smokers' interest in dual use of cigarettes and e-cigarettes. Prev Med 2017;96:144–8.doi:10.1016/j.ypmed.2016.12.025
      OpenUrl
    43. ↵
      2. Watson D ,
      3. Clark LA ,
      4. Tellegen A
      . Development and validation of brief measures of positive and negative affect: the PANAS scales. J Pers Soc Psychol 1988;54:1063–70.doi:10.1037/0022-3514.54.6.1063
      OpenUrlCrossRefPubMedWeb of Science
    44. ↵
      2. Keller PA ,
      3. Block LG
      . Increasing the persuasiveness of fear appeals: the effect of arousal and elaboration. J Consum Res 1996;22:448–59.doi:10.1086/209461
      OpenUrlCrossRef
    45. ↵
      2. Moodie C ,
      3. Mackintosh AM ,
      4. Hastings G , et al
      . Young adult smokers' perceptions of plain packaging: a pilot naturalistic study. Tob Control 2011;20:367–73.doi:10.1136/tc.2011.042911
      OpenUrlAbstract/FREE Full Text
    46. ↵
      2. Hammond D ,
      3. Fong GT ,
      4. McDonald PW , et al
      . Impact of the graphic Canadian warning labels on adult smoking behaviour. Tob Control 2003;12:391–5.doi:10.1136/tc.12.4.391
      OpenUrlAbstract/FREE Full Text
    47. ↵
      2. Hall MG ,
      3. Peebles K ,
      4. Bach LE , et al
      . Social interactions sparked by pictorial warnings on cigarette packs. Int J Environ Res Public Health 2015;12:13195–208.doi:10.3390/ijerph121013195
      OpenUrl
    48. ↵
      2. Morgan JC ,
      3. Southwell BG ,
      4. Noar SM , et al
      . Frequency and content of conversations about pictorial warnings on cigarette packs. Nicotine Tob Res 2018;20:882–7.doi:10.1093/ntr/ntx180
      OpenUrl
    49. ↵
      2. Hall MG ,
      3. Sheeran P ,
      4. Noar SM , et al
      . Reactance to health warnings scale: development and validation. Ann Behav Med 2016;50:736–50.doi:10.1007/s12160-016-9799-3
      OpenUrl
    50. ↵
      2. Hall MG ,
      3. Sheeran P ,
      4. Noar SM , et al
      . A brief measure of reactance to health warnings. J Behav Med 2017;40:520–9.doi:10.1007/s10865-016-9821-z
      OpenUrlPubMed
    51. ↵
      2. Yong HH ,
      3. Borland R ,
      4. Thrasher JF , et al
      . Mediational pathways of the impact of cigarette warning labels on quit attempts. Health Psychol 2014;33:1410–20.doi:10.1037/hea0000056
      OpenUrlCrossRefPubMedWeb of Science
    52. ↵
      2. Ferrer RA ,
      3. Klein WMP ,
      4. Persoskie A , et al
      . The tripartite model of risk perception (TRIRISK): distinguishing Deliberative, affective, and experiential components of perceived risk. Ann Behav Med 2016;50:653–63.doi:10.1007/s12160-016-9790-z
      OpenUrl
    53. ↵
      2. Brewer NT ,
      3. Chapman GB ,
      4. Gibbons FX , et al
      . Meta-analysis of the relationship between risk perception and health behavior: the example of vaccination. Health Psychol 2007;26:136–45.doi:10.1037/0278-6133.26.2.136
      OpenUrlCrossRefPubMedWeb of Science
    54. ↵
      2. Noar SM ,
      3. Rohde JA ,
      4. Horvitz C , et al
      . Adolescents' receptivity to e-cigarette harms messages delivered using text messaging. Addict Behav 2019;91:201–7.doi:10.1016/j.addbeh.2018.05.025
      OpenUrl
    55. ↵
      . R.J. Reynolds Tobacco Company vs United States Food and Drug Administration. Civil Case No. 11-1482 (RJL): United States District Court for the District of Columbia 2011.
    56. ↵
      2. Kraemer JD ,
      3. Baig SA
      . Analysis of legal and scientific issues in court challenges to graphic tobacco warnings. Am J Prev Med 2013;45:334–42.doi:10.1016/j.amepre.2013.05.004
      OpenUrlCrossRefPubMed
    57. ↵
      2. Hiilamo H ,
      3. Crosbie E ,
      4. Glantz SA
      . The evolution of health warning labels on cigarette packs: the role of precedents, and tobacco industry strategies to block diffusion. Tob Control 2014;23:e2.doi:10.1136/tobaccocontrol-2012-050541
    58. ↵
      2. Baig SA ,
      3. Byron MJ ,
      4. Boynton MH , et al
      . Communicating about cigarette smoke constituents: an experimental comparison of two messaging strategies. J Behav Med 2017;40:352–9.doi:10.1007/s10865-016-9795-x
      OpenUrl
    59. ↵
      2. LaVoie NR ,
      3. Quick BL ,
      4. Riles JM , et al
      . Are graphic cigarette warning labels an effective message strategy? A test of psychological reactance theory and source appraisal. Communic Res 2017;44:416–36.doi:10.1177/0093650215609669
      OpenUrl
    60. ↵
      2. Erceg-Hurn DM ,
      3. Steed LG
      . Does exposure to cigarette health warnings elicit psychological reactance in smokers? J Appl Soc Psychol 2011;41:219–37.doi:10.1111/j.1559-1816.2010.00710.x
      OpenUrlCrossRef
    61. ↵
      2. Maynard OM ,
      3. Attwood A ,
      4. O'Brien L , et al
      . Avoidance of cigarette pack health warnings among regular cigarette smokers. Drug Alcohol Depend 2014;136:170–4.doi:10.1016/j.drugalcdep.2014.01.001
      OpenUrlCrossRefPubMedWeb of Science
    62. ↵
      2. McCloud RF ,
      3. Okechukwu C ,
      4. Sorensen G , et al
      . Cigarette graphic health warning labels and information avoidance among individuals from low socioeconomic position in the U.S. Cancer Causes Control 2017;28:351–60.doi:10.1007/s10552-017-0875-1
      OpenUrl
    63. ↵
      2. Hall MG ,
      3. Marteau TM ,
      4. Sunstein CR , et al
      . Public support for pictorial warnings on cigarette packs: an experimental study of US smokers. J Behav Med 2018;41:398–405.doi:10.1007/s10865-018-9910-2
      OpenUrl
    64. ↵
      2. Hall MG ,
      3. Sheeran P ,
      4. Noar SM , et al
      . Negative affect, message reactance and perceived risk: how do pictorial cigarette pack warnings change quit intentions? Tob Control 2018;27:e136–42.
      OpenUrlAbstract/FREE Full Text
    65. ↵
      2. Cho YJ ,
      3. Thrasher JF ,
      4. Yong H-H , et al
      . Path analysis of warning label effects on negative emotions and quit attempts: a longitudinal study of smokers in Australia, Canada, Mexico, and the US. Soc Sci Med 2018;197:226–34.doi:10.1016/j.socscimed.2017.10.003
      OpenUrl
    66. ↵
      2. Thrasher JF ,
      3. Swayampakala K ,
      4. Borland R , et al
      . Influences of self-efficacy, response efficacy, and reactance on responses to cigarette health warnings: a longitudinal study of adult smokers in Australia and Canada. Health Commun 2016;31:1517–26.doi:10.1080/10410236.2015.1089456
      OpenUrl
    67. ↵
      2. Grummon AH ,
      3. Hall MG ,
      4. Taillie LS , et al
      . Increasing the impact of sugar-sweetened beverage health warnings: a randomized experiment. Prev med 2019;121:158–66.
      OpenUrl
    68. ↵
      2. Brewer NT ,
      3. Parada JH ,
      4. Hall MG , et al
      . Understanding why pictorial cigarette pack warnings increase quit attempts [published online ahead of print May 29, 2018]. Ann Behav Med 2018.
    69. ↵
      2. Noar SM ,
      3. Rohde JA ,
      4. Barker JO , et al
      . Pictorial cigarette pack warnings increase some risk appraisals but not perceived risk: a meta-analysis. Under review.
    70. ↵
      2. Berinsky AJ ,
      3. Huber GA ,
      4. Lenz GS
      . Evaluating online labor markets for experimental research: Amazon.com's mechanical Turk. Political Analysis 2012;20:351–68.doi:10.1093/pan/mpr057
      OpenUrlCrossRef
    71. ↵
      2. Weinberg J ,
      3. Freese J ,
      4. McElhattan D
      . Comparing data characteristics and results of an online factorial survey between a population-based and a crowdsource-recruited sample. Sociol Sci 2014;1:292–310.doi:10.15195/v1.a19
      OpenUrl

    Footnotes

    • Contributors NTB had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: NTB, MJ, MGH, SAB, JRM, AJL, SMN, MTK, KMR. Acquisition, analysis or interpretation of data: NTB, SAB. Drafting of the manuscript: NTB, MJ, MGH, JRM, SAB. Critical revision of the manuscript for important intellectual content: NTB, MJ, MGH, SAB, JRM, AJL, SMN, MTK, KMR. Statistical analysis: NTB, SAB. Obtained funding: NTB, KMR. Administrative, technical or material support: NTB, JRM. Study supervision: NTB, JRM.

    • Funding Research reported in this publication was supported by grant number P50CA180907 from the National Cancer Institute and FDA Center for Tobacco Products (CTP). T32-CA057726 from the National Cancer Institute of the National Institutes of Health supported Marissa Hall’s time writing the paper.

    • Disclaimer The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH or the Food and Drug Administration. The funding institutions had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    • Competing interests None of the authors have received funding from tobacco product manufacturers. NTB and KMR have served as paid expert consultants in litigation against tobacco companies. The other authors declare no conflicts of interest.

    • Patient consent for publication Not required.

    • Ethics approval The University of North Carolina institutional review board approved this study.

    • Provenance and peer review Not commissioned; externally peer reviewed.