Article Text
PDF
Abstract
Introduction In recent years, vaping prevention campaigns have proliferated in response to a surge of e-cigarette use among adolescents in the USA. To date, the research literature has provided minimal guidance as to what vaping prevention message elements have the greatest potential for discouraging vaping, are ineffective or have unintended negative effects. The purpose of the current study was to identify and test a large set of vaping prevention ads used by federal, state, local and non-governmental agencies, examining how objectively coded message elements of vaping prevention messages might affect youth.
Methods A convenience sample of adolescents (N=1501) completed an online survey with each participant rating seven randomly selected vaping prevention ads from a pool of 220 ads on perceived message effectiveness (PME) and vaping appeal. Ads were coded on 37 objective elements in three message categories: themes, imagery and other features. Analyses examined how objective elements predicted PME.
Results Addiction, chemicals, negative health symptoms and effects, and cigarette comparison themes were associated with higher PME, as were graphic images and warning symbols. Industry targeting, environmental impact, flavour themes, images of food and people’s faces were associated with lower PME, as were hashtags, statistics and first-person language or the word ‘teen’. Most elements were not associated with appeal, but ads with a flavour theme were associated with increased vaping appeal.
Conclusion Promising vaping prevention messages focus on the adverse consequences of vaping, use negative imagery and avoid speaking for teens using their vernacular or perspective.
- Electronic nicotine delivery devices
- Nicotine
- Prevention
Data availability statement
Data are available upon reasonable request and upon approval of the study guarantors.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Tobacco use causes an estimated one in five adult deaths in the USA, making it the leading cause of preventable death.1 2 Adolescence is a pivotal developmental window for tobacco initiation, with nearly 9 out of 10 adults who smoke cigarettes daily having first tried cigarettes before the age of 18 years.3 Through concerted prevention and intervention efforts on the part of non-governmental and governmental agencies, cigarette smoking among adolescents has plummeted over the past few decades.4 Dramatic reductions in current cigarette smoking among adolescents have been observed, with decreases from 27.5% in 1991 to 6.0% in 2019.5
Although these trends are encouraging, there has been an alarming surge in e-cigarette use (ie, vaping), with rates among high school students skyrocketing from 4.7% in 2011 to 27.5% in 2019.6 In 2014, e-cigarettes became the most commonly used tobacco product among high school-aged youth,7 and according to the 2021 National Youth Tobacco Survey, more than 2 million youth use e-cigarettes, with 24.6% using daily.8 This vaping epidemic among youth is of significant public health concern9 10 and portends the possible reversal of more than 3 decades of steady declines in adolescent cigarette use and a potentially longer term uptick in the use of other tobacco products.11
Well-designed tobacco prevention communication campaigns are a proven strategy for preventing tobacco use.12 13 To date, the research literature has provided minimal guidance as to what vaping prevention message elements have the greatest potential for discouraging vaping. High-profile vaping prevention campaigns such as the Food and Drug Administration’s (FDA) The Real Cost14 and The Truth Initiative15 have the resources and expertise to anticipate the potential impact of their campaign messages prior to campaign launch, typically using perceived message effectiveness (PME) ratings; however, many adolescent vaping prevention campaigns are produced without the benefit of such intensive pre-testing. More rigorous, comprehensive research is needed to identify basic principles concerning what content constitutes maximally effective vaping prevention messaging that also minimises the potential for unintended negative consequences (eg, inadvertently increasing the appeal of vaping).
The nascent literature on vaping prevention messages provides some hints as to which message elements—such as themes, imagery and other features—may best resonate with youth. For example, message themes focused on chemical constituents (eg, arsenic, lead) and health consequences associated with vaping have been linked with increased PME.16 17 In contrast, messages referencing flavours or social norms may be less effective compared with other messaging strategies.17 18 To date, no comprehensive analysis of vaping prevention message elements has been conducted. Although researchers have reliably found vaping prevention messages with higher PME ratings to be more effective,19 20 studies have yet to examine how message themes, imagery and other features of vaping prevention messages influence perceived effectiveness among adolescents.
The purpose of the current study was to identify and test a large set of vaping prevention ads used by federal, state, local and non-governmental agencies, examining how objectively coded message elements of vaping prevention messages might affect youth PME ratings. Although PME ratings have been critiqued as an inadequate substitute for behavioural outcomes,21 multiple studies in tobacco control22–24 and vaping prevention19 20 have found PME ratings to be a good proxy for message impact on beliefs and behaviours. In this study, our primary aim was to understand what message themes, imagery and other features were associated with higher PME ratings, with a secondary aim to examine whether certain message elements could inadvertently increase the appeal of vaping.
Methods
Participants
Participants were a national sample of US adolescents aged 13–17 years recruited via online panels administered by Qualtrics (N=1501). We specified quotas such that 50% of the sample would be female, 20% African American, 20% Hispanic/Latino and at least 40% susceptible to vaping. Parents were invited to consider the survey opportunity for their children and, if interested, completed the parental consent. Once parental consent was obtained, the parent transferred device access to their child to provide assent and complete the survey.
Procedures
Vaping prevention message stimuli
We used a two-phase search to identify vaping prevention messages to serve as stimuli for the current study.25 The search was conducted from 24 September to 10 October 2020. First, we reviewed all vaping prevention campaign messages from the Vaping Prevention Resource (VPR; vapingprevention.org), an educational open-access resource with downloadable vaping prevention media content collected from federal, state, local and non-governmental public health organisations around the country. Second, we conducted a systematic web search to identify additional potentially eligible messages using the following key terms: ‘vaping prevention’, ‘anti-vaping’, ‘vaping prevention messages’ and ‘vaping prevention ads’. To be included as stimuli, a message had to be (1) focused on vaping prevention; (2) relevant to youth and (3) in a static web or print format. Videos, animated graphics interchange formats (GIFs) and other dynamic media formats were not eligible for inclusion. A total of 220 ads (ie, messages) met inclusion criteria, with 177 messages identified from VPR and 43 messages garnered from the supplemental online search. These messages came from 35 different public health organisations.
Message coding
A detailed account of the message coding process and rubric as well as associated descriptive findings have been previously published.25 Briefly, coding categories and items were determined based on an inductive approach of examining text and image features that varied across the set of messages, informed by the research team’s prior work with tobacco prevention and control messages. Text and visual elements were coded as present or absent. Two coders independently coded the 220 messages on each of the resulting 37 features. Coder agreement ranged between 87% and 100%. Gwet AC1 scores were used to quantify inter-rater reliability—all were >0.75, indicating excellent reliability,26 with 95% (n=35) having Gwet AC1 >0.85. The average number of features identified in a message was 7.50 (SD=2.93), with a median of 7 features and range of 3–19 features. Further details on the frequencies and types of message features can be found elsewhere.25
Online study
Participants took part in the online study conducted from 29 October through 3 December 2020. After answering questions about demographics and tobacco use, each participant rated seven vaping prevention messages randomly assigned from the larger pool of 220 messages. Messages were presented in a random order and participants rated each on PME and vaping appeal. The PME and vaping appeal items were presented directly underneath each ad, and participants could gaze at each ad as long as they wanted while answering these items. Participants subsequently answered additional demographic and vaping-related questions. After completing the survey, a standardised incentive was provided via the Qualtrics panel platform.
Measures
Perceived Message Effectiveness
The primary outcome was PME, assessed using the University of North Carolina PME scale,27 which has been validated in multiple tobacco control studies.28 29 Here, we adapted it for use with vaping prevention messages. The three items were: (1) How much does this ad make vaping seem unpleasant to you? (2) How much does this ad make you concerned about the negative effects of vaping? and (3) How much does this ad discourage you from vaping? Response options were (1) not at all, (2) very little, (3) somewhat, (4) quite a bit, (5) a great deal. The mean score of these items formed a composite PME score (Cronbach’s ɑ=0.93).
Appeal
The secondary outcome of vaping appeal was assessed using the following item: How much does this message make vaping seem appealing to you? Response options were (1) not at all, (2) very little, (3) somewhat, (4) quite a bit, (5) a great deal.
Demographics and tobacco product use
We assessed gender, race, ethnicity, age, parent education and tobacco use in the household. Details on these items can be found in table 1. Tobacco product use measures were primarily derived from an ongoing nationally representative study of adolescent tobacco use.30 Prior to assessing e-cigarette use, we provided a brief description of e-cigarettes and vaping accompanied by images of vaping devices. Participants then indicated if they had used e-cigarettes in the past 30 days. If so, they were classified as current users. If participants indicated no past 30-day e-cigarette use, then they answered a set of five e-cigarette susceptibility items, which have been used in prior research.31 If participants answered anything other than ‘definitely not’ to all five susceptibility items, they were classified as ‘susceptible’. Remaining adolescents were classified as ‘non-susceptible’, resulting in three vaping groups, a classification approach used in other tobacco messaging research with adolescents.32 Current use of other tobacco products was assessed by having adolescents identify other tobacco products they had used in the past 30 days, such as cigarettes, traditional cigars, hookah and little cigars and cigarillos.
Sample demographics, N=1501 adolescents
Data analysis
The purpose of the analysis was to identify vaping prevention message elements that were associated with PME as well as those that were associated with vaping appeal. To that end, we tested a series of multilevel models examining the potential effect of each message element, controlling for key demographic factors. An overall model that included all participants as well as models with participants stratified into one of the three e-cigarette user groups—current user, susceptible or non-susceptible, non-user—were tested. We conducted the analyses using maximum likelihood estimation, with the intercept treated as random and two-tailed critical alpha 0.05. Text and tables report findings as unstandardised regression coefficients adjusted for gender, age, race, ethnicity, maternal education, cigarette use and use of one or more other tobacco products. Unadjusted models yielded comparable results.
Results
Participant characteristics
The sample ranged in age from 13 to 17 years (M age=15.3; SD=1.39) and was split approximately equally on gender (51.6% female; table 1). Although a predominantly white sample (70.2%), a sizeable minority identified as African American (22%) and/or Latinx (20.4%). In our sample, 43.8% (n=658) had ever tried e-cigarettes. Of those who had ever tried e-cigarettes, 70.5% (n=464) had used in the past 30 days, 25.8% (n=170) were susceptible and used >30 days prior, and 3.6% (n=24) were non-susceptible and used >30 days prior. When considering the total number of adolescents classified as susceptible (n=649), 73.8% had never tried e-cigarettes. Compared with the e-cigarette susceptible and non-susceptible participants combined, a greater proportion of e-cigarette users were male (60.7% vs 43.0%), white (79.3% vs 66.3%), had maternal education of master’s degree or higher (22.6% vs 14.4%) and were current cigarette smokers (60.3% vs 4.5%).
Perceived message effectiveness
Perceived effectiveness across the set of ads
Each of the 220 messages was rated between 40 and 54 times, and scores ranged from 1.93 (SD=1.19) to 4.57 (SD=0.79). Mean PME across the 10 507 ratings was 3.31 on a five-point scale (SD=1.34); the median was 3.32, with a score of 3 indicating a message is ‘somewhat’ effective.
Message themes
Message themes about nicotine addiction (b=0.25), chemicals (b=0.42), negative health effects (b=0.40), health-related symptoms (b=0.37), comparison to cigarettes (b=0.31), not harmless water vapour (b=0.17), death (b=0.25), unknown health effects (b=0.25) and gateway effects (b=0.55) were associated with greater PME (p <0.05; table 2). Industry targeting (b=−0.23), flavours (b=−0.25) and environmental impact (b=−0.32) themes, on the other hand, were associated with lower PME.
Multilevel models predicting perceived message effectiveness (PME) for message themes
We observed some differences with message theme effects by vaping use/susceptibility group. Although the nicotine addiction theme was perceived as effective by all vaping groups, using the word ‘addiction’ was associated with higher PME only for non-users. Content on lung effects was associated with higher PME for users (b=0.16) but not for non-users. The opposite was observed for brain effects and not harmless water vapour; these were associated with higher PME for susceptible and non-susceptible non-users (p <0.05) but not current users. References to death were associated with increased PME for users (b=0.31) and susceptibles (b=0.27, p <0.05) but not for non-susceptibles.
Message imagery
Imagery associated with higher PME included warning symbols (b=0.36), graphic images (b=0.73) and combustible cigarettes (b=0.30; p<0.05; table 3). The nicotine chemical symbol was associated with increased PME for current users (b=0.34) and susceptibles (b=0.47, p<0.05) but not the non-susceptibles. Faces, both adult (b=−0.15) and teen (b=−0.20), as well as food (b=−0.38), showed consistently negative PME effects (p<0.05). Vaping device imagery overall appeared to have no effect on PME (b=−0.03), with no consistent pattern of results across specific product types (table 3).
Multilevel models predicting perceived message effectiveness (PME) for message imagery
Other message features
Use of a hashtag, statistic, bright/vivid colours, memes and use of first-person (‘I’ or ‘we’) or the word ‘teen’ were all associated with lower levels of PME (p <0.05; table 4). Messages citing a specific study or using the term ‘you’ were associated with higher PME (p <0.05).
Multilevel models predicting perceived message effectiveness (PME) for other message features
Appeal
Mean appeal scores ranged from 1.03 (SD=0.69) to 2.52 (SD=1.31). Mean appeal score was 1.76 on a five-point scale (SD=0.69) and the median was 1.00, with a score of 2 indicating a message has ‘very little’ appeal.
The influence of theme on vaping appeal was largely null or in the negative direction (online supplemental table 1). The sole exception was flavour content; inclusion of this thematic element was associated with increased levels of vaping appeal overall (b=0.13, p <0.05) and across all three vaping groups. The influence of imagery on vaping appeal was also largely null. Food was associated with higher vaping appeal (b=0.15, p<0.05), although this effect varied by vaping group such that there was no observed effect for current users and a modest positive effect for non-users, especially susceptibles (online supplemental table 2). No consistent pattern of effects was observed with respect to the influence of the other features on vaping appeal (online supplemental table 3).
Supplemental material
Discussion
The purpose of this study was to examine how objective features of vaping prevention messages predict adolescents’ perceptions of those messages—that is, PME and vaping appeal. Results reveal that elements that were similar to more traditional tobacco prevention and control messaging—chemical constituents, negative health effects, graphic images and warning imagery—showed the largest and most consistent improvements in PME.16 33 34 Our findings also reveal that likening e-cigarettes to cigarettes consistently appears to improve PME. These findings dovetail with studies showing that negative perceptions of cigarette smoking are pervasive among adolescents,35 and thus associating vaping with smoking may be a fruitful messaging strategy. Care must be taken, however, not to create a false equivalency among the two products. The health risks of combustible tobacco use are extensive and well-documented,2 and messages should not imply that e-cigarettes pose an equivalently serious risk.
Not all health effect message features had equal effects among groups. For example, using the word ‘addiction’ increased PME for susceptible and non-susceptible non-users but not for current users. Referencing lung effects increased PME for current users but not for susceptible or non-susceptible non-users. The opposite was observed for brain effects and not harmless water vapour; these themes increased PME for susceptible and non-susceptible non-users but not for current users. Linking death to vaping increased effectiveness for users and susceptibles but not for non-susceptible non-users. These findings suggest that certain message features may be advantageous to emphasise in contexts where messages can be targeted to subgroups, although a lack of effect among certain subgroups (eg, non-susceptibles) is not necessarily problematic if messages are effective for the at-risk groups that are reached through large-scale campaigns (eg, users, susceptibles).
We infer from these findings that the immediacy of potential vaping harms makes especially compelling message content. Users appear to be not as concerned about health risks that they have already been exposed to (eg, addiction) but instead may be more concerned about credible unknown or future potential health effects (eg, lung effects, death). By extension, we speculate that long-term effects of vaping are largely hypothetical to non-users, which is why they tend to be concerned about more immediate harms of vaping (eg, cognitive effects).
Framing of seemingly equivalent message content can affect PME. For example, although referring to chemicals was generally linked with increased effectiveness, content contextualising toxicity of these chemicals with ‘also found in’ language (eg, ‘Arsenic, also found in bug spray‘) did not. Using the word ‘addiction’ was not linked with increased effectiveness for users; however, content and symbols about nicotine were associated with higher PME for users. In effect, message features that may seem equivalent to message designers (the word ‘nicotine’, nicotine symbol) may not be perceived similarly by adolescents. Therefore, those creating vaping prevention messages should carefully consider the relevance and specificity of content. We also suggest inclusion of multiple effective message elements and use of pre-testing before message deployment. A lack of pre-testing could result in wasted resources, loss of organisational credibility with the target audience and unintended negative message effects.
The presence of a vaping device, accessory, vapour, or person vaping showed minimal or no effects, especially for users. These findings, therefore, do not support the view that the presence of vaping imagery reduces the efficacy of vaping prevention messages. Some experimental research has suggested that vaping cues could increase the urge to vape or smoke cigarettes for smokers36 37; additional research on this issue is needed. Notably, imagery ubiquitous to many social media platforms currently favoured by adolescents, such as animals, memes, foods and vivid colours, did not increase, and in many cases decreased, PME. Faces, both teen and adult, and language attempting to ‘speak for’ the teen all reduced the effectiveness of the message. Taken together, these findings suggest that vaping prevention messages trying to communicate using teen vernacular or perspective may be counterproductive.
Attractive imagery not typically used in tobacco prevention, namely food and flavours, appeared to increase the appeal of vaping, especially for susceptible non-users. To a lesser extent, the presence of faces, the word ‘teen’ and bright/vivid colours were linked with greater vaping appeal for susceptible, non-users. Our interpretation of these data is that prevention messages should be designed in ways that associate vaping with negative—not positive—imagery.
Finally, it is worth noting that the average PME message rating was a 3.31—which is just above ‘somewhat’ effective on a five-point scale. The largest effect sizes observed for PME and appeal were a shift in half a point, which is a meaningful difference. These findings should serve as a clarion call to tobacco prevention researchers and public health officials alike, indicating that the development of more effective vaping prevention messages is both imperative and possible. Indeed, a set of prevention messages that even moderately improves vaping-related outcomes has real potential to move the needle on vaping prevention at the public health level when implemented at scale.
Strengths and limitations
This is the first study that we are aware of that has systematically decomposed the existing opus of vaping prevention messages into its component parts. Study strengths include the use of a large set of messages employed by public health practitioners, coding of objective message elements combined with adolescent perceptions of those messages and a large national sample of adolescents. Notably, our results echo those from a recent large set of in-depth interviews and other qualitative work evaluating creative concepts for adolescent vaping prevention campaigns.17
Several study limitations should be noted. Findings are qualified by the type, number and framing of features included in this pool of real-world messages. Stimuli were limited to static prevention campaign ads. Perceptions of more dynamic media, such as GIFs and videos, might show a different pattern of effects. The large number and varied combinations of message features necessitated each feature being tested separately in models adjusted for key demographic characteristics. Because of this analytic approach, we cannot draw conclusions as to what may be the most effective combination of message features. Some message features may also be confounded with others (eg, the word ‘nicotine’ and nicotine symbol in the same message). Experiments isolating the effects of specific key features are warranted. Effects of features represented by only a small subset of messages should be interpreted with caution. It is unclear whether the pattern of effects observed with this sample would hold for other populations, such as young adults. Despite these limitations, our findings are highly relevant to the development of maximally effective vaping prevention messaging for adolescents, especially in light of the study’s rigorous nested stimuli design,38 systematic coding of messages features and ecological validity.
Public health implications
The goal of this study was to determine what message elements in existing vaping prevention ads are associated with higher PME ratings. Of particular concern was the finding that message content referencing flavours not only decreased PME but appeared to heighten vaping appeal, including for non-susceptible, non-users. We have identified a set of vaping prevention messaging principles intended as a guide for public health researchers and practitioners, a summary of which is provided in table 5. We urge those creating vaping prevention campaigns to base their messaging strategies on the emerging research literature and to prioritise pre-testing vaping prevention messages before implementation. We also call for more systematic message testing approaches for identifying novel and effective communication strategies to reduce adolescent vaping.
Guiding principles for vaping prevention messages for adolescents
Key messages
What is already known on this subject
Tobacco prevention campaigns have been an effective tool for reducing youth smoking. It is unclear what types of messages are most effective for campaigns that attempt to reduce youth vaping or whether certain message features could result in negative unintended consequences.
What this study adds
Promising vaping prevention messages focus on the adverse consequences of vaping, use negative imagery and avoid using teens’ vernacular. Flavour content and imagery may inadvertently increase the appeal of vaping and should be avoided in prevention messages.
How this study might affect research, practice or policy
This research provides a set of principles that can be used to create more effective vaping prevention messages geared to adolescents.
Data availability statement
Data are available upon reasonable request and upon approval of the study guarantors.
Ethics statements
Patient consent for publication
Ethics approval
All procedures were approved by the UNC Institutional Review Board (#20-0979). This study involves human participants but University of North Carolina at Chapel Hill exempted this study. Participants gave informed consent to participate in the study before taking part.
Acknowledgments
We thank Kurt Ribisl and Hannah Prentice-Dunn for their contributions to Vaping Prevention Resource, which provided messages for the current study.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors MHB contributed to conceptualisation, methods, statistical analyses, original draft preparation; NS and SMN to conceptualisation, methods, original draft preparation; WB and AK, methods; ELS and PS, conceptualisation. All study authors provided substantive comments to the manuscript and approved the final version. MHB and SMN are the guarantors of this work.
Funding This work was supported the National Institute on Drug Abuse and the FDA Centre for Tobacco Products (CTP) [R01DA049155]. This work was supported in part by Vaping Prevention Resource, a project at UNC Lineberger Comprehensive Cancer Centre and its Connected Health Applications & Interventions (CHAI) Core. These are respectively supported by the University Cancer Research Fund and NCI P30 CA16086.
Disclaimer The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH or the Food and Drug Administration.
Competing interests SMN has served as a paid expert witness in litigation against tobacco and e-cigarette companies.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.