Article Text

Download PDFPDF

The impact of flavour, device type and warning messages on youth preferences for electronic nicotine delivery systems: evidence from an online discrete choice experiment
  1. Ce Shang1,
  2. Jidong Huang2,
  3. Frank J Chaloupka1,3,
  4. Sherry L Emery4
  1. 1 Health Policy Center, Institute for Health Research and Policy, University of Illinois at Chicago, Chicago, Illinois, USA
  2. 2 School of Public Health, Georgia State University, Atlanta, Georgia, USA
  3. 3 Department of Economics, University of Illinois at Chicago, Chicago, Illinois, USA
  4. 4 Health Media Collaboratory, NORC at the University of Chicago, Chicago, Illinois, USA
  1. Correspondence to Dr Ce Shang, Health Policy Center, Institute for Health Research and Policy, University of Illinois at Chicago, Chicago, IL 60608, USA; cshang{at}


Objective To examine the impact of flavour, device type and health warning messages on youth preference for electronic nicotine delivery systems (ENDS), and to provide evidence and data to inform the Food and Drug Administration’s potential regulatory actions on ENDS.

Design An online discrete choice experiment was conducted in September 2015. Each participant was given nine choice sets and asked to choose one out of two alternative ENDS products, with varying characteristics in three attributes (flavour, device type and warning message). The impact of the attributes on the probability of choosing ENDS was analysed using conditional and nested logit regressions, controlling for individual sociodemographic characteristics and current smoking status.

Setting and participants A general population sample of 515 participants (50 ever-users and 465 never-users of ENDS) aged 14–17 years were recruited to complete the experiment using an online panel.

Results Fruit/sweets/beverage flavours significantly increase the probability of choosing ENDS among youth (p<0.01 for never-users and <0.1 for ever-users) and flavour has the most pronounced impact among three attributes. Among never-users, menthol flavour also increases (p<0.05) the probability of choosing ENDS compared with tobacco flavour. Vaping devices that are modifiable, compared with cigarette-like e-cigarettes, increase (p<0.05) the probability of choosing ENDS among adolescent never-users. Warning messages reduce (p<0.01) the probability of choosing ENDS among never-users.

Conclusions and relevance Restricting fruit/sweets/beverage flavours in ENDS, regulating modifiable vaping devices and adopting strong health warning messages may reduce the uptake of ENDS among youth.

  • electronic nicotine delivery devices
  • public policy
  • priority/special populations
  • packaging and labelling
  • prevention
View Full Text

Statistics from


Recent years have seen a striking increase in the use of electronic nicotine delivery systems (ENDS), particularly among youth and young adults.1 2 According to the National Youth Tobacco Surveys, ENDS use among high school students increased from 1.5% in 2011 to 16.0% in 2015, rendering it the most-used tobacco product by this group.3 ENDS use among middle school students showed a similar, but less dramatic, trend. Nevertheless, 5.3% of middle school students were currently using ENDS in 2015, which was the most-used tobacco product among this group as well.3

The 2009 Family Smoking Prevention and Tobacco Control Act gave the US Food and Drug Administration (FDA) authority to regulate the manufacturing, distribution and marketing of tobacco products, including ENDS. The future FDA’s regulatory decisions will hinge on the net public health impact of ENDS, which is not well studied.1–7 On the one hand, nicotine has adverse effects on youth’s central nervous system8 and ENDS may become a gateway product for youth to transition into combustible tobacco products.9–12 As most tobacco use initiate at a young age,13 14 preventing youth from ever taking up ENDS is the key to future tobacco control and public health.2 On the other hand, from the perspective of the continuum of risks, because ENDS are less harmful than conventional combustible tobacco products, switching to ENDS may reduce the harmful health consequences of smoking.15–18 In the final deeming rule, the FDA considered these two aspects and requested more evidence in each aspect.1


Characterising flavours in cigarettes other than menthol has been banned by the FDA since 2009. Flavours in ENDS, however, are a feature potentially salient to youth use and which remain unregulated.1 There is a great variety of flavours other than menthol or tobacco flavours in ENDS, including fruit, sweets and beverage flavours.19–21 Sweet and fruit flavours have been shown to be very appealing to youth and young adults.21–25 Beverage flavours in ENDS are often mixed drink flavours such as pina colada, mojito and margarita, which are sweet and fruity as well and potentially appealing to young people.26

Flavour is a risk factor associated with ENDS initiation among youth and young adults. Compared with older adults, young people are more likely to use flavoured ENDS at onset and preferred sweet flavours.21 22 For adolescents, flavours are reported as a common reason for them to experiment with or choose ENDS and the primary reason for their first try.27–29 This association between flavour and ENDS onset may be moderated by risk perceptions. Studies show that youth perceive fruit-flavoured and other flavoured ENDS less harmful than tobacco flavoured ones, and are more interested in trying menthol-flavoured, candy/sweet-flavoured and fruit-flavoured ENDS compared with tobacco-flavoured ones.23–25 Marketing of flavoured ENDS may also play a role in use onset among youth. One recent study shows that flavoured ENDS ads were more appealing to youth than non-flavoured ones, and that youth reported greater interest in buying and trying ENDS if they were exposed to ads of flavoured ENDS.30

Nonetheless, characterising flavours are not completely risk-free—they may contain chemicals that irritate respiratory systems.19 20 In addition, several cinnamon-flavoured e-liquids contain a toxic chemical—cinnamaldehyde.31 Menthol-flavoured, coffee-flavoured and strawberry-flavoured aerosols were found to reduce cell viability and metabolic activity, with strawberry flavoured products being the most toxic among all tested flavours.32

Flavoured ENDS use was also associated with susceptibility of cigarette smoking among youth non-smokers,33 suggesting the role of characterising flavours in the possible ‘gateway effect’ of ENDS. However, some evidence suggests that flavours may be an incentive for smokers to switch to ENDS, and thus may help them quit smoking combustible cigarettes.1 34 35 In light of insufficient evidence on this issue, the FDA specifically calls for additional data and research to address the effect of flavours on youth initiation, use and dual use of ENDS and tobacco products.1

Device types

Unlike other tobacco products, ENDS encompass a variety of devices that differ in shapes, sizes and names. According to a systematic review by Glasser et al,36 ENDS devices can be broadly classified into subtypes ‘cigarette-like e-cigarettes’ type (including rechargeable and disposable) and the ‘modifiable vaping products’ type that is rechargeable, including e-GO style (vape pens, vape pipes, e-cigars, e-hookahs and e-pipes) and ‘open tank’ style.37–39 Some e-GO and open-tank style ENDS allow for personalisation of devices and modifications of nicotine levels, as well as mixing of e-juices or flavours.

Studies further show that various device types may have differential appeals to different users. Modifiable or advance types were perceived by current ENDS users to be more effective in helping smoking abstinence and more satisfying.40 Established ENDS users were also more likely to use advance types than to use cigarette-like e-cigarettes.41 In contrast, ever-users of ENDS were more likely to use ‘cigarette-like’ devices than to use advance devices.41 It was also reported that ENDS users commonly initiated with ‘cigarette-like’ devices and later transitioned into using more advanced types.42

Among US adolescents in Connecticut, rechargeable devices were shown to be more prevalent than disposable devices for both first use and current use of ENDS.43 However, it is unclear whether youth initiated ENDS by using ‘cigarette-like’ e-cigarettes or advance types such as e-GO and open-tank styles. In addition, very little is understood about how device types influence ENDS uptake and choices, in relation to other features such as flavours.

Warning messages

Health warning messages are another important attribute that may influence youth choice of ENDS through impacting their belief and perceptions related to tobacco.44 45 A review of the effectiveness of health warning messages concluded that comprehensive warning messages may prevent youth from smoking initiation.46 Recent experiments found similar evidence that Canadian youth and US young adults are less likely to choose ENDS if a health warning message is present.29 47

While the deeming rule requires a nicotine warning statement in ENDS ads and on their packages, the health warning message specified in the deeming rule is limited only to the addictive nature of nicotine.1 There is an additional health warning statement on the FDA Center for Tobacco Products (CTP) website that describes the potential risks and benefits of ENDS as unknown. Moreover, some ENDS products carry voluntary warnings that are deemed stronger than the one the FDA requires.48–50 For example, MarkTen, an ENDS brand owned by Altria Group, has a warning message that contains more risk information about ENDS use, such as ‘very toxic by inhalation’ and ‘increase your heart rate and blood pressure’.48–50 The effects of these warning messages, including the FDA required one and alternative warning messages, on youth uptake and choices of ENDS, remain unknown.51

This study aimed to better understand how different attributes influence youth’s decisions to choose ENDS by simultaneously analysing the impacts of flavours, health warning messages and device types on ENDS choices, using an online discrete choice experiment (DCE). DCE is a stated-preference technique that has been increasingly used in tobacco research in recent years.29 47 52–54 This method also allows us to compare the relative importance of flavours, health warning messages and device types, and thus identify the most salient attribute in youth ENDS choices for regulatory purposes. Finally, given that the existing DCE studies in ENDS use either come from Canada28 or solely focus on the US adult population,29 53 54 this study fills an important research gap by providing the first DCE study on the impact of product attributes on ENDS choices among US adolescents.


A general population sample of 515 adolescents, aged 14–17 years, were recruited through the KnowledgePanel, which used both probability-based and random-digit-dialling sampling to recruit panellists and their youth household members. In September 2015, adolescents were recruited through parents who were panellists and provided consents. Poststratification weights were constructed to adjust for sample design and survey non-responses. Among the 515 adolescents, 50 were ever-users of ENDS and 465 were never-users. The incidence of ever-users is very close to that found in the Population Assessment of Tobacco and Health Wave 1 survey, which was 11%.55

Experiment design

The design (table 1) has one two-level attribute for device type (cigalike e-cigarettes, e-GO/Mods/APVs), one three-level attribute for flavours (tobacco, menthol and fruit/sweets/beverage) and one four-level attribute for warning messages (none, FDA-proposed warning message, FDA CTP warning message and MarkTen warning message from Altria Group). The levels of each attribute were chosen as they are distinctive from each other and may inform different regulatory actions,1 26 36 48–50 and together lead to 24 (2×3×4) possible hypothetical products.

Table 1

Product attributes and corresponding levels

In the next step, Sawtooth software and the Balanced Overlap method were used to select two hypothetical products to form the choice set or pair. For all participants regardless of their ENDS use history, these choice sets contain an opt-out option of not using any hypothetical ENDS. In addition, ever-users were given an additional opt-out option to choose their most-used ENDS product, and thereby chose among two hypothetical products and two opt-out options. An algorithm was used to guarantee that neither of the two hypothetical ENDS products are identical to their-most used ENDS product in those three attributes, while ensuring all 10 versions of choice sets were asked to ever-user participants to preserve the integrity of the design. Examples of these choice sets can be found in figures 1 and 2.

Figure 1

Example of a DCE choice set for ever-users of ENDS. AVP, advanced personal vaporizer; DCE, discrete choice experiment; ENDS, electronic nicotine delivery systems; FDA, Food and Drug Administration.

Figure 2

Example of a DCE choice set for never-users of ENDS. DCE, discrete choice experiment; ENDS, electronic nicotine delivery systems; FDA, Food and Drug Administration.

Finally, the design resulted in 90 unique choice sets with a D-efficiency of 0.98, very close to a fully balanced and orthogonal design that has a D-efficiency of 1. To avoid exhaustion and errors from making too many choices,56 these choice sets were then divided into 10 versions, each containing 9 choice sets, and respondents were randomly assigned to answering 1 out of these 10 versions.

Methods and analyses

Following previous studies,29 47 52–54 conditional logit regressions were used to analyse the effects of flavours, warning messages and device types on the choice of using e-cigarettes. In addition, according to the power analysis described in de Bekker-Grob et al,57 both the sample size of ENDS never-users and that of ever-users exceed the sample size requirement to detect an effect size of 0.1 with a statistical power of 0.8 and a 95% CI. Therefore, we analysed ENDS never-user and ever-user samples separately. Furthermore, as we have a large sample of never-users, nested logit regressions were also used to analyse these attributes for this group.

Compared with conditional logit, nested logit makes the assumption that decision-making takes two steps: participants first choose between the ‘opt-out’ option and hypothetical products, and then, conditional on choosing hypothetical products instead of opting out, choosing between the two hypothetical products. In other words, this method models decision trees with two branches, with one branch leading to opting out and the other leading to the use or uptake of a hypothetical product (see figures 1 and 2).52 Because never-users have not initiated ENDS use, their decision-making is more likely to follow this format, which is the rationale for estimating a nested logit model for never-users.

Two alternative specifications or models were employed to analyse the effects of these attributes on the probability of choosing ENDS. In the first model (model A), the flavour attribute was constructed as an ordinal variable (1=tobacco, 2=menthol, 3=fruit/sweets/beverages), whereas in the second model (model B), it was constructed as a dichotomous variable (0=tobacco and 1=menthol and fruit/sweets/beverage). In addition, in model A, health warning messages were constructed as an ordinal variable (1=no warning messages, 2=FDA deeming warning message, 3=FDA CTP warning message, and 4=MarkTen warning message); in model B, health warning messages were constructed as a dichotomous variable (0=no warning message or FDA deeming messages, and 1=FDA CTP and MarkTen warning messages). In both models, device type was constructed as a dichotomous variable with the value of 1 indicating e-Go/Mods/APVs style ENDS and the value of 0 indicating cigarette-like e-cigarettes.

In addition to these attributes, individual characteristics such as gender, age, race/ethnicity, family income, household size, parent’s education and current smoking status were controlled for in all analyses. Standard errors were clustered at the individual level to take account of the correlation among choices made by the same participant. It is also worth noting that both the conditional and nested logit models intrinsically assume independence of irrelevant alternatives (IIA) that the relative probability of choosing A over B is independent of an additional item C.52 We conducted Hausman and McFadden and likelihood-ratio tests to test this assumption. All analyses were conducted using Stata V.14.


Table 2 provides separate descriptive statistics for key variables by ENDS use status (never-users and ever-users), with detailed definitions of the variables presented in the online supplementary table. For both groups, about half of the sample was of man. On average, the participants were a little over 15 years old, living in a household with four people. The two most frequent parental education level were ‘≤12th grade or no diploma’ (28%) and ‘high school graduate or diploma’ (21%) for ever-users, and ‘high school graduate or diploma’ (29%) and ‘bachelor’s degree’ (25%) for never-users, respectively. The two most frequent family income levels among never-users of ENDS were $85 000–$124 999 (25%) and <$40 000 (24%), whereas among ever-users, they were <$40 000 (52%), and $60 000–$84 999 (19%). The composition of race/ethnicity was similar between the two groups: 50%–56% of both samples were White, non-Hispanic; 21%–28% were Hispanic; 13%–14% were Black, non-Hispanic; and the rest were non-Hispanics of other or multiple races. The only significant difference between the two samples was the prevalence of currently smoking cigarettes, which was 42% among ever-users, whereas among never-users, it was only 4%.

Supplementary file 1

Table 2

Summary statistics (total n=515)

Table 3 shows the results of the analyses for ENDS ever-users for both of the two specifications discussed in the Method section. The sample contains 1800 observations, generating from 50 respondents choosing among four options for nine times (50×9×4). Both models suggest that flavours marginally (p<0.1) increase the likelihood of choosing ENDS. Results from model B further suggest that, compared with tobacco flavour, the fruit/sweets/beverage flavour marginally significantly increases (p<0.1) the probability of choosing an ENDS product, whereas menthol flavour does not. In addition, device types and warning messages do not significantly influence ever-users’ choice of an ENDS product. The IIA test did not reject the null hypothesis that the IIA assumption holds. Results of the analyses for ENDS never-users are presented in table 4. Both conditional and nested logit were analysed in two models. Since each of 465 respondents chose among three options (two hypothetical ENDS products and the opt-out option (not using ENDS)) for nine times, the original sample size was 12 555 (465×9×3). After dropping the non-responses and skipped choices, the final analytical sample contains 12 525 observations. About 84% of youth never-users always chose not to use any product, whereas only 34% of youth ever-users always chose not to use any product. The IIA test did not reject the null hypothesis that the IIA assumption holds at the 5% level.

Table 3

The effects of attributes on the probability of choosing ENDS—ever ENDS users, conditional logit regressions

Table 4

The effects of attributes on the probability of choosing ENDS—never-users of ENDS

For ENDS never-users, conditional logit regression results (column 3) suggest that, compared with tobacco flavour, both menthol (p<0.05) and fruit/sweets/beverage (p<0.01) flavours significantly increase the probability of choosing ENDS, with the latter having a larger impact. Vaping devices that are modifiable, compared with cigarette-like e-cigarettes, increase (p<0.1 in Columns 1 and 3; p<0.05 in columns 2 and 4) the probability of choosing ENDS among adolescent never-users. Warning messages measured in ordinal levels marginally (p<0.1 in columns 1 and 2) reduce the probability of choosing ENDS. In addition, FDA CTP statement and MarkTen warning messages, compared with FDA-proposed warning messages or no warning message, (p<0.01; column 3) significantly decrease the probability of choosing ENDS among never-users.

Although the coefficient estimates in these models do not provide a direct interpretation of the effect size,53 their relative magnitudes illustrate the relative impact of different attributes. A comparison indicates that among never-users, fruit/sweet/beverage flavour has the largest impact on ENDS choices, followed by menthol flavour, FDA-CTP/MarkTen warning messages and modifiable devices.

Conclusions and discussion

Our findings have several important policy implications. First, corroborating the previous findings on the role of characterising flavours in increasing smoking initiation and escalation among youth and young adults,21–25 27–30 we found that flavours likely play a similar role in ENDS uptake and preference among youth. Specifically, both menthol and fruit/sweets/beverage flavours increase the probability that a youth never-user chooses ENDS. In addition, fruit/sweets/beverage flavours may also matter to youth ENDS ever-users. Therefore, regulating characterising flavours could have the potential to reduce youth initiating ENDS.

Modifiable vaping devices, compared with cigarette-like e-cigs, also increase the probability that a youth never-user chooses ENDS. This is consistent with existing evidence that youth ENDS users tend to initiate with rechargeable ENDS.43 This finding further adds to the evidence that youth ENDS never-users show more interest in advanced or modifiable device types for their first try. Unlike adult smokers who may initiate cigarette-like e-cigarettes that mimic cigarettes,58 youth may consider vaping devices more acceptable than cigarette-like e-cigs that may be associated with smoking. Their preference for advanced devices is also independent from their preference for flavours.

Finally, youth never-users were responsive to health warning messages, suggesting that warnings may be effective in deterring youth uptake of ENDS. This finding is consistent with a Canadian study that shows youth non-smokers, compared with youth smokers and adults, are more likely to choose ENDS when there are no warning messages.29 Our results also suggest that, compared with no warnings or FDA deeming warning messages, FDA CTP statement and MarkTen warning messages reduce the probability that a never-user would choose ENDS. This finding is consistent with a recent focus group study showing that participants considered MarkTen warnings to be stronger that a nicotine statement.49 It also corroborates recent evidence that young adult smokers, compared with older adult smokers, are less likely to choose ENDS when exposed to strong health warning messages.47 The combined evidence suggests that the warning message required by the FDA may deter youth uptake of ENDS, but stronger warnings could have a greater impact.

Our research has several limitations. DCE is a method based on hypothetical choices, and thus, it contains bias that deviates from behaviours in reality. About 10% participants completed the survey in <8 min which may render their answers less reliable. Nonetheless, the results and conclusions still hold after dropping these participants who finished the survey relatively quickly. The demographic group was limited to a general population sample of youth aged 14–17 years and did not include young adults who are also at a high risk of initiating ENDS. We also did not include prices in the attributes to reduce the burden on youth participants. But prices can be an important attribute and should be further studied in future youth DCEs. Finally, future studies may consider conducting DCEs among subgroups stratified by gender, age and ethnicities.

Findings from this study nonetheless fill an important research gap by providing the first evidence of simultaneous consideration of flavours, warning messages and device types on youth choices of ENDS in the USA.53 Together with other studies,47 53 our findings shed light on potential consequences of future FDA regulatory actions on ENDS related to flavours, warning messages and device types. From the perspective of population health, a regulation that has a positive impact on both cessation and initiation (increasing the probability of quitting a tobacco product among current users, while simultaneously decreasing the probability of initiation among never-users) is preferred. However, when a regulation imposes opposite impacts on cessation and initiation, the benefits from one outcome should be weighed against the costs from the other outcome.

In addition, how different populations value ENDS attributes differently is centre to the consideration of a regulation. The net public health benefits may be achieved through regulating attributes that are more important to ENDS initiation than to adult smokers’ transition to ENDS, especially when implemented with other policies that incentivise them to quit smoking or switch to ENDS. This implies a potential opportunity to achieve positive public health benefits by regulating characterising flavours, as they are the most important to youth among the three attributes, yet may not be as important to adult smokers in their quitting or switching behaviours.47 53 Similarly, regulating modifiable device types and strong warning messages may be effective in deterring youth never-users from choosing ENDS if these policies do not deter adult smokers to switch from combustible cigarettes to ENDS.49 51 Future studies may focus on product standards and how to better communicate warning messages to youth through stronger or more visible warnings on ENDS packages or in advertisements.

Findings from this study may also inform ENDS regulatory policies in other counties where the products are becoming more popular. In particular, many European Union member states have requested health warnings,59 which may deter youth from initiating ENDS use. Future consideration of regulating charactering flavours may further reduce the likelihood of ENDS initiation among youth in these countries.

What this paper adds

  • Characterising flavours, device type and health warning messages may be associated with youth preference for electronic nicotine delivery systems (ENDS).

  • It is unclear how characterising flavours, device type and health warning messages simultaneously influence ENDS choices among US adolescents.

  • There is no evidence on the relative effects of these three attributes on youth initiation.

  • Characterising flavours have the most pronounced impact among three attributes.

  • Restricting fruit/sweets/beverage flavours in ENDS, regulating device type and adopting strong health warning messages may reduce the uptake of ENDS among youth.


GfK was paid for data collection.


View Abstract


  • Contributors CS: conducted data analysis and drafted the paper; had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. SLE, JH and FJC: contributed to data interpretation and revised the draft. All authors: designed the experiment, reviewed and approved the final version of the manuscript.

  • Funding This study was funded by the NCI U01CA154254 and 1U01CA154248-04.

  • Competing interests None declared.

  • Patient consent Guardian consent obtained.

  • Ethics approval The experiment and survey were conducted with ethics clearance from the institutional review boards/ethics committee at the University of Illinois at Chicago.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.