You are here

Article Text

Article menu
Review
Sensory methods to evaluate perception of flavours in tobacco and other nicotine-containing products: a review
  1. Jennifer K Bernat1,
  2. Kia J Jackson1,
  3. Erna J Z Krüsemann2,3,
  4. Sanne Boesveldt3,
  5. Susan F Rudy1,
  6. Reinskje Talhout2
  1. 1 Center for Tobacco Products, Office of Science, US Food and Drug Administration, Silver Spring, Maryland, USA
  2. 2 Centre for Health Protection, National Institute for Public Health and the Environment, Bilthoven, The Netherlands
  3. 3 Division of Human Nutrition and Health, Wageningen University & Research, Wageningen, The Netherlands
  1. Correspondence to Dr Jennifer K Bernat, Center for Tobacco Products, Office of Science, US Food and Drug Administration, Silver Spring, MD 20903, USA; Jennifer.Bernat{at}fda.hhs.gov

Abstract

Objectives Sensory methods use human senses to evaluate product attributes. This review provides an overview of the types of sensory methods used to evaluate the perception of flavour in tobacco and other nicotine-containing (ToNic) products and to discuss how sensory data could inform flavoured ToNic product policy.

Data sources PubMed, Embase and Web of Science.

Study selection All peer-reviewed studies evaluating ToNic products using a sensory method published before 23 May 2020.

Data extraction Two independent coders completed title/abstract and full-text screening to choose articles for inclusion (Cohen’s kappa=0.85, strong agreement). Each coder completed data extraction on half the articles, recording relevant information (eg, sensory methods used, results). The coders categorised sensory methods and generated overarching themes.

Data synthesis Of 110 articles identified, we included 29 articles containing 35 studies that used sensory methods to investigate ToNic products. The sensory methods included analytic methods such as discrimination and descriptive tests and hedonic methods such as liking tests. Six themes emerged regarding how sensory methods can be used to understand consumer perception and liking of ToNic products and to inform ToNic product policy.

Conclusions The identified studies highlight that sensory data can inform ToNic product policy. Analytic and sensory hedonic ratings can be used to assess a ToNic product’s ability to promote addiction in the user (ie, abuse liability). Lastly, hedonic ratings can provide information to assess potential use behaviours.

  • public policy
  • nicotine
  • electronic nicotine delivery devices
  • non-cigarette tobacco products
  • hand-rolled/RYO tobacco

https://doi.org/10.1136/tobaccocontrol-2021-056681

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Data from the 2019 National Youth Tobacco Survey indicate that 69.6% (approximately 4.3 million) middle and high school current tobacco users in the USA report using flavoured (ie, menthol (mint), alcohol (wine or cognac), candy, fruit, chocolate or other sweets) products, such as cigarettes, cigars, electronic cigarettes (e-cigarettes), smokeless tobacco, hookah and pipe tobacco.1 Available data from other regions also support that flavours other than tobacco are popular among youth. In the European Union (EU), younger respondents are much more likely to prefer fruit-flavoured e-cigarettes (72% compared with 17% of the oldest cohort) and somewhat more likely to prefer candy-flavoured e-cigarettes (22% compared with 11%).2 Also, the market for flavoured capsule cigarettes has increased globally, with five Latin American countries (Chile, Peru, Guatemala, Mexico and Argentina) having the highest market share for these tobacco products.3 4 Research shows that younger individuals are more likely to use flavoured capsule cigarettes,5 and youth rate flavoured capsule cigarettes as more attractive than non-capsule cigarettes and indicate more willingness to try them.6

    Flavours play a key role in how users and non-users, particularly youth, initiate and continue using tobacco and other nicotine-containing (ToNic) products.7 Longitudinal data from the US Population Assessment of Tobacco and Health Study (2013–2015) show that initiating with a flavoured product can lead to regular use for youth (ages 12–17), young adults (ages 18–24) and adults (ages 25+).8 Research also shows that flavours influence product appeal and harm perceptions. Flavours can modulate the sensory (eg, sweetness, cooling, irritation) and reinforcing effects of ToNic products, thereby facilitating product use and potentially increasing product abuse liability.9–11 Some flavours may alter product use behaviour and nicotine exposure in users, which influences product abuse liability.12–15 Additionally, data show that youth who currently or had ever used e-cigarettes were more likely than non-users to perceive flavoured e-cigarettes as less harmful than non-flavoured e-cigarettes.16

    Eleven countries and the EU have some type of flavoured tobacco regulation.17 In the USA, the Family Smoking Prevention and Tobacco Control Act (FSPTCA) outlines flavour regulations for cigarettes and roll-your-own (RYO) tobacco. The FSPTCA prohibits a cigarette or any of its component parts (ie, tobacco, filter, paper) from containing a characterising flavour other than tobacco or menthol. In February 2020, the US Food and Drug Administration (FDA) finalised a guidance prioritising enforcement against flavoured (other than tobacco or menthol) prefilled pod or cartridge-based e-cigarettes. There are currently no flavour restrictions for other tobacco products; however, FDA has taken action against companies that sell or distribute e-liquids imitating packaging for food products that are often marketed and appeal to youth, such as candy, cereal and/or soda.

    In the EU, the Tobacco Product Directive (TPD) also outlines flavour regulations for cigarettes and RYO tobacco and prohibits the marketing of cigarettes and RYO tobacco with a characterising flavour other than tobacco. No overarching flavour regulation currently exists for cigars, hookah, smokeless tobacco or other tobacco products such as heated tobacco. The TPD gives member states the authority to regulate e-liquid flavours.

    Sensory research uses human senses (eg, smell, taste) to investigate perception of product attributes such as flavours. Employing sensory methodologies to understand human perception of flavoured ToNic products could inform ToNic product policies.

    The objectives of our research are twofold: to provide an overview of the types of sensory methods used to evaluate the perception of flavour in ToNic products by systematically reviewing peer-reviewed literature and to discuss how sensory methodologies could inform ToNic product policy.

    Methods

    Data sources

    We worked with an informationist and a panel of sensory research experts to develop the search strings for the literature search. We included general (eg, tobacco) and specific (eg, e-cigarette) tobacco product terms, as well as general (eg, sensory) and specific (eg, chemesthesis) sensory terms. The final search strings for each data source are included in online supplemental file 1. We searched three data sources (PubMed, Embase and Web of Science) with no starting limits on the time period. We completed the search on 22 May 2020. We identified two additional articles from citation tracking.

    Study selection

    Figure 1 depicts the Preferred Reporting Items for Systematic Reviews and Meta-Analyses diagram for this review. To select studies for this review, the first two authors independently completed title/abstract screening of the citations and excluded studies that were not in English, not about tobacco and/or nicotine-containing products, did not use a sensory method, review articles, animal studies and conference abstracts. The coders had strong agreement for title/abstract screening (Cohen’s kappa=0.85).18 The coders discussed any disagreements and made a joint decision on whether to include or exclude the study. Next, the coders completed full-text screening and had moderate agreement (Cohen’s kappa=0.72).18 Again, the coders discussed any disagreements and made a joint decision on whether to include or exclude the study. Twenty-nine articles containing 35 studies are included in this systematic review.

    Figure 1
    Figure 1

    PRISMA flow diagram. From Moher et al.66 For more information, visit www.prisma-statement.org. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

    Data extraction and risk of bias assessment

    After full-text screening, each coder completed data extraction on half of the included articles. Specifically, coders recorded data on the ToNic product studied, research objective, research subjects, study design, sensory methods used, results and the main discussion points from each study. The coders noted any risk of biases, such as type of research (eg, academic), funding source, relevance, strength of evidence (ie, low, medium or high quality) and validity issues. After data extraction, the coders discussed any risk of bias findings and made a final determination to include or exclude the article. The coders did not exclude any articles after the risk of bias assessment; however, they identified industry-funded research as a potential risk of bias. For transparency, industry-funded research is flagged with an asterisk in the review.

    Coded variables

    To categorise the sensory methods used in each study, both coders categorised all sensory methods as analytic or hedonic and then specified the test (eg, descriptive, liking). Analytic methods quantify the chemical and physical properties of products. Examples of analytic methods include discrimination tests, which provide scientific evidence that two products are or are not perceptually different, and descriptive tests, which usually use trained panellists to obtain complete sensory descriptions of products on various attributes, making it possible to identify underlying (ingredient or process) variables.19 Hedonic methods describe the degree of consumer acceptance and preference for products. An example of a hedonic method includes liking tests, which measure how much a participant likes a product.19 When the coders disagreed, they discussed the coding decision until they came to an agreement.

    The first coder reviewed the extracted data and generated themes using a conventional content analysis method, focusing on the contextual meaning of the text.20 This approach is informed by grounded theory where the goal is to broadly describe the phenomenon with no preconceived ideas of categories or themes that might emerge from the data.21 The second coder reviewed the extracted data as well as the suggested themes from the first coder. Both coders discussed and came to an agreement on the final themes that emerged from the studies, as well as the final categorisation of studies by theme.

    Results

    The ToNic products included were cigarettes and RYO tobacco (n=13),22–34 e-cigarettes and e-liquids (n=10),11 35–43 nicotine replacement therapy (NRT; eg, nicotine gum; n=5),44–48 and other products (eg, ‘novel hybrid tobacco vapor product’; n=3).27 30 49

    The sensory methods used included analytic methods such as discrimination and descriptive tests and hedonic methods such as liking tests. While the search identified articles using focus groups, where participants rated or described their perceptions of different ToNic products, we did not include these studies because the researchers did not use sensory methods to collect information on human senses.

    Six themes emerged related to how sensory methods and data can be used to understand consumer perception and liking of ToNic products and to inform ToNic product policy. Table 1 summarises the extracted themes by citation, product tested, sensory method and type of sensory test used.

    View this table:
    Table 1

    Extracted themes

    Theme 1: sensory methods can inform ToNic product development and acceptance

    Similar to food and consumer science research, expert sensory panels (ie, individuals trained to assess attributes) are used in research and development of new ToNic products. In one study, Poynton and colleagues49* used two expert panels and both descriptive and discrimination methods to inform the research and development of a ‘novel hybrid tobacco vapor product’: an electronic vapour device using non-flavoured propylene glycol (PG), vegetable glycerin (VG), water, nicotine and a 130 mg blended tobacco plug. An expert panel (23 panellists) compared the hybrid product with a closed system device using ‘blended tobacco’ e-liquid and confirmed significant sensory differences between them. Another expert panel (eight panellists) used descriptive methods to assess the sensory attributes of the hybrid product, including mechanics (ease of draw), strength, flavour and taste, mouthfeel, and consistency of flavour. Additionally, Lin and colleagues23 conducted a study to inform the research and development of cigarettes with lower toxicant yields. An expert panel (11 men, 2 women) used descriptive methods to assess the sensory attributes (impact, irritation, off-taste, aroma and softness) of tobacco treated to reduce alkaloids. Overall, researchers found relationships between the presence of tobacco alkaloids and sensory attributes. Specifically, they found positive correlations between off-taste, irritation and impact attributes, and negative correlations between alkaloid levels and aroma and softness attributes.

    Sensory panels have also been used to conduct research on making current ToNic products more acceptable to consumers. Yin and colleagues24 characterised the sensory attributes of Chinese faint-scent cigarettes to understand which chemical compounds were responsible for specific sensory attributes in the cigarettes; the knowledge would be helpful for improving the characteristic aroma of Chinese faint-scent cigarettes. The study combined a sensory panel with chemical analyses to show that various attributes are correlated with specific compounds. An expert panel (eight men, four women) used descriptive methods to assess 12 sensory attributes (freshness, scorched, baked, spicy, acidic, sweet, fruity, creamy, flowery, balsamic, fresh-green and herbaceous) of eight different brands of Chinese cigarettes. The researchers then used gas chromatography-mass spectrometry to detect volatile compounds in the mainstream smoke. Using partial least squares regression, the researchers identified 67 volatile compounds correlating to freshness, acidic and flowery aroma attributes. Researchers have also used untrained consumers to complete hedonic and descriptive sensory assessments of product acceptability after using ToNic products. Five clinical laboratory studies researched the acceptability of oral or inhaled NRT products or cigarettes.27 30 46–48 These studies provided sensory information such as liking, satisfaction, mouthfeel and perceived strength of flavour.

    Theme 2: smelling can detect flavour and evaluate liking of ToNic products

    Four manuscripts described methods where participants smelled (instead of used) a ToNic product and completed sensory measures.22 29 * 33 39 Krüsemann and colleagues29 conducted a study where 20 non-smoking women detected menthol odour in cigarettes using discrimination methods. Two studies described training expert panels to smell tobacco products and identify sensory attributes using descriptive methods,22 *33 both for European regulatory purposes. In another study, Krüsemann and colleagues39 compared participants’ hedonic ratings of various flavoured e-liquids while smelling and vaping. The group found that smelling and vaping hedonic measures correlated strongly; therefore, smelling could be used in future studies where recruiting participants to use nicotine-containing products could be unethical (eg, nicotine-naïve individuals, youth).

    Theme 3: individual human factors can influence sensory perception

    Studies found that sensory perceptions of ToNic products are influenced by a variety of individual human factors including tobacco user status, age, sex, taste phenotype and brand preference. Researchers have used hedonic methods to show that liking for some flavours differed across different groups of ToNic product users. For example, adult cigarette smokers reported no difference in liking between fruit-flavoured and sweet-flavoured (ie, cherry, chocolate) versus unflavoured e-cigarettes,35 while young adult current e-cigarette users reported increased appeal with fruit-flavoured e-cigarettes compared with tobacco flavour.42 Other studies have found that menthol flavour (compared with all other flavours) had higher e-cigarette liking scores and improved taste across youth, young adult and adult product users.11 35 42 Additionally, among cigarette smokers, Ashley and colleagues*28 analysed the effect of menthol concentration and noted differences in regular and occasional menthol smokers in Japan and Poland using the following attributes: irritation, throat catch, strength of menthol taste, cooling effect and overall liking. They found that occasional menthol smokers in Japan perceived menthol cigarettes as irritating, and throat catch intensity increased with menthol concentration. Regular menthol smokers from both countries found no difference in perceived irritation between menthol concentrations, reported significant reductions in perceived throat catch intensity at higher menthol concentrations, and indicated increased overall product liking compared with occasional menthol smokers. Krüsemann and colleagues39 recruited both smokers and non-smokers to compare hedonic ratings of smelling and vaping e-liquids. They found that non-smokers’ mean liking ratings were highest for sweet flavours like strawberry, watermelon and caramel, and smokers’ mean liking ratings were highest for mint and peppermint. Despite these suggestive differences, the overall differences in flavour liking between non-smokers and smokers for the individual e-liquid flavours were not statistically significant.

    A smaller body of literature evaluated the influence of sex and genetic differences on sensory perception. Using hedonic measures in a laboratory vaping study, Pang and colleagues40 investigated sex differences in ratings of e-cigarette flavour liking and willingness to use. The results indicated slight preference differences between men and women: men preferred fruit-flavoured e-cigarettes, while women preferred both fruit-flavoured and menthol-flavoured e-cigarettes. Ahijevych and colleagues45 assessed the relationship between individuals’ bitterness perception from their ability to taste n-6-propylthiouracil and sensory experience following use of NRT (ie, nicotine lozenges and inhaler). The n-6-propylthiouracil taste phenotype was not associated with strength of sensation, liking or satisfaction of NRT.

    Research also suggests that brand name preference can alter sensory perception of ToNic products. Two studies assessed the impact of brand names on participants’ sensory evaluation of cigarettes in Australia.31 32 In one study, participants smoked two identical cigarettes: one depicting a premium brand name and one depicting a value (ie, less expensive) brand name. Participants rated the cigarettes using hedonic and descriptive measures.32 Cigarettes smoked from packs displaying premium brand names were rated as having better taste, being less harsh and being less dry than identical cigarettes smoked from packs displaying value brand names. In the second study, participants smoked two identical cigarettes: one depicting a premium brand name and one with the brand name masked.31 Using hedonic and descriptive measures, participants rated the branded cigarette as having more favourable and less stale taste and reported higher purchase intent compared with the masked cigarette.

    Theme 4: flavours can enhance sensory attributes and liking of ToNic products

    Sensory research using both analytic and hedonic measures shows that flavours affect consumer perception of ToNic products. Kim and colleagues43 assessed the relationship between consumer perception of sweetness, coolness, bitterness, harshness and flavour and participants’ liking of e-cigarettes. They found that sweetness and coolness were positively correlated with liking, while harshness and bitterness were negatively correlated with liking. Two studies used descriptive methods to determine that fruit (eg, cherry) flavoured e-cigarettes were perceived as sweeter, smoother and less bitter than unflavoured and tobacco-flavoured e-cigarettes among adult ToNic product users.35 42 In another study of youth e-cigarette users, researchers observed that cooling sensations increased with increased menthol e-liquid concentration relative to no menthol.11 Participants in a study using hedonic methods to assess the efficacy of NRT products rated peppermint and cinnamon flavours as more appealing than plain flavour (ie, unflavoured).48

    Theme 5: sensory methods can identify perceptions of non-flavoured ToNic product attributes

    Our search identified articles that used sensory methods to evaluate perceptions of non-flavoured ToNic product attributes. Nicotine is the primary addictive constituent found in ToNic products50 and is described as bitter tasting.51 52 Nicotine also activates multiple sensory systems53 and produces measurable sensory effects in consumers. Studies using descriptive methods have found that nicotine in e-cigarettes increased perceived throat hit, bitterness, irritation and harshness in users35 36 41 42; these effects of nicotine increased with increasing nicotine concentration. Three studies assessed smokers’ sensory perception of nicotine in cigarettes.25 * 26 34 The studies used analytic methods to provide descriptive measures of sensory attributes (eg, impact to the nose, mouth, chest and throat, smoothness, harshness, strength, and tobacco taste)25 * 26 34 and hedonic measures of satisfaction (ie, liking) following product use.25 * 26 While the results for some sensory attributes did not provide interpretable patterns for determining sensory differences by nicotine yield, product acceptance and liking were significant. The results yielded similar patterns, with the higher nicotine/tar cigarettes having greater acceptance/liking than the low nicotine/tar cigarettes. Jaffe and Glaros34 also used analytic discrimination methods and found that smokers could detect differences between commercial cigarettes of varying levels of nicotine, carbon monoxide and tar.

    Pritchard and colleagues* 26 noted that sensory factors may be as important as nicotine pharmacology in affecting smokers’ ratings of liking and product acceptance. Another study evaluated the effects of nicotine and interactions with menthol inside the mouth. Healthy non-smokers provided descriptive ratings after chewing varying doses of nicotine gum.46 Compared with placebo (ie, 0 mg nicotine gum), gums containing nicotine were associated with greater perceptions of bitterness, warming sensation, burning sensation and strength. The results revealed that nicotine-containing gums (compared with placebo) elevated pain and irritation intensity. Similarly, Leischow and colleagues44 conducted a dose–response study to compare the short-term sensory effects of five nicotine gum formulations (three placebo and two treatment). The results revealed sensory differences between the nicotine formulations.

    Sensory methods have also been used to evaluate the effect of PG and VG content in e-cigarettes on nicotine, overall product liking and other sensory attributes (eg, throat hit, strength of the puffs).38 Using descriptive sensory methods, e-cigarette users detected inhalation sensation differences among different mixtures of PG and VG.

    Theme 6: flavours can interact with nicotine-specific sensory attributes to influence product acceptability, palatability and liking

    Studies support that sensory methods can evaluate how flavours interact with nicotine-specific sensory attributes to impact product acceptability and liking, including reducing nicotine’s harshness and masking its bitter taste.* 26 27 36 37 41 42 46 47 For example, Rosbrook and Green37 used descriptive and hedonic measures to evaluate the interaction of menthol and nicotine in e-cigarettes in current menthol cigarette smokers. Researchers instructed subjects to inhale colour-coded only (ie, no flavour or nicotine concentration information) e-cigarettes and to complete ratings of overall sensation, coolness/cold, harshness/irritancy and liking/disliking. The results showed that a high concentration of menthol increased the overall sensation of e-cigarettes containing a low concentration of nicotine. In addition, participants’ ratings of menthol coolness/cold were higher among higher nicotine concentrations, and menthol reduced the perceived harshness of high nicotine concentrations. Researchers have identified similar findings among young adult current e-cigarette users, where descriptive and hedonic measures identified that nicotine increased bitterness and harshness and reduced appeal and smoothness in e-cigarettes; these effects of nicotine were attenuated by fruit and menthol e-liquid flavours to a greater extent than by tobacco e-liquid flavour.42

    Discussion

    We identified a growing body of literature describing sensory methods used in ToNic product research. Overall, we found 29 peer-reviewed articles containing 35 studies that investigated cigarettes and RYO tobacco, e-cigarettes and e-liquids, NRT, and other products using various sensory methods. The studies used analytic methods to measure descriptive sensory attributes (eg, harshness) and to detect differences, and hedonic methods to measure liking. Extracted themes show that sensory methods can inform both product development and product acceptance, as well as identify individual and product differences in sensory perception. Also, smelling can be used as a proxy for using ToNic products. The results show that (1) flavours can enhance liking of ToNic products, (2) sensory methods can identify perceptions of non-flavoured ToNic product attributes, and (3) flavours can interact with nicotine’s sensory attributes to influence the product’s acceptability, palatability and liking.

    Implications

    Results from studies using sensory methods to evaluate ToNic products can inform policymakers’ knowledge of use behaviours. Analytic data may inform whether ingredient changes (eg, flavours, sweeteners) or changes in nicotine content are perceived by consumers and how those sensory perceptions differ between products. This information can inform regulation, given that sensory perceptions during product use can influence product acceptance and use behaviours.54 Hedonic ratings in conjunction with nicotine pharmacokinetic or exposure assessments can be used to assess a ToNic product’s abuse liability, which informs the likelihood of addiction in current users and non-users who might initiate use. Drug liking is associated with self-administration and has been shown to be the most sensitive and reliable subjective effects measure of abuse liability.55 Generally, drugs with greater positive ratings of liking have greater abuse liability.55

    Sensory findings may also provide insight into potential ToNic product use behaviours, which, from a regulatory perspective, is important for understanding population impact. To understand participants’ perception and use intentions of flavoured ToNic products, studies have combined hedonic ratings (ie, liking) with self-reported measures of appeal (ie, willingness to use again).36 40 42 Sensory data also support that flavours improve the taste and mask the harshness and bitterness of nicotine in ToNic products, which makes it easier to initiate and regularly use them.56 57 Furthermore, studies have combined descriptive and hedonic sensory methods with mouth level exposure*28 and topography measures,38 which allows for an assessment of how sensory perceptions may influence individual product use behaviours. Hedonic ratings of products may reflect their ability to serve as suitable substitutes when evaluating product switching among current product users (eg, the likelihood of cigarette smokers completely switching to e-cigarette use).

    Data from sensory panels assessing flavours can also be used for ToNic regulatory purposes. For example, sensory methods could assess whether an additive imparts a characterising flavour other than tobacco for regulatory and compliance purposes.58 The EU uses sensory methods to evaluate flavours in two consumer products, olive oil59 and butter.60 Expert panels trained to assess several attributes are used to classify the quality grade of oil and to determine butter quality.61 62 Talhout and colleagues58 also discuss key decisions to consider when developing and validating sensory methods to assess flavours for regulatory and compliance purposes. First, one must determine a set of reference products for assessing against. Second, one must define the cut-offs for characterising flavour (eg, is it based on the percentage of the panel that can discern the flavour?). Third, one must decide when the test should occur (ie, before, during or after consumption). Fourth, one must determine the specificity of the characterising flavour description (eg, non-tobacco vs fruit-flavoured tobacco). Fifth, one must decide whether an expert or consumer panel can be used. Sixth, one must decide whether it is necessary to prove the tobacco additives impart the characterising flavour.

    The EU also uses an expert panel complemented by a chemical assessment to assess via smelling whether characterising flavours are present in cigarettes and RYO tobacco. The methodology for sensory analysis is based on a comparison of aroma attributes of the test product with those of reference products.63 The Health Effects Tobacco Composition consortium developed a proof of principle and recommended this concept to the European Commission.64

    As training and maintaining sensory panels are costly, and thus a potential barrier for policymakers, a supranational sensory panel supporting several jurisdictions with sensory data could be considered. This panel could either be based in one location or in several locations worldwide. The latter would allow for intercultural comparisons and could be facilitated by holding testing sessions online. The classic sensory panel testing paradigm would need to be adapted with respect to factors such as the need for a common language, simultaneous testing by all panellists, distribution of samples and the role of the panel leader.

    In addition to evaluating flavours, sensory methods can be included in regulatory policies. In June 2021, Health Canada proposed measures to restrict flavours in vaping products.65 Their proposal included a measure to prescribe sensory attributes standards to prevent a sensory perception (smell, taste or chemesthesis) other than one that is typical of tobacco or of mint/menthol. This sensory standards measure in combination with the other measures (ie, restrictions on promotion/labelling and flavour ingredients including all sugars/sweeteners) would limit consumers’ perceptual experience to tobacco or mint/menthol. Additionally, limiting the promotion/labelling to tobacco or mint/menthol would align the promoted flavour, the ingredients and the user experience.

    Limitations

    We limited our literature search to published literature in peer-reviewed journals; thus, we excluded tobacco industry documents from the Truth Tobacco Industry Documents archive and other non-peer-reviewed sources. Although we identified and included four articles written by industry-funded researchers,* 26 * 28 * 33 * 49 we recognise that the industry may have conducted sensory research that is not represented in this review. Additionally, our search strategy did not include an exhaustive list of potential sensory terms. Consequently, there may be additional sensory research that is not represented in this review. Since our goal was to broadly identify ToNic product research using general sensory methods, future reviews can expand the search to include more specific sensory terms.

    Conclusions

    The studies in this review highlight that sensory methods can inform ToNic product policy. Sensory methods provide information on the ability of flavours and other product attributes to enhance liking of ToNic products and the ability to mask the harshness, irritation and bitterness associated with high nicotine concentrations in products. As ToNic products continue to be used throughout the world and new ToNic products are developed, future research can further inform sensory science’s role in policy.

    What this paper adds

    • Sensory research uses human senses (eg, smell, taste) to investigate perception of product attributes such as flavours.

    • This study provides the first review of sensory methods used to evaluate perception of flavours in tobacco and other nicotine-containing (ToNic) products.

    • Sensory methods can inform knowledge of actual and potential ToNic product use behaviours.

    • Sensory methods can assess whether an additive imparts a characterising flavour other than tobacco.

    • Sensory methods can be included in ToNic product regulatory policies.

    Ethics statements

    Patient consent for publication

    Acknowledgments

    The authors would like to thank Vicky Spitalniak for her assistance in conducting the literature search for the articles included in this review, Deborah Neveleff for her scientific editing of this review, and Kees de Graaf for his input in developing the search strings and feedback on initial drafts.

    References

    1. 1.↵
      2. Wang TW ,
      3. Gentzke AS ,
      4. Creamer MR
      . Tobacco product use and associated factors among middle and high school students - United States, 2019. MMWR Surveil Summ 2019;68:1–22.doi:10.15585/mmwr.ss6812a1
      OpenUrl
    2. 2.↵
      1. European Commission
      . Attitudes of Europeans toward tobacco and electronic cigarettes. 458. Special Eurobarometer, 2017.
    3. 3.↵
      2. Thrasher JF ,
      3. Islam F ,
      4. Barnoya J , et al
      . Market share for flavour capsule cigarettes is quickly growing, especially in Latin America. Tob Control 2017;26:468–70.doi:10.1136/tobaccocontrol-2016-053030 pmid:http://www.ncbi.nlm.nih.gov/pubmed/27329114
      OpenUrlFREE Full Text
    4. 4.↵
      2. Moodie C ,
      3. Thrasher JF ,
      4. Cho YJ , et al
      . Flavour capsule cigarettes continue to experience strong global growth. Tob Control 2019;28:595.doi:10.1136/tobaccocontrol-2018-054711 pmid:http://www.ncbi.nlm.nih.gov/pubmed/30368482
      OpenUrlFREE Full Text
    5. 5.↵
      2. Paraje G ,
      3. Araya D ,
      4. Drope J
      . The association between flavor capsule cigarette use and sociodemographic variables: evidence from Chile. PLoS One 2019;14:e0224217.doi:10.1371/journal.pone.0224217 pmid:http://www.ncbi.nlm.nih.gov/pubmed/31644598
      OpenUrlPubMed
    6. 6.↵
      2. Abad-Vivero EN ,
      3. Thrasher JF ,
      4. Arillo-Santillán E , et al
      . Recall, appeal and willingness to try cigarettes with flavour capsules: assessing the impact of a tobacco product innovation among early adolescents. Tob Control 2016;25:e113–9.doi:10.1136/tobaccocontrol-2015-052805 pmid:http://www.ncbi.nlm.nih.gov/pubmed/27060099
      OpenUrlAbstract/FREE Full Text
    7. 7.↵
      2. Huang L-L ,
      3. Baker HM ,
      4. Meernik C , et al
      . Impact of non-menthol flavours in tobacco products on perceptions and use among youth, young adults and adults: a systematic review. Tob Control 2017;26:709.doi:10.1136/tobaccocontrol-2016-053196 pmid:http://www.ncbi.nlm.nih.gov/pubmed/27872344
      OpenUrlAbstract/FREE Full Text
    8. 8.↵
      2. Villanti AC ,
      3. Johnson AL ,
      4. Glasser AM , et al
      . Association of flavored tobacco use with tobacco initiation and subsequent use among US youth and adults, 2013-2015. JAMA Netw Open 2019;2:e1913804.doi:10.1001/jamanetworkopen.2019.13804 pmid:http://www.ncbi.nlm.nih.gov/pubmed/31642927
      OpenUrlPubMed
    9. 9.↵
      2. Wackowski OA ,
      3. Evans KR ,
      4. Harrell MB , et al
      . In their own words: young adults' menthol cigarette initiation, perceptions, experiences and regulation perspectives. Nicotine Tob Res 2018;20:1076–84.doi:10.1093/ntr/ntx048 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28339669
      OpenUrlPubMed
    10. 10.↵
      2. Audrain-McGovern J ,
      3. Strasser AA ,
      4. Wileyto EP
      . The impact of flavoring on the rewarding and reinforcing value of e-cigarettes with nicotine among young adult smokers. Drug Alcohol Depend 2016;166:263–7.doi:10.1016/j.drugalcdep.2016.06.030 pmid:http://www.ncbi.nlm.nih.gov/pubmed/27426010
      OpenUrlCrossRefPubMed
    11. †11.↵
      2. Krishnan-Sarin S ,
      3. Green BG ,
      4. Kong G , et al
      . Studying the interactive effects of menthol and nicotine among youth: an examination using e-cigarettes. Drug Alcohol Depend 2017;180:193–9.doi:10.1016/j.drugalcdep.2017.07.044 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28915478
      OpenUrlCrossRefPubMed
    12. 12.↵
      2. D'Ruiz CD ,
      3. O'Connell G ,
      4. Graff DW , et al
      . Measurement of cardiovascular and pulmonary function endpoints and other physiological effects following partial or complete substitution of cigarettes with electronic cigarettes in adult smokers. Regul Toxicol Pharmacol 2017;87:36–53.doi:10.1016/j.yrtph.2017.05.002 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28476553
      OpenUrlCrossRefPubMed
    13. 13.↵
      2. St Helen G ,
      3. Shahid M ,
      4. Chu S , et al
      . Impact of e-liquid flavors on e-cigarette vaping behavior. Drug Alcohol Depend 2018;189:42–8.doi:10.1016/j.drugalcdep.2018.04.032 pmid:http://www.ncbi.nlm.nih.gov/pubmed/29879680
      OpenUrlCrossRefPubMed
    14. 14.↵
      2. St Helen G ,
      3. Dempsey DA ,
      4. Havel CM , et al
      . Impact of e-liquid flavors on nicotine intake and pharmacology of e-cigarettes. Drug Alcohol Depend 2017;178:391–8.doi:10.1016/j.drugalcdep.2017.05.042 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28704768
      OpenUrlPubMed
    15. 15.↵
      2. Robinson RJ ,
      3. Hensel EC ,
      4. Al-Olayan AA , et al
      . Effect of e-liquid flavor on electronic cigarette topography and consumption behavior in a 2-week natural environment switching study. PLoS One 2018;13:e0196640.doi:10.1371/journal.pone.0196640 pmid:http://www.ncbi.nlm.nih.gov/pubmed/29718974
      OpenUrlPubMed
    16. 16.↵
      2. Cooper M ,
      3. Harrell MB ,
      4. Pérez A , et al
      . Flavorings and perceived harm and Addictiveness of e-cigarettes among youth. Tob Regul Sci 2016;2:278–89.doi:10.18001/TRS.2.3.7 pmid:http://www.ncbi.nlm.nih.gov/pubmed/27722185
      OpenUrlPubMed
    17. 17.↵
      2. Erinoso O ,
      3. Clegg Smith K ,
      4. Iacobelli M , et al
      . Global review of tobacco product flavour policies. Tob Control 2021;30:373–9.doi:10.1136/tobaccocontrol-2019-055454
      OpenUrlAbstract/FREE Full Text
    18. 18.↵
      2. McHugh ML
      . Interrater reliability: the kappa statistic. Biochem Med 2012;22:276–82.pmid:http://www.ncbi.nlm.nih.gov/pubmed/23092060
      OpenUrlPubMed
    19. 19.↵
      2. Lawless H ,
      3. Heymann H
      . Sensory Evaluation of Food. In: Principles and practices. 2 edn. New York, NY: Springer, 2010.
    20. 20.↵
      2. Hsieh H-F ,
      3. Shannon SE
      . Three approaches to qualitative content analysis. Qual Health Res 2005;15:1277–88.doi:10.1177/1049732305276687 pmid:http://www.ncbi.nlm.nih.gov/pubmed/16204405
      OpenUrlCrossRefPubMedWeb of Science
    21. 21.↵
      2. Glaser B ,
      3. Strauss A
      . The discovery of Grounded theory. strategies for qualitative research. New Brunswick, NJ: Aldine Transaction, 1967.
    22. †22.↵
      2. Krüsemann EJZ ,
      3. Lasschuijt MP ,
      4. de Graaf C , et al
      . Sensory analysis of characterising flavours: evaluating tobacco product odours using an expert panel. Tob Control 2019;28:152–60.doi:10.1136/tobaccocontrol-2017-054152 pmid:http://www.ncbi.nlm.nih.gov/pubmed/29792305
      OpenUrlAbstract/FREE Full Text
    23. †23.↵
      2. Lin S ,
      3. Zhang X ,
      4. Song S , et al
      . Tobacco alkaloids reduction by casings added/enzymatic hydrolysis treatments assessed through PLSR analysis. Regul Toxicol Pharmacol 2016;75:27–34.doi:10.1016/j.yrtph.2015.12.016 pmid:http://www.ncbi.nlm.nih.gov/pubmed/26739812
      OpenUrlPubMed
    24. †24.↵
      2. Yin F ,
      3. Zhang XM ,
      4. Song SQ
      . Identification of aroma types and their characteristic volatile compounds of Chinese faint-scent cigarettes based on descriptive sensory analysis and GC-MS and partial least squares regression. Eur Food Res Technol 2016;242:869–80.
      OpenUrl
    25. †25.↵
      2. Kochhar N ,
      3. Warburton DM
      . Puff-by-puff sensory evaluation of a low to middle TAR medium nicotine cigarette designed to maintain nicotine delivery to the smoker. Psychopharmacology 1990;102:343–9.doi:10.1007/BF02244102 pmid:http://www.ncbi.nlm.nih.gov/pubmed/2251332
      OpenUrlCrossRefPubMed
    26. *†26.↵
      2. Pritchard WS ,
      3. Robinson JH ,
      4. Guy TD , et al
      . Assessing the sensory role of nicotine in cigarette smoking. Psychopharmacology 1996;127:55–62.doi:10.1007/BF02805975 pmid:http://www.ncbi.nlm.nih.gov/pubmed/8880944
      OpenUrlCrossRefPubMed
    27. †27.↵
      2. Rose JE ,
      3. Behm FM ,
      4. Murugesan T , et al
      . Silver acetate interactions with nicotine and non-nicotine smoke components. Exp Clin Psychopharmacol 2010;18:462–9.doi:10.1037/a0021966 pmid:http://www.ncbi.nlm.nih.gov/pubmed/21186921
      OpenUrlPubMed
    28. *†28.↵
      2. Ashley M ,
      3. Dixon M ,
      4. Sisodiya A , et al
      . Lack of effect of menthol level and type on smokers' estimated mouth level exposures to TAR and nicotine and perceived sensory characteristics of cigarette smoke. Regul Toxicol Pharmacol 2012;63:381–90.doi:10.1016/j.yrtph.2012.04.010 pmid:http://www.ncbi.nlm.nih.gov/pubmed/22634246
      OpenUrlCrossRefPubMed
    29. †29.↵
      2. Krüsemann EJZ ,
      3. Cremers JWJM ,
      4. Visser WF , et al
      . The sensory difference threshold of menthol odor in flavored tobacco determined by combining sensory and chemical analysis. Chem Senses 2017;42:233–8.doi:10.1093/chemse/bjw123 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28034900
      OpenUrlPubMed
    30. †30.↵
      2. Levin ED ,
      3. Rose JE ,
      4. Behm F
      . Development of a citric acid aerosol as a smoking cessation aid. Drug Alcohol Depend 1990;25:273–9.doi:10.1016/0376-8716(90)90152-5 pmid:http://www.ncbi.nlm.nih.gov/pubmed/2189719
      OpenUrlCrossRefPubMedWeb of Science
    31. †31.↵
      2. Skaczkowski G ,
      3. Durkin S ,
      4. Kashima Y , et al
      . Influence of premium vs masked cigarette brand names on the experienced taste of a cigarette after tobacco plain packaging in Australia: an experimental study. BMC Public Health 2018;18:295.doi:10.1186/s12889-018-5200-8 pmid:http://www.ncbi.nlm.nih.gov/pubmed/29526164
      OpenUrlPubMed
    32. †32.↵
      2. Skaczkowski G ,
      3. Durkin S ,
      4. Kashima Y , et al
      . Influence of premium versus value brand names on the smoking experience in a plain packaging environment: an experimental study. BMJ Open 2017;7:e014099.doi:10.1136/bmjopen-2016-014099 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28093441
      OpenUrlAbstract/FREE Full Text
    33. *†33.↵
      2. Chambers E ,
      3. Paschke T
      . Validation of a recommended practice for assessing "characterizing flavor" to meet requirements of the EU Tobacco Product Directive (2014/40/EU). J Sens Stud 2019;34.
    34. †34.↵
      2. Jaffe AJ ,
      3. Glaros AG
      . Taste dimensions in cigarette discrimination: a multidimensional scaling approach. Addict Behav 1986;11:407–13.doi:10.1016/0306-4603(86)90019-5 pmid:http://www.ncbi.nlm.nih.gov/pubmed/3812050
      OpenUrlCrossRefPubMedWeb of Science
    35. †35.↵
      2. Mead EL ,
      3. Duffy V ,
      4. Oncken C , et al
      . E-cigarette palatability in smokers as a function of flavorings, nicotine content and propylthiouracil (ProP) taster phenotype. Addict Behav 2019;91:37–44.doi:10.1016/j.addbeh.2018.11.014 pmid:http://www.ncbi.nlm.nih.gov/pubmed/30470456
      OpenUrlCrossRefPubMed
    36. †36.↵
      2. Goldenson NI ,
      3. Kirkpatrick MG ,
      4. Barrington-Trimis JL , et al
      . Effects of sweet flavorings and nicotine on the appeal and sensory properties of e-cigarettes among young adult vapers: application of a novel methodology. Drug Alcohol Depend 2016;168:176–80.doi:10.1016/j.drugalcdep.2016.09.014 pmid:http://www.ncbi.nlm.nih.gov/pubmed/27676583
      OpenUrlCrossRefPubMed
    37. †37.↵
      2. Rosbrook K ,
      3. Green BG
      . Sensory effects of menthol and nicotine in an e-cigarette. Nicotine Tob Res 2016;18:1588–95.doi:10.1093/ntr/ntw019 pmid:http://www.ncbi.nlm.nih.gov/pubmed/26783293
      OpenUrlCrossRefPubMed
    38. †38.↵
      2. Harvanko A ,
      3. Kryscio R ,
      4. Martin C , et al
      . Stimulus effects of propylene glycol and vegetable glycerin in electronic cigarette liquids. Drug Alcohol Depend 2019;194:326–9.doi:10.1016/j.drugalcdep.2018.08.039 pmid:http://www.ncbi.nlm.nih.gov/pubmed/30471584
      OpenUrlPubMed
    39. †39.↵
      2. Krüsemann EJZ ,
      3. Wenng FM ,
      4. Pennings JLA , et al
      . Sensory evaluation of e-liquid flavors by smelling and vaping yields similar results. Nicotine Tob Res 2020;22:798–805.doi:10.1093/ntr/ntz155 pmid:http://www.ncbi.nlm.nih.gov/pubmed/31437266
      OpenUrlPubMed
    40. †40.↵
      2. Pang RD ,
      3. Goldenson NI ,
      4. Kirkpatrick M , et al
      . Sex differences in the appeal of flavored e-cigarettes among young adult e-cigarette users. Psychol Addict Behav 2020;34:303–7.doi:10.1037/adb0000548 pmid:http://www.ncbi.nlm.nih.gov/pubmed/31961168
      OpenUrlPubMed
    41. †41.↵
      2. Pullicin AJ ,
      3. Kim H ,
      4. Brinkman MC , et al
      . Impacts of nicotine and flavoring on the sensory perception of e-cigarette aerosol. Nicotine Tob Res 2020;22:806–13.doi:10.1093/ntr/ntz058 pmid:http://www.ncbi.nlm.nih.gov/pubmed/30997500
      OpenUrlPubMed
    42. †42.↵
      2. Leventhal A ,
      3. Cho J ,
      4. Barrington-Trimis J , et al
      . Sensory attributes of e-cigarette flavours and nicotine as mediators of interproduct differences in appeal among young adults. Tob Control 2020;29:679–86.doi:10.1136/tobaccocontrol-2019-055172 pmid:http://www.ncbi.nlm.nih.gov/pubmed/31852818
      OpenUrlAbstract/FREE Full Text
    43. †43.↵
      2. Kim H ,
      3. Lim J ,
      4. Buehler SS , et al
      . Role of sweet and other flavours in liking and disliking of electronic cigarettes. Tob Control 2016;25:ii55-ii61.doi:10.1136/tobaccocontrol-2016-053221 pmid:http://www.ncbi.nlm.nih.gov/pubmed/27708124
      OpenUrlPubMed
    44. †44.↵
      2. Leischow SJ ,
      3. Sachs DP ,
      4. Hansen MD , et al
      . Nicotine polacrilex dose effects: serum nicotine levels and sensory characteristics. Psychopharmacology 1995;117:125–9.doi:10.1007/BF02245177 pmid:http://www.ncbi.nlm.nih.gov/pubmed/7753957
      OpenUrlCrossRefPubMed
    45. †45.↵
      2. Ahijevych K ,
      3. Tepper BJ ,
      4. Graham MC , et al
      . Relationships of ProP taste phenotype, taste receptor genotype, and oral nicotine replacement use. Nicotine Tob Res 2015;17:1149–55.doi:10.1093/ntr/ntu281 pmid:http://www.ncbi.nlm.nih.gov/pubmed/25542917
      OpenUrlCrossRefPubMed
    46. †46.↵
      2. Arendt Nielsen T ,
      3. Nielsen BP ,
      4. Wang K , et al
      . Psychophysical and vasomotor responses of the oral tissues: a nicotine dose-response and menthol interaction study. Nicotine Tob Res 2016;18:596–603.doi:10.1093/ntr/ntv163 pmid:http://www.ncbi.nlm.nih.gov/pubmed/26242288
      OpenUrlCrossRefPubMed
    47. †47.↵
      2. Jensen TK ,
      3. Andersen MV ,
      4. Nielsen KA , et al
      . Interaction between intra-oral cinnamaldehyde and nicotine assessed by psychophysical and physiological responses. Eur J Oral Sci 2016;124:349–57.doi:10.1111/eos.12279 pmid:http://www.ncbi.nlm.nih.gov/pubmed/27282133
      OpenUrlPubMed
    48. †48.↵
      2. Muramoto ML ,
      3. Ranger-Moore J ,
      4. Leischow SJ
      . Efficacy of oral transmucosal nicotine lozenge for suppression of withdrawal symptoms in smoking abstinence. Nicotine Tob Res 2003;5:223–30.doi:10.1080/1462220031000073270 pmid:http://www.ncbi.nlm.nih.gov/pubmed/12745495
      OpenUrlCrossRefPubMed
    49. *†49.↵
      2. Poynton S ,
      3. Sutton J ,
      4. Goodall S , et al
      . A novel hybrid tobacco product that delivers a tobacco flavour note with vapour aerosol (Part 1): product operation and preliminary aerosol chemistry assessment. Food Chem Toxicol 2017;106:522–32.doi:10.1016/j.fct.2017.05.022 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28576286
      OpenUrlPubMed
    50. 50.↵
      2. Benowitz NL
      . Nicotine addiction. N Engl J Med 2010;362:2295–303.doi:10.1056/NEJMra0809890 pmid:http://www.ncbi.nlm.nih.gov/pubmed/20554984
      OpenUrlCrossRefPubMedWeb of Science
    51. 51.↵
      2. Field J ,
      3. Magoun H ,
      4. Hall V
      . The sense of taste. In: Field J , Magoun H , Hall V , eds. Handbook of physiology neurophysiology. Washington, DC: American Physiological Society, 1959: 507–33.
    52. 52.↵
      2. Carstens E E ,
      3. Carstens MI
      . Sensory effects of nicotine and tobacco. Nicotine Tob Res 2021. doi:doi:10.1093/ntr/ntab086. [Epub ahead of print: 06 May 2021].pmid:http://www.ncbi.nlm.nih.gov/pubmed/33955474
      OpenUrlPubMed
    53. 53.↵
      2. Thuerauf N ,
      3. Markovic K ,
      4. Braun G , et al
      . The influence of mecamylamine on trigeminal and olfactory chemoreception of nicotine. Neuropsychopharmacology 2006;31:450–61.doi:10.1038/sj.npp.1300842 pmid:http://www.ncbi.nlm.nih.gov/pubmed/16123771
      OpenUrlCrossRefPubMedWeb of Science
    54. 54.↵
      2. Carpenter CM ,
      3. Wayne GF ,
      4. Connolly GN
      . The role of sensory perception in the development and targeting of tobacco products. Addiction 2007;102:136–47.doi:10.1111/j.1360-0443.2006.01649.x pmid:http://www.ncbi.nlm.nih.gov/pubmed/17207131
      OpenUrlCrossRefPubMedWeb of Science
    55. 55.↵
      2. Carter LP ,
      3. Griffiths RR
      . Principles of laboratory assessment of drug abuse liability and implications for clinical development. Drug Alcohol Depend 2009;105:S14–25.doi:10.1016/j.drugalcdep.2009.04.003 pmid:http://www.ncbi.nlm.nih.gov/pubmed/19443137
      OpenUrlCrossRefPubMedWeb of Science
    56. 56.↵
      2. Carpenter CM ,
      3. Wayne GF ,
      4. Pauly JL , et al
      . New cigarette brands with flavors that appeal to youth: tobacco marketing strategies. Health Aff 2005;24:1601–10.doi:10.1377/hlthaff.24.6.1601 pmid:http://www.ncbi.nlm.nih.gov/pubmed/16284034
      OpenUrlAbstract/FREE Full Text
    57. 57.↵
      2. Cummings KM ,
      3. Morley CP ,
      4. Horan JK , et al
      . Marketing to America's youth: evidence from corporate documents. Tob Control 2002;11:i5.doi:10.1136/tc.11.suppl_1.i5 pmid:http://www.ncbi.nlm.nih.gov/pubmed/11893810
      OpenUrlAbstract/FREE Full Text
    58. 58.↵
      2. Talhout R ,
      3. van de Nobelen S ,
      4. Kienhuis AS
      . An inventory of methods suitable to assess additive-induced characterising flavours of tobacco products. Drug Alcohol Depend 2016;161:9–14.doi:10.1016/j.drugalcdep.2015.12.019 pmid:http://www.ncbi.nlm.nih.gov/pubmed/26774948
      OpenUrlPubMed
    59. 59.↵
      1. Official Journal of the European Union. European Union CR
      . Amending regulation (EEC) NO 2568/91 on the characteristics of olive oil and olive-residue oil and on the relevant methods of analysis. Official Journal of the European Union, 2013: 31–67.
    60. 60.↵
      1. Official Journal of the European Union. European Union CIRE
      . Amending implementing regulation (EU) 2016/1240 as regards methods for the analysis and quality evaluation of milk and milk products eligible for public intervention and aid for private storage. 61. Official Journal of the European Union, 2018: 11–14.
      OpenUrl
    61. 61.↵
      1. ISO
      . Milk and milk products — sensory analysis — Part 1: general guidance for the recruitment, selection, training and monitoring of assessors, 2009: 22935–1.
    62. 62.↵
      2. Angerosa F ,
      3. Campestre C
      . Chapter 14 Sensory Quality: Methodologies and Applications. In: Handbook of olive oil: analysis and properties. New York: Springer Science Business Media, 2013.
    63. 63.↵
      1. European Commission
      . Methodology for the technical assessment of test products assisting in determining tobacco products with a characterising flavour: application to cigarettes and roll your own products, 2021. Available: https://ec.europa.eu/health/sites/health/files/tobacco/docs/methodology_technical-assessment_test-products_en.pdf [Accessed 18 Mar 2021].
    64. 64.↵
      1. European Commission HETOC Consortium
      . Mapping of best practices and development of testing methods and procedures for identification of characterising flavours in tobacco products. Luxembourg: Publications Office of the European Union, 2016.
    65. 65.↵
      1. Health Canada Tobacco Control Directorate
      . Canada Gazette part I - order amending schedules 2 and 3 to the tobacco and vaping products act (Flavours). Ottawa: Queen’s Printer for Canada, 2021: 2926–99.
    66. 66.↵
      2. Moher D ,
      3. Liberati A ,
      4. Tetzlaff J , et al
      . Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009.doi:10.1371/journal.pmed1000097

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Contributors All authors contributed to conceptualisation, interpretation of results and editing of the manuscript. JKB conducted the literature search with assistance from an informationist. JKB and KJJ performed all coding tasks. JKB, KJJ and RT wrote the first draft. EJZK, SB and SFR provided extensive feedback and editing of all drafts. All authors reviewed and approved the final version.

    • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

    • Disclaimer This publication represents the views of the authors and does not represent FDA/CTP position or policy.

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Author note References with a dagger indicate articles included in the systematic review, while references with an asterisk indicate industry-funded research.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.