You are here

Article Text

Article menu

Download PDFPDF

Original research
Understanding e-cigarette content and promotion on YouTube through machine learning
  1. Grace Kong1,
  2. Alex Sebastian Schott1,
  3. Juhan Lee1,
  4. Hassan Dashtian2,
  5. Dhiraj Murthy2
  1. 1 Psychiatry, Yale School of Medicine, New Haven, Connecticut, USA
  2. 2 The School of Journalism, The University of Texas at Austin, Austin, Texas, USA
  1. Correspondence to Dr Grace Kong, Psychiatry, Yale School of Medicine, New Haven, CT 06519, USA; grace.kong{at}yale.edu

Abstract

Introduction YouTube is a popular social media used by youth and has electronic cigarette (e-cigarette) content. We used machine learning to identify the content of e-cigarette videos, featured e-cigarette products, video uploaders, and marketing and sales of e-cigarette products.

Methods We identified e-cigarette content using 18 search terms (eg, e-cig) using fictitious youth viewer profiles and predicted four models using the metadata as the input to supervised machine learning: (1) video themes, (2) featured e-cigarette products, (3) channel type (ie, video uploaders) and (4) discount/sales. We assessed the association between engagement data and the four models.

Results 3830 English videos were included in the supervised machine learning. The most common video theme was ‘product review’ (48.9%), followed by ‘instruction’ (eg, ‘how to’ use/modify e-cigarettes; 17.3%); diverse e-cigarette products were featured; ‘vape enthusiasts’ most frequently posted e-cigarette videos (54.0%), followed by retailers (20.3%); 43.2% of videos had discount/sales of e-cigarettes; and the most common sales strategy was external links for purchasing (34.1%). ‘Vape trick’ was the least common theme but had the highest engagement (eg, >2 million views). ‘Cannabis’ (53.9%) and ‘instruction’ (49.9%) themes were more likely to have external links for purchasing (p<0.001). The four models achieved an F1 score (a measure of model accuracy) of up to 0.87.

Discussion Our findings indicate that on YouTube videos accessible to youth, a variety of e-cigarette products are featured through diverse videos themes, with discount/sales. The findings highlight the need to regulate the promotion of e-cigarettes on social media platforms.

  • Advertising and Promotion
  • Electronic nicotine delivery devices
  • Media
  • Social marketing
  • Surveillance and monitoring

Data availability statement

Data are available upon reasonable request. Data used from this study are publicly available data from YouTube. However, we can provide data upon reasonable request.

https://doi.org/10.1136/tobaccocontrol-2021-057243

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    Electronic cigarette (e-cigarette) use among youth is an epidemic in the USA.1 In 2021, past-month e-cigarette use among US high school students was 11.3%, with 43.6% of these students using e-cigarettes on 20 or more days in the past month. E-cigarette use among youth is also high around the globe.2–4 A main source of youth appeal of e-cigarettes may stem from how these products are portrayed on social media. Social media is mostly unregulated and provides a unique opportunity for the e-cigarette industry to market and promote their products. E-cigarettes are promoted on social media as part of the youth culture, with themes that appeal to youth, such as ‘vape tricks’ (ie, blowing large amounts of clouds or shapes using exhaled aerosol)5 and images that are visually appealing, which could elicit youth to engage with the content by liking or sharing it on their own social network.6 Pro-e-cigarette content on social media has the potential to shape positive social norms surrounding use among youth. Indeed, experimental and observational studies showed that youth who use social media frequently are more willing to try, have positive attitudes towards, have less harm perceptions to and are more likely to initiate e-cigarettes.7–9 Thus, surveillance of e-cigarettes on social media is critical to inform tobacco control efforts.

    In this study, we examined YouTube videos to understand how and which e-cigarette products are promoted to youth, who are promoting these content and how e-cigarettes are being sold. YouTube is a social media platform that allows users to view, upload, post comments and rate videos. YouTube use is steadily growing and is currently used by 2.1 billion users around the globe.10 YouTube continues to be popular among the youth despite other emerging social media platforms; recent data show that 77% of US youth use YouTube.11

    Most studies that examined e-cigarettes on YouTube used human coders. However, human coding is limited by the number of videos that could be analysed, examining 22–350 videos. YouTube provides a plethora of data that cannot be fully examined using current methods. One study identified that YouTube had 28 000 videos on e-cigarettes, posted by 10 000 unique accounts, viewed over 100 million times and commented on more than 280 000 times.12 Advances in methods such as machine learning (ML) can enhance current methods to analyse more content than with human alone.

    A recent scoping review found that 32 of 74 studies that used ML to understand tobacco content examined social media.13 Most of the studies in this review examined Twitter and only one study examined YouTube, which identified that 97% of YouTube videos had protobacco content.14 However, less known is whether ML could be used to identify complex content relevant for surveillance, such as the video content, e-cigarette products, video uploaders and presence of discount/sales. Video themes such as instructions on ‘how to’ modify an e-cigarette or how to conduct vape tricks could reveal trends in use, which may have implications for product regulation and health messaging. For example, the US Food and Drug Administration can prohibit open-system e-cigarette products if these products are used in unsafe ways.15 Furthermore, identifying the features (eg, flavours, nicotine concentrations and other additives) that are marketed in popular ‘product review’ videos can provide insight into product characteristics and marketing strategies that appeal to youth. Identifying video themes with high engagement could also provide insight into advertising targets and wants and needs of users who are engaging with this content.

    To identify who and how e-cigarettes are promoted to the youth, identifying the video uploader (ie, channel) and the presence of discount/sales on videos accessible to youth are important. There could be a combination of promotional strategies used to increase user engagement: paid media (ie, promotion through paid advertisements), earned media (ie, promotion through a third party, such as an influencer) and owned media (ie, promotion through a company’s own website/social media account).16 Furthermore, it is widely known that social media platforms, including YouTube, custom-tailor their content to user characteristics, but their algorithm is proprietary.

    To address these gaps, we created fictitious viewer profiles separated by gender (male/female), age group (16 years old, 24 years old) and race/ethnicity (white, black, Hispanic) to mimic youth searching for e-cigarettes on YouTube. Additionally, to identify e-cigarette content and source of these videos, we used supervised ML to classify (1) video themes, (2) featured e-cigarette products, (3) channel type and (4) discount/sales (see table 1 for definitions). We also assessed whether video themes differed by featured e-cigarette products and channel type, and whether the presence of discount/sales differed by video themes, featured e-cigarette products and channel type.

    View this table:
    Table 1

    Definition of constructs classified by supervised machine learning

    Methods

    Search procedures

    We created 16 fictitious youth profiles to simulate youth searching for e-cigarettes on YouTube.17 For each fictitious youth profile, we used Orbot, a mobile app that uses an anonymised ‘Tor’ network, which is a set of servers around the world that relay traffic through each other to obfuscate Internet Protocol (IP) address and other information such as geographical location.18 Through this app, we searched the following words on YouTube in June 2020: ‘e-cigarette’, ‘e-cig’, ‘electronic cigarette’, ‘e-liquid’, ‘ENDS’, ‘e-juice’, ‘vape’, ‘vaping’, ‘vape juice’, ‘box mods’, ‘cigalikes’, ‘disposable e-cigs’, ‘disposables’, ‘disposable vape’, ‘pod mods’, ‘vape mods’, ‘vape pens’ and ‘vape pods’. Also using Orbot, we scraped 140 videos, which is equivalent to the number of videos shown on the first seven pages for each search word. For each fictitious profile, we used a new Subscriber Identity Module (SIM) card and a phone number and performed a factory reset of the Android phone to conduct these searches to ensure that previous search did not affect the search outcomes. We obtained 4201 unique videos and extracted metadata such as the title, description of the video, view counts, number of likes/dislikes, comments, date of when the video was posted and the channel name from the YouTube application programming interface (API; an application that allows access to data from a service/programme such as YouTube).

    Supervised ML procedures

    The overall ML procedures involved (1) human labelling, (2) text preprocessing, (3) training the supervised ML classification algorithms and (4) evaluating the algorithm’s performance.

    Human labelling

    We used deductive and inductive approaches to create the codebook (table 1). The deductive approach involved creating a codebook with constructs and definitions based on the existing literature and topics relevant to inform e-cigarette regulation. The inductive approach involved refining each category through a human viewing randomly selected videos, reading the video titles and the descriptions, and following the external links to verify that these links led to e-cigarette retailers to confirm that they were retailers and the presence of sales of e-cigarette products. A second independent coder trained in the codebook used similar procedures to confirm these categories and amended them after discussing with the first coder. Then, a third independent coder trained in the modified codebook randomly labelled 10% of the videos used for the training set to determine inter-rater reliability, but due to some videos being removed from YouTube the coder labelled 74 videos. We obtained a Cohen’s kappa of 0.93 (p<0.001). Finally, the second coder viewed and labelled 1000 videos. This labelled data set was used as an input to supervised ML algorithms to train and test the rest of the corpus of videos.

    Text preprocessing

    We used the Python Natural Language Toolkit19 to preprocess the texts of video titles and descriptions. We tokenised the data to split the raw text into tokens and removed punctuations, stop words (eg, ‘is’, ‘the’, ‘on’) and capitalisations and applied stemming (ie, removing suffixes such as ‘ing’) and lemmatisation (ie, identifying groups of words with the same root word; eg, ‘e-cig’ and ‘e-cigarette’ are considered the same). We used a tokeniser to give each word a unique number with a capped vocabulary size of 800 (1200 for classifying the channel type because the data contained a larger vocabulary) and then computed the most common tokens.

    Supervised ML

    The ML model we developed is a sequential model with two layers. The first is an embedding layer, which uses the weights taken from the GloVe word embeddings. GloVe is a set of word embedding which uses a word co-occurrence matrix that was built using a collection of billions of textual inputs.20 This layer turns each word into a vector which represents its semantic meaning. The next layer is a bidirectional long short-term memory networks (BLSTM) layer, followed by the output layer. BLSTM is a deep, neural ML method that takes the complete sequential information of words before and after the target word to provide the context to enhance classification. This architecture has shown to be optimal for processing semantic data.21

    In classifying video themes and featured e-cigarette product, we used video titles and descriptions as inputs to the ML model. We did not include video transcripts because they did not improve the model prediction. We excluded non-English videos (n=371), using the Python langdetect library,22 from the training set and the prediction set. Videos which were classified as ‘other irrelevant’ (n=239; ie, non-e-cigarette videos identified from the video theme construct) were excluded from classifying featured e-cigarette product, channel type and discount/sales.

    For classifying sales, we scraped the first six external links (excluding links to other social media platforms) provided on YouTube’s video description. We developed a parser to extract the meta description (ie, Hyper Text Markup Language (HTML) element that provides a brief summary of the website) on the external website to obtain the web developer’s website description as additional input for the classification. This description captured whether the website sold e-cigarette products (eg, ‘buy vape now’). For classifying channel type, in addition to video titles and descriptions, we also added up to 50 of the most recent videos posted by each channel to expand our training data set for this construct, which eliminated the need for humans to label more videos.

    Performance evaluation

    The metrics used for performance evaluation were precision scores (true positive; ie, the ML algorithm correctly classifying the categories within each construct, divided by the sum of both true and false positive), recall (true positive divided by the sum of true positive and false negative), F1 (harmonic average between precision and recall scores) and accuracy (the sum of true positive and true negative divided by the sum of true positive, true negative, false positive and false negative).

    Data analysis

    Using the BLSTM classification model we developed, trained and tested, we classified the videos into video theme, featured e-cigarette product, channel type and discount/sales. We calculated the median and the IQR for engagement variables (ie, number of views, likes, dislikes, comments, upload date (2007–2018, 2019–2020)) that were non-normally distributed, and we calculated the frequencies for categorical variables (table 2). To assess the association between each construct and engagement variable, we conducted Kruskal-Wallis test and Pearson’s χ2 test (table 2). We also conducted Pearson’s χ2 test to assess whether video themes differed by featured e-cigarette product and channel type (figure 1), and whether discount/sales differed by video theme, featured e-cigarette product and channel type (figure 2). The ML procedures and descriptive analyses were conducted with Python and STATA V.16.0, respectively.

    Figure 1
    Figure 1

    Percentages of video theme by featured e-cigarette product (A) and channel type (B). e-cigarette, electronic cigarette.

    Figure 2
    Figure 2

    Percentages of presence of sales/marketing by video theme (A), e-cigarette product (B) and channel type (C). e-cigarette, electronic cigarette.

    View this table:
    Table 2

    Engagement statistics by video theme, e-cigarette product type, channel type and presence of discount/sales (N=3830)*

    Results

    Of the 4201 videos in our data set, we excluded 371 non-English videos, so 3830 videos were used in supervised ML to classify video themes. In classifying the featured e-cigarette product, channel type and discount/sales, we also excluded ‘irrelevant, non-e-cigarette videos’ (n=239), which were determined from video themes. Table 3 lists the performance evaluation for each construct.

    View this table:
    Table 3

    Evaluation metrics of supervised machine learning models for each construct

    Video theme

    The most common video theme was product review (48.9%), followed by instruction (17.3%) and health information (11.3%). In general, engagement (eg, likes) was low for all video themes except for videos that featured vape tricks. Vape trick videos had the most views (median=2 290 086, IQR=10 156 343), followed by instruction videos (median=47 728, IQR=181 672). Higher proportion of instruction (69.6%) and vape trick (67.4%) videos were uploaded earlier (ie, 2007–2018) and cannabis (57.1%) videos later (ie, 2019–2020).

    Featured e-cigarette product

    The most featured e-cigarette product was a ‘non-specific’ device (29.8%; eg, vaping kit, e-cigarette subscription service), followed by box mods (25.1%), e-liquid (14.7%) and disposable pods (11.9%). Videos that did not feature a specific device and videos that featured box mods, vape pens and pod systems had more engagement than videos that featured ‘other’ e-cigarette products. Videos featuring cigalikes, e-liquid and vape pens were uploaded in earlier years, whereas videos featuring disposable pods and pod systems were uploaded in recent years (table 2).

    Channel type

    The most common channel type was vape enthusiasts (54.0%), followed by retailers (20.3%). Overall, engagement was highest for videos uploaded by private users, followed by vape enthusiasts. A larger proportion of retailers posted videos in earlier years (61.9%) than in recent years (38.1%).

    Discount/sales

    Of the videos, 43.2% had discount/sales. The most common discount/sales strategies were external links for purchasing an e-cigarette product (34.1%), followed by ‘other’ promotional strategies (7.5%; eg, e-cigarette product giveaways, non-e-cigarette merchandise such as T-shirts with e-cigarette images) and discount codes (1.6%; eg, ‘enter XYZ code to get 20% off’ to purchase e-cigarette products). Videos that had ‘other’ promotional strategies had the most engagement and were posted in recent years, whereas videos that offered discounts and external links for purchasing were posted in earlier years (table 2).

    Video theme by e-cigarette product and channel type

    The most featured e-cigarette products for each video theme (figure 1A) were box mods (34.7%) in product review and e-liquid (24.1%) and box mods (22.9%) in instruction. Health information, ‘other relevant’, vape trick and cannabis themed videos did not feature a specific e-cigarette product.

    Video theme by channel type comparisons (figure 1B) showed that vape enthusiasts were most likely to post vape trick (75.0%), product review (69.5%) and instruction videos (44.2%). Retailers were the second most likely to post instruction (39.7%) and product review (16.4%) videos. The medical community most often posted health information videos (41.5%). ‘Other channel’ and retailers most often posted cannabis videos (50.0% and 18.6%, respectively).

    Discount/sales by video theme, e-cigarette products and channel type

    Discount/sales by video theme (figure 2A) comparisons showed that the majority of vape tricks (90.9%), health information (88.6%) and ‘other relevant’ (73.7%) video themes did not have direct discount/sales. The most common promotional strategy used in cannabis vaping videos was external links for purchasing (53.9%). Instruction and product review videos were mixed; some had no direct discount/sales and some had links for purchasing (product review: 53.8% no discount/sales, 36.9% purchasing links; instruction: 46.1% no discount/sales, 49.9% purchasing links). ‘Other’ promotional methods were more common among cannabis (15.4%) and product review (7.1%) themes than other themes.

    Comparisons of discount/sales by e-cigarette product type (figure 2B) and channel type (figure 2C) showed that e-liquid videos (57.5%) had more purchasing links relative to other e-cigarette products. Retailers (62.8%) had the greatest purchasing links than purchasing links presented in other channel types.

    Discussion

    Principal findings

    To the best of our knowledge, our study is the first to use supervised ML to classify complex constructs such as video theme, e-cigarette product, video uploader (ie, channel type) and discount/sales of e-cigarettes on YouTube. Consistent with prior studies on YouTube, we also identified that the common video themes were product reviews and instructions on how to use/modify/create e-cigarette products, some of which had external links for purchasing.5 23 24 These similar themes may not be surprising because we used similar search terms and did not restrict the years of the video upload. However, it is notable that these themes persisted over time. For example, instruction videos that were posted earlier were still prominent, which suggests relative popularity of this topic. This finding is consistent with prior findings that observed that youth 1) find the ability to customise their e-cigarettes appealing,25 2) customize their e-cigarettes (eg, changing flavours, propylene glycol (PG)/vegetable glycerine (VG) ratio, voltage),25 and 3) learn how to customize/modify their e-cigarettes by viewing YouTube videos.26 27

    We also observed new trends; ‘other’ promotional strategies (which also had the most engagement) were posted in recent years, suggesting that the e-cigarette industry is using novel methods to promote their product. Such ‘other’ promotional strategies included giveaways of e-cigarette products and sales of non-e-cigarette merchandise that alludes to e-cigarettes, such as clothing with e-cigarette-related logos/images. These findings highlight the utility of conducting surveillance of e-cigarettes on YouTube to identify novel promotional trends and trends that persist over time to inform e-cigarette regulation.

    We also observed that ‘vape enthusiasts’ and retailers were the most common channel who posted e-cigarette videos about product reviews and instructions, which is a novel way to promote e-cigarettes without using traditional paid ads that could be tracked and regulated. This finding is consistent with previous qualitative studies that observed that vape shop retailers considered social media as a new effective marketing channel to use strategies, such as featuring new e-cigarette products.28 29 We used the term ‘vape enthusiasts’ to refer to channels that primarily posted videos about e-cigarettes.30 However, we cannot confirm whether they are ‘influencers’ who get paid by the e-cigarette industry to promote their product because funding source is not disclosed. ‘Vape enthusiasts’ may also include individuals who are not paid by the industry but who are trying to become ‘influencers’ by collecting followers through posting e-cigarette videos. Indeed, a recent study observed that youth who use e-cigarettes are motivated to becoming influencers to promote vaping products to get paid.31 It is also notable that a non-negligible number of ‘private users’ are also posting external purchasing links and it is unclear whether these ‘private users’ have ties to the e-cigarette industry.

    Our findings also indicate that youth who search for e-cigarettes may be inadvertently exposed to cannabis vaping content. Cannabis vaping among youth is high.32 Moreover, recent research showed that e-cigarette use is one of the predictors of cannabis vaping among youth.33 More research is needed on how cannabis and nicotine vaping products are promoted to prevent youth use of both substances.

    Regulation of e-cigarette promotion on YouTube

    Overall, our findings underscore the presence of e-cigarette promotion on YouTube videos accessible to youth. While some of the promotions fall under ‘earned media’, in which third-party retailers such as vape shops are marketing and selling e-cigarette products through posting video themes such as product reviews and instruction, most of the promotions do not fall under media promotions that can be regulated. For instance, e-cigarettes are not advertised using traditional paid advertisements(ie, ‘paid media’), the presence of ‘influencers’ cannot be verified due to the lack of clear financial disclosure (ie, ‘earned media’) and e-cigarette manufacturers/brands are not directly selling their products on their YouTube accounts (ie, ‘owned media’). However, we did not examine content such as paid advertisement banners, so it is possible that paid advertisement exists on YouTube.

    Currently, YouTube has self-imposed policies that attempt to restrict tobacco content including e-cigarettes. YouTube prohibits the sales of e-cigarettes through posting contact information including external links.34 YouTube also discourages e-cigarette content by limiting advertising revenues that could be earned on videos with tobacco content.34 Finally, e-cigarette content, such as product reviews of e-cigarettes, is restricted to underage youth.35 Despite these efforts, our study findings indicate the need for stricter enforcement of these policies to protect the youth from e-cigarette content.

    The duty to protect the youth does not fall on YouTube alone. The government entities could set restrictions on tobacco marketing, including e-cigarette marketing, on venues frequented by youth, such as social media. There should be greater efforts to counteract pro-e-cigarette content on social media. For instance, we observed that the medical community’s videos that showcased the health risks of e-cigarette use through interviews with experts and research presentations were prominent but had the lowest engagement (eg, <2600 views). Health information videos are competing with many diverse pro-e-cigarette content on YouTube and novel methods are needed to make this content appeal to youth and increase engagement.

    Strengths/weaknesses

    Social media’s algorithm, including YouTube’s, tailors the search results based on a variety of factors, such as one’s profile information, search and view history, and other conglomerate factors, which are proprietary information. Existing studies have not used any personalisation when identifying e-cigarette or other tobacco content, and our study is the first to use fictitious youth viewer profiles to search for e-cigarette videos. Despite this strength, fictitious viewer profiles are only the initial step in identifying the types of content that youth may be exposed to, and future research should leverage this method to obtain more relevant and accurate information on e-cigarettes on social media. For instance, actual youth may be searching for other terms related to e-cigarettes and not just a single word. Relatedly, our findings should be interpreted with the caveat that our search words were broad and general and did not include brand-specific search words like ‘JUUL’ or ‘Puffbar’. Despite these limitations, our findings showed that youth who do not have any viewing history related to e-cigarettes could be exposed to diverse e-cigarette content, including concerning content on YouTube, such as direct sales and discounts of e-cigarettes and cannabis vaping.

    Another strength is that the ML performance was robust and comparable with those identified in other studies that examined complicated themes, such as user sentiment to e-cigarette content on Twitter,36 which demonstrates that ML models can be used to identify relevant themes for e-cigarette surveillance on social media. However, we acknowledge areas for improvement; our lowest F1 score was in classifying the featured e-cigarette product. We used the title and the video description in our ML model to classify each construct, but future research may improve model performance by using visual classification. Visual classification may also be able to identify the characteristics of ‘models’ featured on e-cigarette videos and other visual components such as the presence of warning labels.

    Although we analysed many videos (N=3830), our method trades off volume for personalised results. We could not conduct automated queries to the YouTube API to obtain more videos because our method of using fictitious profiles involved having to factory-reset the phone for each fictitious viewer profile and then web-scraping the results for each search term. Since we did not examine all videos, it is possible that emerging themes such as COVID-19-related themes were not detected at the time of our search (July 2020).

    Our search was conducted using English so the content that we obtained may be relevant to the English-speaking US population. Non-English speakers may be exposed to e-cigarette content in other languages, so non-English content should be examined in future studies as both YouTube and e-cigarettes have global reach.

    Summary

    In summary, we identified video themes, featured e-cigarette products, who posted these videos and discount/sales on youth-accessible YouTube videos. Our findings highlight the utility of using advanced ML methods to conduct surveillance of e-cigarette use trends and marketing/sales strategies on social media platforms such as YouTube. Our findings also underscore the need for comprehensive federal regulations to protect the youth from exposure to promotion of e-cigarette and cannabis vaping on YouTube.

    What this paper adds

    What is already known on this topic

    • Pro-electronic cigarette (e-cigarette) content is prolific on social media platforms such as YouTube.

    • It is unknown whether machine learning can be used to classify complicated themes that can inform tobacco control, such as video themes, featured e-cigarette products, uploader type and presence of discount/sales.

    What this study adds

    • We identified YouTube videos related to e-cigarettes using fictitious youth viewer profiles.

    • Our supervised machine learning identified video themes (eg, product review, instruction), e-cigarette products, uploader type (eg, retailers, ‘vape enthusiasts’) and presence of discount/sales.

    How this study might affect research, practice and/or policy

    • Despite YouTube’s policies to restrict tobacco content to youth, we found that promotion of e-cigarettes was prevalent, indicating the need for comprehensive marketing restrictions and enforcement.

    • Machine learning can be used to monitor tobacco promotion on YouTube.

    Data availability statement

    Data are available upon reasonable request. Data used from this study are publicly available data from YouTube. However, we can provide data upon reasonable request.

    Ethics statements

    Patient consent for publication

    Ethics approval

    This observational study of publicly available data was deemed exempt as human subjects research by the Yale Institutional Review Board (HIC# 2000028350).

    References

    1. ↵
      2. Park-Lee E ,
      3. Ren C ,
      4. Sawdey MD , et al
      . Notes from the field: E-cigarette use among middle and high school students - national youth tobacco survey, united states, 2021. MMWR Morb Mortal Wkly Rep 2021;70:1387–9.doi:10.15585/mmwr.mm7039a4 pmid:http://www.ncbi.nlm.nih.gov/pubmed/34591834
      OpenUrlCrossRefPubMed
    2. ↵
      2. Gottschlich A ,
      3. Mus S ,
      4. Monzon JC , et al
      . Cross-sectional study on the awareness, susceptibility and use of heated tobacco products among adolescents in Guatemala City, Guatemala. BMJ Open 2020;10:e039792.doi:10.1136/bmjopen-2020-039792 pmid:http://www.ncbi.nlm.nih.gov/pubmed/33318112
      OpenUrlAbstract/FREE Full Text
    3. ↵
      2. Tarasenko Y ,
      3. Ciobanu A ,
      4. Fayokun R , et al
      . Electronic cigarette use among adolescents in 17 European study sites: findings from the global youth tobacco survey. Eur J Public Health 2022;32:126–32.doi:10.1093/eurpub/ckab180 pmid:http://www.ncbi.nlm.nih.gov/pubmed/34694383
      OpenUrlCrossRefPubMed
    4. ↵
      1. World Health Organization
      . Report on the global tobacco epidemic. Geneva, 2019.
    5. ↵
      2. Kong G ,
      3. LaVallee H ,
      4. Rams A , et al
      . Promotion of vape tricks on YouTube: content analysis. J Med Internet Res 2019;21:e12709.doi:10.2196/12709
    6. ↵
      2. Alpert JM ,
      3. Chen H ,
      4. Riddell H , et al
      . Vaping and instagram: a content analysis of e-cigarette posts using the content appealing to youth (cay) index. Subst Use Misuse 2021;56:879–87.doi:10.1080/10826084.2021.1899233 pmid:http://www.ncbi.nlm.nih.gov/pubmed/33749515
      OpenUrlCrossRefPubMed
    7. ↵
      2. Vogel EA ,
      3. Ramo DE ,
      4. Rubinstein ML , et al
      . Effects of social media on adolescents' willingness and intention to use e-cigarettes: an experimental investigation. Nicotine Tob Res 2021;23:694–701.doi:10.1093/ntr/ntaa003 pmid:http://www.ncbi.nlm.nih.gov/pubmed/31912147
      OpenUrlCrossRefPubMed
    8. ↵
      2. Zheng X ,
      3. Li W ,
      4. Wong S-W , et al
      . Social media and e-cigarette use among US youth: longitudinal evidence on the role of online advertisement exposure and risk perception. Addict Behav 2021;119:106916.doi:10.1016/j.addbeh.2021.106916 pmid:http://www.ncbi.nlm.nih.gov/pubmed/33798917
      OpenUrlCrossRefPubMed
    9. ↵
      2. Lee J ,
      3. Tan ASL ,
      4. Porter L , et al
      . Association between social media use and vaping among Florida adolescents, 2019. Prev Chronic Dis 2021;18:E49.doi:10.5888/pcd18.200550 pmid:http://www.ncbi.nlm.nih.gov/pubmed/33988495
      OpenUrlCrossRefPubMed
    10. ↵
      2. Ceci L
      . YouTube - Statistics & Facts. Statista 2021.
    11. ↵
      2. Ceci L
      . Number of YouTube users worldwide from 2016 to 2021. Statista 2021.
    12. ↵
      2. Huang J ,
      3. Kornfield R ,
      4. Szczypka G , et al
      . A cross-sectional examination of marketing of electronic cigarettes on Twitter. Tob Control 2014;23 Suppl 3:iii26–30.doi:10.1136/tobaccocontrol-2014-051551 pmid:http://www.ncbi.nlm.nih.gov/pubmed/24935894
      OpenUrlPubMed
    13. ↵
      2. Fu R ,
      3. Kundu A ,
      4. Mitsakakis N , et al
      . Machine learning applications in tobacco research: a scoping review. Tob Control 2023;32:99–109. doi:10.1136/tobaccocontrol-2020-056438 pmid:34452986
      OpenUrlAbstract/FREE Full Text
    14. ↵
      2. Kim K ,
      3. Gibson LA ,
      4. Williams S , et al
      . Valence of media coverage about electronic cigarettes and other tobacco products from 2014 to 2017: evidence from automated content analysis. Nicotine Tob Res 2020;22:1891–900.doi:10.1093/ntr/ntaa090 pmid:http://www.ncbi.nlm.nih.gov/pubmed/32428214
      OpenUrlCrossRefPubMed
    15. ↵
      1. Food and Drug Administration
      . Deeming tobacco products to be subject to the federal food, drug, and cosmetic act. Fed Regist 2014.
    16. ↵
      2. Stephen A ,
      3. Galak J
      . The effects of traditional and social earned media on sales: a study of a microlending marketplace. Soc Sci Res Network 2012.doi:10.2139/ssrn.1480088
    17. ↵
      2. Dashtian H ,
      3. Murthy D ,
      4. Kong G
      . An exploration of e-cigarette–related search items on YouTube: network analysis. J Med Internet Res 2021;24.doi:10.2196/30679
    18. ↵
      Tor project, 2022. Available: https://www.torproject.org/
    19. ↵
      Natural language toolkit, 2021. Available: https://www.nltk.org/
    20. ↵
      2. Pennington J ,
      3. Socher R ,
      4. Manning CD
      . Glove: global vectors for word representation, 2014. Available: https://nlp.stanford.edu/projects/glove/
    21. ↵
      2. Song M ,
      3. Zhao X ,
      4. Liu Y
      . Text sentiment analysis based on convolutional neural network and bidirectional lstm model. In: 24th International Conference on automation and computing Singapore. Springer Singapore, 2018: 55–68.
    22. ↵
      Langdetect 1.0.9, 2021. Available: https://pypi.org/project/langdetect/
    23. ↵
      2. Luo C ,
      3. Zheng X ,
      4. Zeng DD , et al
      . Portrayal of electronic cigarettes on YouTube. BMC Public Health 2014;14:1028 http://www.biomedcentral.com/1471-2458/14/1028 doi:10.1186/1471-2458-14-1028 pmid:http://www.ncbi.nlm.nih.gov/pubmed/25277872
      OpenUrlCrossRefPubMed
    24. ↵
      2. Guy MC ,
      3. Helt J ,
      4. Palafox S , et al
      . Orthodox and unorthodox uses of electronic cigarettes: a surveillance of YouTube video content. Nicotine Tob Res 2019;21:1378–84.doi:10.1093/ntr/nty132 pmid:http://www.ncbi.nlm.nih.gov/pubmed/29961828
      OpenUrlCrossRefPubMed
    25. ↵
      2. Camenga DR ,
      3. Morean M ,
      4. Kong G , et al
      . Appeal and use of Customizable e-cigarette product features in adolescents. Tob Regul Sci 2018;4:51–60.doi:10.18001/TRS.4.2.5 pmid:http://www.ncbi.nlm.nih.gov/pubmed/33163582
      OpenUrlCrossRefPubMed
    26. ↵
      2. Kong G ,
      3. Morean ME ,
      4. Bold KW , et al
      . Dripping and vape tricks: alternative e-cigarette use behaviors among adolescents. Addict Behav 2020;107:106394.doi:10.1016/j.addbeh.2020.106394 pmid:http://www.ncbi.nlm.nih.gov/pubmed/32222561
      OpenUrlCrossRefPubMed
    27. ↵
      2. Amin S ,
      3. Dunn AG ,
      4. Laranjo L
      . Exposure to e-cigarette information and advertising in social media and e-cigarette use in Australia: a mixed methods study. Drug Alcohol Depend 2020;213:108112.doi:10.1016/j.drugalcdep.2020.108112 pmid:http://www.ncbi.nlm.nih.gov/pubmed/32574981
      OpenUrlCrossRefPubMed
    28. ↵
      2. Yang JS ,
      3. Lee E
      . A qualitative assessment of business perspectives and tactics of tobacco and vape shop retailers in three communities in orange County, Ca, 2015-2016. Arch Public Health 2018;76:57.doi:10.1186/s13690-018-0307-z pmid:http://www.ncbi.nlm.nih.gov/pubmed/30349691
      OpenUrlPubMed
    29. ↵
      2. Cheney M ,
      3. Gowin M ,
      4. Wann TF
      . Marketing practices of vapor store owners. Am J Public Health 2015;105:e16–21.doi:10.2105/AJPH.2015.302610 pmid:http://www.ncbi.nlm.nih.gov/pubmed/25880960
      OpenUrlPubMed
    30. ↵
      2. Kong G ,
      3. LaVallee H ,
      4. Rams A , et al
      . Promotion of vape tricks on YouTube: content analysis. J Med Internet Res 2019;21:e12709.doi:10.2196/12709 pmid:http://www.ncbi.nlm.nih.gov/pubmed/31215510
      OpenUrlCrossRefPubMed
    31. ↵
      2. Vassey J ,
      3. Metayer C ,
      4. Kennedy CJ , et al
      . #vape: Measuring e-cigarette influence on instagram with deep learning and text analysis. Front Commun 2020;4.doi:10.3389/fcomm.2019.00075
    32. ↵
      2. Morean ME ,
      3. Kong G ,
      4. Camenga DR , et al
      . High school students' use of electronic cigarettes to Vaporize cannabis. Pediatrics 2015;136:611–6.doi:10.1542/peds.2015-1727 pmid:http://www.ncbi.nlm.nih.gov/pubmed/26347431
      OpenUrlCrossRefPubMed
    33. ↵
      2. Lee J ,
      3. Kong G ,
      4. Kassas B , et al
      . Predictors of vaping marijuana initiation among US adolescents: results from the population assessment of tobacco and health (path) study wave 3 (2015-2016) and wave 4 (2016-2018). Drug Alcohol Depend 2021;226:108905.doi:10.1016/j.drugalcdep.2021.108905 pmid:http://www.ncbi.nlm.nih.gov/pubmed/34304122
      OpenUrlCrossRefPubMed
    34. ↵
      Youtube. community guidelines, 2022. Available: https://www.youtube.com/yt/policyandsafety/communityguidelines.html
    35. ↵
      Youtube. Age-restricted content., 2022. Available: https://support.google.com/youtube/answer/2802167?hl=en
    36. ↵
      2. Visweswaran S ,
      3. Colditz JB ,
      4. O'Halloran P , et al
      . Machine learning classifiers for Twitter surveillance of vaping: comparative machine learning study. J Med Internet Res 2020;22:e17478.doi:10.2196/17478 pmid:http://www.ncbi.nlm.nih.gov/pubmed/32784184
      OpenUrlCrossRefPubMed

    Footnotes

    • Contributors GK: is the guarantor of the study. As an guarantor, GK accepts full responsibility for the finished work and the conduct of the study, has access to the data, and controlled the decision to publish. She conceptualised and designed the study, obtained funding for the study, interpreted the results, wrote the first draft. DM: designed the study, acquired the data, analysed the data, interpreted the results, wrote sections. JL: wrote sections, interpreted the results. ASS: analysed the data, interpreted the results, wrote sections. HD: assisted in data analysis.

    • Funding The research reported in this publication was supported by Grant Number R01DA049878 from the National Institute on Drug Abuse (NIDA) and the US Food and Drug Administration (FDA) Center for Tobacco Products (CTP).

    • Disclaimer The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH or the FDA.

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.