NOT PEER REVIEWED
I would like to make three comments by way of a brief post-publication review.
1. The impacts of vaping tax on smoking have been completely overlooked
For a study of e-cigarette taxation to have any public health relevance, it must consider the impact of e-cigarette prices on *cigarette* demand. Cigarettes and e-cigarettes are economic substitutes. The demand for one responds to changes in the price of the other, an idea well understood in economics and quantified through the concept of cross-elasticity. The paper appears to pay no regard to the impact of vaping taxes on cigarette demand, Yet such effects might easily overwhelm any benefits from reduced e-cigarette use - in fact, impact on demand for other tobacco products and the development of informal markets are by far the most important impacts of a vaping tax. By way of example, a 2020 paper by Pesko et al. [1] concluded:
"Our results suggest that a proposed national e-cigarette tax of $1.65 per milliliter of vaping liquid would raise the proportion of adults who smoke cigarettes daily by approximately 1 percentage point, translating to 2.5 million extra adult daily smokers compared to the counterfactual of not having the tax."
2. The case for reducing adult vaping by taxation has not been made
The authors have based their paper on an unexamined assumption that it is a justifiable goal of policy to lower rates of adult e-cigarette use. Why should...
NOT PEER REVIEWED
I would like to make three comments by way of a brief post-publication review.
1. The impacts of vaping tax on smoking have been completely overlooked
For a study of e-cigarette taxation to have any public health relevance, it must consider the impact of e-cigarette prices on *cigarette* demand. Cigarettes and e-cigarettes are economic substitutes. The demand for one responds to changes in the price of the other, an idea well understood in economics and quantified through the concept of cross-elasticity. The paper appears to pay no regard to the impact of vaping taxes on cigarette demand, Yet such effects might easily overwhelm any benefits from reduced e-cigarette use - in fact, impact on demand for other tobacco products and the development of informal markets are by far the most important impacts of a vaping tax. By way of example, a 2020 paper by Pesko et al. [1] concluded:
"Our results suggest that a proposed national e-cigarette tax of $1.65 per milliliter of vaping liquid would raise the proportion of adults who smoke cigarettes daily by approximately 1 percentage point, translating to 2.5 million extra adult daily smokers compared to the counterfactual of not having the tax."
2. The case for reducing adult vaping by taxation has not been made
The authors have based their paper on an unexamined assumption that it is a justifiable goal of policy to lower rates of adult e-cigarette use. Why should this be a policy goal any more than reducing caffeine use or moderate alcohol use? The goal of public health policy is to address significant harms or self-destructive patterns of use, not to modify behaviours that the authors find distasteful. What are the harms that justify state intervention to reduce adult vaping with a tax? Further, they appear indifferent to welfare costs and the distributional impact of imposing a regressive tax burden on people who use vaping products. Tobacco control advocates should become more familiar with the idea that punitive policies impose harm on users, even though these users are supposed to be the intended beneficiaries. For example, a vaping tax harms families by drawing on the household budget of those who continue to vape.
3. The analysis to support the policy recommendations is wholly inadequate
The authors make over-confident policy recommendations without considering the full range of impacts of the measures they are proposing.
"Our findings suggest that adopting a vaping product excise tax policy may help reduce ENDS use and suppress the increase of ENDS use prevalence among young adults. Considering that there are still a number of US states that have not implemented vaping product excise tax policy, wider adoption of such policy across the nation would likely help mitigate ENDS use prevalence."
Without considering all the possible responses to the tax they support, they may easily be proposing tobacco control policies that do more harm than good. In fact, the most important public health impact of this policy is entirely excluded from the analysis. That is the effect of a vaping tax on smoking or other tobacco use. Given the two orders of magnitude difference in risk between smoking and vaping, only a tiny uptick in smoking would be needed to completely offset the benefits, if any, arising from reduced vaping
NOT PEER REVIEWED
I have a number of concerns with the paper as currently written.
1) The authors write: “Besides, none of the previous studies except Pesko et al (15) that examined the associations between vaping product excise tax adoption and ENDS use has accounted for the clustering of respondents within the same localities…” This is not accurate, as citation 19 also clusters standard errors at the locality level in all specifications.
2) The authors write: "A working paper reported reduced ENDS sales, but not ENDS use prevalence or behaviours, after implementation of a vaping product excise tax policy. (19)” This is not accurate, as the cited study uses the magnitude of e-cigarette tax values, rather than an indicator variable for tax implementation. States have adopted e-cigarette taxes of different magnitudes and a number of them (such as California) have changed the magnitudes of these taxes after adoption. All of this variation is used in citation 19, contrary to the current study’s description. It's also unclear from the sentence whether citation 19 studied use and found imprecise estimates, or did not study use. It's the latter and this should be clarified. It's also unclear why the authors did not use magnitude of e-cigarette taxes themselves in the current paper, as has been commonly done in the referenced literature.
3) Authors write they use a “nationally representative sample of US young adults.” I do not beli...
NOT PEER REVIEWED
I have a number of concerns with the paper as currently written.
1) The authors write: “Besides, none of the previous studies except Pesko et al (15) that examined the associations between vaping product excise tax adoption and ENDS use has accounted for the clustering of respondents within the same localities…” This is not accurate, as citation 19 also clusters standard errors at the locality level in all specifications.
2) The authors write: "A working paper reported reduced ENDS sales, but not ENDS use prevalence or behaviours, after implementation of a vaping product excise tax policy. (19)” This is not accurate, as the cited study uses the magnitude of e-cigarette tax values, rather than an indicator variable for tax implementation. States have adopted e-cigarette taxes of different magnitudes and a number of them (such as California) have changed the magnitudes of these taxes after adoption. All of this variation is used in citation 19, contrary to the current study’s description. It's also unclear from the sentence whether citation 19 studied use and found imprecise estimates, or did not study use. It's the latter and this should be clarified. It's also unclear why the authors did not use magnitude of e-cigarette taxes themselves in the current paper, as has been commonly done in the referenced literature.
3) Authors write they use a “nationally representative sample of US young adults.” I do not believe this is not accurate. The TUS-CPS sample itself may be nationally representative, but this representativeness may be lost when subgroups are explored.
4) The “vaping product excise tax policy” variable in Table 3 appears to be re-defined mid-table. Based on the discussion of the results, in column 1 it appears that this variable is an indicator equal to 1 only at the time when a state has an e-cigarette tax in place. In column 2 though, this indicator equals 1 when a state ever has an e-cigarette tax in place (even prior to it being in place). The use of the same row for a variable that changes across columns is unusual and can easily lead to the wrong interpretation.
We appreciate the comments from Bates and the opportunity for us to respond and clarify.
First, Bates' argument heavily relies on the assumption that e-cigarettes and combustible cigarettes are substitutes, which is theoretically possible as some consider vaping as a harm reduction alternative to combustible cigarettes. Empirically, however, there have been mixed findings about whether e-cigarettes and combustible cigarettes are substitutes (or complements). Bates cited Pesko et al. (2020) that concludes e-cigarettes and combustible cigarettes are substitutes, whereas other studies have shown that they are complements. For example, Cotti et al. (2018) found that higher cigarette excise taxes, in fact, decrease sales of both e-cigarettes and combustible cigarettes, suggesting that they are complements. Such mixed results abate Bates' argument that taxing ENDS could lead to more use of combustible cigarettes.
Second, Bates might have ignored that our study focused on young adults aged 18-24 years rather than general adults when examining the effect of vaping product tax on e-cigarette use. Although Pesko et al. (2020) suggests that e-cigarettes and combustible cigarettes are substitutes, the findings are based on the general adult population (average age: 55 years) which may not be generalizable to the young adult population. In fact, one study conducted by Abouk and Adams (2017) indicates that e-cigarettes and combustible ci...
We appreciate the comments from Bates and the opportunity for us to respond and clarify.
First, Bates' argument heavily relies on the assumption that e-cigarettes and combustible cigarettes are substitutes, which is theoretically possible as some consider vaping as a harm reduction alternative to combustible cigarettes. Empirically, however, there have been mixed findings about whether e-cigarettes and combustible cigarettes are substitutes (or complements). Bates cited Pesko et al. (2020) that concludes e-cigarettes and combustible cigarettes are substitutes, whereas other studies have shown that they are complements. For example, Cotti et al. (2018) found that higher cigarette excise taxes, in fact, decrease sales of both e-cigarettes and combustible cigarettes, suggesting that they are complements. Such mixed results abate Bates' argument that taxing ENDS could lead to more use of combustible cigarettes.
Second, Bates might have ignored that our study focused on young adults aged 18-24 years rather than general adults when examining the effect of vaping product tax on e-cigarette use. Although Pesko et al. (2020) suggests that e-cigarettes and combustible cigarettes are substitutes, the findings are based on the general adult population (average age: 55 years) which may not be generalizable to the young adult population. In fact, one study conducted by Abouk and Adams (2017) indicates that e-cigarettes and combustible cigarettes are not substitutes for young people. Established cigarette smokers may use e-cigarettes as a cessation tool but it is less common in young adults. In addition, even if e-cigarettes and combustible cigarettes are substitutes to some degree, the direction of substitution as well as co-use versus subsequent use should not be overlooked. Studies have shown that e-cigarettes may serve as a gateway to future combustible cigarette smoking among young people. For example, a study conducted by Hair et al. (2021) shows that youth and young adults who reported ever e-cigarette use had significantly higher odds of ever cigarette use one year later. Therefore, e-cigarette use versus combustible cigarette smoking is not simply an issue of substitution in particular among young people.
Disclosure: We did not receive any funding from the tobacco industry.
References:
1. Abouk, R., & Adams, S. (2017). Bans on electronic cigarette sales to minors and smoking among high school students. Journal of Health Economics, 54, 17-24.
2. Cotti, C., Nesson, E., & Tefft, N. (2018). The relationship between cigarettes and electronic cigarettes: Evidence from household panel data. Journal of Health Economics, 61, 205-219.
3. Hair, E. C., Barton, A. A., Perks, S. N., Kreslake, J., Xiao, H., Pitzer, L., ... & Vallone, D. M. (2021). Association between e-cigarette use and future combustible cigarette use: Evidence from a prospective cohort of youth and young adults, 2017–2019. Addictive Behaviors, 112, 106593.
4. Pesko, M. F., Courtemanche, C. J., & Maclean, J. C. (2020). The effects of traditional cigarette and e-cigarette tax rates on adult tobacco product use. Journal of Risk and Uncertainty, 60(3), 229-258.
NOT PEER REVIEWED
We thank Pesko for his comments and the opportunity for us to respond and clarify.
First, we appreciate Pesko’s clarification that Cotti et al. (2020) clustered standard errors to account for clustering. In the present study, we used multilevel analysis not only to account for clustering of respondents (i.e., design effects) but also to incorporate different error terms for different levels of the data hierarchy which yields more accurate Type I error rates than nonhierarchical methods where all unmodeled contextual information ends up pooled into a single error term of the model.
Second, we understand that Cotti et al. (2020) evaluated the magnitude of e-cigarette tax values, which does not contradict to our statement because our study focused on the effects of e-cigarette excise tax policies on individual e-cigarette use and prevalence rather than aggregated sales at state or county levels. We also clearly described the reason why we examined the e-cigarette excise tax policy implementation indicator rather than its magnitude in our paper’s discussion section.
Third, our study used a nationally representative sample of young adults (rather than a nationally representative sample of general adult population). While we understand Pesko’s concern that a sample’s representativeness might be lost when subgroups are explored, we believe our use of sampling weights in analysis has reduced such a concern.
NOT PEER REVIEWED
We thank Pesko for his comments and the opportunity for us to respond and clarify.
First, we appreciate Pesko’s clarification that Cotti et al. (2020) clustered standard errors to account for clustering. In the present study, we used multilevel analysis not only to account for clustering of respondents (i.e., design effects) but also to incorporate different error terms for different levels of the data hierarchy which yields more accurate Type I error rates than nonhierarchical methods where all unmodeled contextual information ends up pooled into a single error term of the model.
Second, we understand that Cotti et al. (2020) evaluated the magnitude of e-cigarette tax values, which does not contradict to our statement because our study focused on the effects of e-cigarette excise tax policies on individual e-cigarette use and prevalence rather than aggregated sales at state or county levels. We also clearly described the reason why we examined the e-cigarette excise tax policy implementation indicator rather than its magnitude in our paper’s discussion section.
Third, our study used a nationally representative sample of young adults (rather than a nationally representative sample of general adult population). While we understand Pesko’s concern that a sample’s representativeness might be lost when subgroups are explored, we believe our use of sampling weights in analysis has reduced such a concern.
Fourth, in Table 3, please note that vaping product excise tax policy indicator is a time-variant variable in Model 1. However, to present results of a standard difference-in-differences model with a binary indicator, the policy implementation status was operationalized as a time-invariant variable in Model 2, which is not unusual.
Disclosure: We did not receive any funding from the tobacco industry.
References
1. Cotti, C. D., Courtemanche, C. J., Maclean, J. C., Nesson, E. T., Pesko, M. F., & Tefft, N. (2020). The effects of e-cigarette taxes on e-cigarette prices and tobacco product sales: evidence from retail panel data. National Bureau of Economic Research. NBER Working Paper No. w26724.
Clive Bates’ commentary on our paper repeats claims we previously addressed [1]. Here, we address seven points, the first is contextual and the remaining are raised in his letter.
1. We note the failure of the author to acknowledge Māori perspectives, in particular their support for endgame measures, concerns in relation to harm minimisation [2] as outlined in his “all in” strategy, and ethical publishing of research about Indigenous peoples. [3]
2. We reject the assertion that the basis of our modelling is “weak”. While there is uncertainty around the potential effect of denicotinisation, as this policy hasn’t been implemented, there are strong grounds to believe that it will have a profound impact on reducing smoking prevalence. This is based on both theory and logic (i.e., nicotine is the main addictive component of cigarettes and why most people smoke), and the findings of multiple randomized controlled trials (RCTs) showing that smoking very low nicotine cigarettes (VLNCs) increases cessation rates for diverse populations of people who smoke [4-7].
Our model’s estimated effect on smoking prevalence had wide uncertainty, namely a median of 85.9% reduction over 5 years with a 95% uncertainty interval of 67.1% to 96.3% that produced (appropriately) wide uncertainty in the health impacts. The derivation of this input parameter through expert knowledge elicitation (EKE) is described in the Appendix of our paper. Univariate se...
Clive Bates’ commentary on our paper repeats claims we previously addressed [1]. Here, we address seven points, the first is contextual and the remaining are raised in his letter.
1. We note the failure of the author to acknowledge Māori perspectives, in particular their support for endgame measures, concerns in relation to harm minimisation [2] as outlined in his “all in” strategy, and ethical publishing of research about Indigenous peoples. [3]
2. We reject the assertion that the basis of our modelling is “weak”. While there is uncertainty around the potential effect of denicotinisation, as this policy hasn’t been implemented, there are strong grounds to believe that it will have a profound impact on reducing smoking prevalence. This is based on both theory and logic (i.e., nicotine is the main addictive component of cigarettes and why most people smoke), and the findings of multiple randomized controlled trials (RCTs) showing that smoking very low nicotine cigarettes (VLNCs) increases cessation rates for diverse populations of people who smoke [4-7].
Our model’s estimated effect on smoking prevalence had wide uncertainty, namely a median of 85.9% reduction over 5 years with a 95% uncertainty interval of 67.1% to 96.3% that produced (appropriately) wide uncertainty in the health impacts. The derivation of this input parameter through expert knowledge elicitation (EKE) is described in the Appendix of our paper. Univariate sensitivity analyses comparing the 67.1% and 96.3% estimates (all other input parameters held at their median value) produced HALY gains ranging from 545,000 to 653,000. Our paper presents this uncertainty transparently.
3. The assertion that the effect size estimate of denicotinisation is based on one randomized trial is incorrect. The author has been informed that this assertion is false on several occasions but even so continues to repeat this claim. We used an EKE process, which is described in the Appendix of our paper. The experts considered many ‘inputs’ to their estimation, of which just one was the evidence from the multiple existing RCTs.
4. We disagree with the author’s characterisation of the EKE process as “arbitrary guesswork”. As Bates himself has noted, expert judgement can provide valuable insight in situations of uncertainty and can “provide a risk-perception ‘anchor’ … following assessment of the evidence that exists.” [8] We believe that ≥ 5 RCTs demonstrating a relationship between VLNCs and increased smoking cessation constitute a reasonable evidence base to draw upon, particularly when supported by theory/logic and other lines of evidence.[9]
Policy-making often occurs in a context of uncertainty. Denicotinisation is one such example, as we will not know its ‘real world’ impact until it has been implemented. To inform that policy making, it is astute to have estimates of the likely health impact – which requires EKE. Over time, as evidence accrues, such modelling should be updated.
5. As stated in our paper, we did not explicitly model an illicit market. Tight border security in an island nation with no land borders within 1,000 km, reduces the potential of a significant illicit tobacco market. Furthermore, the Aotearoa/New Zealand (A/NZ) Government announced new measures against tobacco smuggling in preparation for the introduction of its ‘endgame’ legislation. [10] The impact of an illicit tobacco market may be greater in other countries. In A/NZ, the illicit market is small (around 5-6% max) and has not increased greatly despite 10 years of above inflation tobacco excise increases and the introduction of plain packs – interventions which the tobacco industry routinely claims will result in an explosion in the illicit market. This suggests enforcement measures work well in the A/NZ context. Furthermore, given the widespread availability and use by people who smoke of nicotine-containing vaping products in A/NZ, seeking to replace VLNCs with illicit cigarettes is likely to be significantly less common than in jurisdictions where vaping products are not available.
6. It is possible – as Bates asserts – that we have overestimated the health gains from denicotinisation and other endgame policies because the smoking prevalence since 2020, appears to be falling more rapidly than we modelled (meaning the ‘room’ for health gains from an endgame policy is less). We discussed this in our paper.
7. Discussing the public health philosophy of denicotinisation was beyond the scope of our paper. Our focus was only on evaluating the potential health and equity impacts of four interventions included the A/NZ Smoke-free Action Plan 2025.
[2] Waa A, Robson B, Gifford H, Smylie J, Reading J, Henderson JA, Henderson PN, Maddox R, Lovett R, Eades S, Finlay S. Foundation for a smoke-free world and healthy Indigenous futures: an oxymoron?. Tobacco Control. 2020 Mar 1;29(2):237-40.
[3] Maddox R, Drummond A, Kennedy M, et al. Ethical publishing in ‘Indigenous’ contextsTobacco Control Published Online First: 13 February 2023. doi: 10.1136/tc-2022-057702
[4] Donny EC, Denlinger RL, Tidey JW, Koopmeiners JS, Benowitz NL, Vandrey RG, Al’Absi M, Carmella SG, Cinciripini PM, Dermody SS, Drobes DJ. Randomized trial of reduced-nicotine standards for cigarettes. New England Journal of Medicine. 2015 Oct 1;373(14):1340-9.
[5] Smith TT, Koopmeiners JS, Tessier KM, Davis EM, Conklin CA, Denlinger-Apte RL, Lane T, Murphy SE, Tidey JW, Hatsukami DK, Donny EC. Randomized trial of low-nicotine cigarettes and transdermal nicotine. American journal of preventive medicine. 2019 Oct 1;57(4):515-24.
[6] Walker N, Howe C, Bullen C, Grigg M, Glover M, McRobbie H, Laugesen M, Parag V, Whittaker R. The combined effect of very low nicotine content cigarettes, used as an adjunct to usual Quitline care (nicotine replacement therapy and behavioural support), on smoking cessation: a randomized controlled trial. Addiction. 2012 Oct;107(10):1857-67.
[7] Higgins ST, Tidey JW, Sigmon SC, Heil SH, Gaalema DE, Lee D, Hughes JR, Villanti AC, Bunn JY, Davis DR, Bergeria CL. Changes in cigarette consumption with reduced nicotine content cigarettes among smokers with psychiatric conditions or socioeconomic disadvantage: 3 randomized clinical trials. JAMA network open. 2020 Oct 1;3(10):e2019311-.
I have published a summary critique of this modelling exercise on PubPeer. [1] This summarises concerns raised in post-publication reviews of this paper while it was in pre-print form by experts from New Zealand and Canada, and me. [2][3]
By way of a brief summary:
1. All the important modelled findings flow from a single assumption that denicotinisation will reduce smoking prevalence by 85% over five years. Yet the basis for this assumption is weak and disconnected from the reality of the market system being modelled.
2. The central assumption is based partly on a smoking cessation trial that bears no relation to the market and regulatory intervention that is the subject of the simulation. Even so, the trial findings do not support the modelling assumption.
3. The central assumption also draws on expert elicitation. Yet, there is no experience with this measure as it would be novel, and there is no relevant expertise in this sort of intervention. Where experts have been asked to assess the impacts, their views diverge widely, suggesting that their estimates are mainly arbitrary guesswork.
4. The authors have only modelled benefits and have not included anything that might be a detriment or create a trade-off. The modelling takes no account of the black market or workarounds. These are inevitable consequences of such 'endgame' prohibitions, albeit of uncertain size. Though it may be challenging to mo...
I have published a summary critique of this modelling exercise on PubPeer. [1] This summarises concerns raised in post-publication reviews of this paper while it was in pre-print form by experts from New Zealand and Canada, and me. [2][3]
By way of a brief summary:
1. All the important modelled findings flow from a single assumption that denicotinisation will reduce smoking prevalence by 85% over five years. Yet the basis for this assumption is weak and disconnected from the reality of the market system being modelled.
2. The central assumption is based partly on a smoking cessation trial that bears no relation to the market and regulatory intervention that is the subject of the simulation. Even so, the trial findings do not support the modelling assumption.
3. The central assumption also draws on expert elicitation. Yet, there is no experience with this measure as it would be novel, and there is no relevant expertise in this sort of intervention. Where experts have been asked to assess the impacts, their views diverge widely, suggesting that their estimates are mainly arbitrary guesswork.
4. The authors have only modelled benefits and have not included anything that might be a detriment or create a trade-off. The modelling takes no account of the black market or workarounds. These are inevitable consequences of such 'endgame' prohibitions, albeit of uncertain size. Though it may be challenging to model, the simulation does not account for the negative behavioural or perceptual impacts of trying to force people to quit or switch by using the law to remove their regular cigarettes. It should not be assumed that these are zero or immaterial to policy assessment.
5. The real-world progress in reducing smoking in New Zealand through tobacco harm reduction and the rise of vaping has been rapid and highly positive, outpacing both the business-as-usual baseline assumptions in the modelling and the impact of the intervention. This suggests the modelled benefits are greatly overstated.
6. The denicotinisation policy should not be compared to a flawed and inflated hypothetical business-as-usual baseline but to an alternative policy that embraces a different public health philosophy. The denicotinisation measure uses the power of the law to try to force behaviour change onto smokers by removing their regular cigarettes from the market. This may be effective, but it also carries risks of black market activity and a public or political backlash once the consequences are understood by those affected. The alternative would position the state as an enabler, maximising support, encouragement and incentives to switch to smoke-free alternatives or quit. This is not business as usual but would mean going “all in” on tobacco harm reduction, with the goal of reducing smoking as rapidly as possible but without resorting to using the coercive power of the law. Such a policy may prove effective but also have lower risks and be less susceptible to unintended consequences.
[2] Bates, C., Youdan, B., Bonita, R., Laking, G., Sweanor, D., Beaglehole, R. (2022). Review of: “Tobacco endgame intervention impacts on health gains and Māori:non-Māori health inequity: a simulation study of the Aotearoa-New Zealand Tobacco Action Plan.” Qeios. https://doi.org/10.32388/8WXH0J
[3] Bates, C., Youdan, B., Bonita, R., Sweanor, D., & Beaglehole, R. (2022). Review of: “The case for denicotinising tobacco in Aotearoa NZ remains strong: response to online critique.” Qeios. https://doi.org/10.32388/ZZAUQM
¶ The authors make some points in their article that are reasonable: 1) the generalizability of San Francisco's flavor ban compared to other places is an open question, and 2) the original study uses the San Francisco ban effective date rather than enforcement date. The original author (Friedman), who does not accept tobacco industry funding and is a well-respected scientist in the field, had pointed to both facts in her original article. So that information isn’t new.
¶ The current authors appear to construct a straw man argument claiming that Friedman argued that she was studying the effect of San Francisco enforcing its flavor ban policy. Friedman specifically wrote in her original article that she was studying, “a binary exposure variable [that] captured whether a complete ban on flavored tobacco product sales was in effect in the respondent’s district on January 1 of the survey year.” She specifically uses effect in the above sentence, so there is no ambiguity that she is studying effective date. San Francisco’s flavor ban effective date was July 2018 (Gammon et al. 2021).
¶ The authors found new information that the San Francisco YRBSS survey was collected between November to December of 2018. Gammon et al. 2021 (Appendix Figure 1) shows that flavored e-cigarette sales declined in San Francisco between the effective date and the end of August 2018 (compensating for a 30-day look-back period for the YRBSS question wording), even though the flavor ban...
¶ The authors make some points in their article that are reasonable: 1) the generalizability of San Francisco's flavor ban compared to other places is an open question, and 2) the original study uses the San Francisco ban effective date rather than enforcement date. The original author (Friedman), who does not accept tobacco industry funding and is a well-respected scientist in the field, had pointed to both facts in her original article. So that information isn’t new.
¶ The current authors appear to construct a straw man argument claiming that Friedman argued that she was studying the effect of San Francisco enforcing its flavor ban policy. Friedman specifically wrote in her original article that she was studying, “a binary exposure variable [that] captured whether a complete ban on flavored tobacco product sales was in effect in the respondent’s district on January 1 of the survey year.” She specifically uses effect in the above sentence, so there is no ambiguity that she is studying effective date. San Francisco’s flavor ban effective date was July 2018 (Gammon et al. 2021).
¶ The authors found new information that the San Francisco YRBSS survey was collected between November to December of 2018. Gammon et al. 2021 (Appendix Figure 1) shows that flavored e-cigarette sales declined in San Francisco between the effective date and the end of August 2018 (compensating for a 30-day look-back period for the YRBSS question wording), even though the flavor ban was not yet fully enforced. This could be due to early supply-side responses to the flavor ban (e.g., some businesses discontinuing selling flavored e-cigarettes immediately upon the law’s effective date), or demand for e-cigarettes falling due to publicity related to the flavor ban effective date. The fact that e-cigarette sales continued falling in the latter half of 2018 until full enforcement kicked in on 1/1/2019 does not by itself invalidate Friedman’s model specifically looking at effective date. Therefore, there is nothing flawed about the concept of studying the effect that the flavor ban effective date (which led to a documented decline in flavored e-cigarette sales in San Francisco between July 2018 through the end of August 2018) had on youth cigarette use measured in the San Francisco YRBSS in November to December of 2018 (compared to other locations not adopting flavor bans).
¶ The current TC paper makes many inaccurate statements that appear to undermine most of the paper.
¶ • "Thus, the San Francisco survey preceded the enforcement of its flavoured tobacco sales restriction (January 2019), making the 2019 YRBSS an inappropriate data source for evaluating the effects of the city’s flavoured tobacco sales restriction."
¶ This is not true. The decline in flavored e-cigarette sales between the July 2018 effective date to the end of August 2018 could have clearly resulted in spillover effects in the youth cigarette use marketplace. The authors provide no acknowledgement of this in their paper.
¶ • "If youth smoking rates increased similarly in Oakland following that city’s sales restriction, this would lend credence to the call for caution against flavoured tobacco sales restrictions. However, if the patterns differ, we should identify alternate explanations for the rise in San Francisco’s youth smoking prevalence."
¶ This is faulty logic. It's entirely possible that policies adopted in two separate cities could exhibit different effects (including one having an effect and the other having no effect) depending on the population's underlying preferences for tobacco products and different evasion opportunities. I don’t know if there is a reason that this could be the case or not, but that’s irrelevant. What is relevant is that the loose language as currently written is inaccurate and could lead people to conclude the wrong thing in other contexts. The authors also fail to provide statistical testing of their Oakland model as required by STROBE guidelines, nor do they acknowledge that unlike the original study their own pre-post analysis is limited by not having a counterfactual group of non-treated areas, and so there is no ability to control for trends over time.
¶ • "Since there was no ban on non-menthol cigarettes sales, we would have expected to see an increase in sales of cigarettes if youth had been switching products."
¶ • “The study actually found an overall trend of a reduction in both total tobacco sales and cigarette sales in San Francisco following the flavoured tobacco product sales restriction, further suggesting that flavoured products were not being substituted by other unflavoured tobacco products or cigarettes.”
¶ Assuming for a moment that we can observe cigarette sales sold to youth, it would be entirely possible that these cigarette sales could decline in San Francisco but decline by more in the control areas due to secular trends; therefore, suggesting the flavor ban would need to increase cigarette sales to youth is inaccurate. And of course the authors do not observe who buys these cigarettes (youth or adults), so sales data for the population as a whole does not necessarily refute youth use patterns.
¶ • “However, in order to imply causality, there cannot be ambiguous temporal precedence.”
¶ • “do not include the policy enactment and enforcement dates that are required to avoid erroneous conclusions like those in the recent analysis of the San Francisco flavoured sales restriction.”
¶ The authors state that Friedman is ambiguous about the policy timing, but this is not the case as she clearly states she is studying effective date. That is not ambiguous. The authors also state that Friedman’s study has erroneous conclusions. I do not see anything erroneous about the limited scope of her research question studying effective date.
¶ The authors also refer in their references to conversation with the CDC-Office on Smoking and Health regarding the YRBSS data collection date. This reference is incomplete per STROBE guidelines, and should include a specific individual that the authors spoke with and a date of the conversation. Since this conversation was with a government employee it is especially important that there is not the perception of the government leaking information to certain groups of scientists but not others, so full disclosure is needed here. Other researchers have tried to get effective dates for the YRBSS survey from the CDC before but have been rebuffed, creating concerns regarding inequal access to data, as well as concerns regarding if this communication between the CDC and the researchers was authorized or not.
¶ Additionally, I found the author’s discussion of the tobacco industry promoting Friedman’s study as irrelevant. This discussion has the unfortunate effect of muddying the waters of what is supposed to be a focus on the science of Freidman’s article, and could easily lead people to conclude that Friedman herself has industry funding, which is not true. None of us are impervious to industry attempts to use our research for their own gain; in fact, if we start to attack researchers whose work is used by industry, this gives industry an easy way to discredit the researchers they are most threated by (by finding a way to cite their research in industry reports and publications, etc.). How research is used after the publication process is not relevant to this debate over the merits of the science of Friedman’s original article.
¶ Reference:
¶ Gammon, Doris G., Todd Rogers, Jennifer Gaber, James M. Nonnemaker, Ashley L. Feld, Lisa Henriksen, Trent O. Johnson, Terence Kelley, and Elizabeth Andersen-Rodgers . "Implementation of a comprehensive flavoured tobacco product sales restriction and retail tobacco sales." Tobacco Control (2021).
Pesko’s central argument is that it does not matter that Friedman’s assessment of the effect of San Francisco’s ban on the sale of flavored tobacco products is not based on any data collected after the ban actually went into force. In particular, Friedman’s “after” data were collected in fall 2018, before the ordinance was enforced on January 1, 2019.[1] Pesko incredibly argues that Friedman’s “before-after” difference-in-difference analysis is valid despite the fact that she does not have any “after” data.
Pesko justifies this position on the grounds that the effective date of the San Francisco ordinance was July, 2018. While this is true, it is a matter of public record that the ordinance was not enforced until January 1, 2019 because of the need for time for merchant education and issuing implementing regulations.[2]
Friedman is aware of the fact that the enforcement of the ordinance started on January 1, 2019 and used that date in her analysis. In her response[3] to critiques[4] of her paper, she stated “retailer compliance jumped from 17% in December 2018 to 77% in January 2019 when the ban went into effect.” Friedman thought the YRBSS data was collected in Spring 2019; she only learned that the “2019” San Francisco YRBSS data she used were in fact collected in fall 2018 from our paper.[1]
Rather than simply accepting this as an honest error and suggesting Friedman withdraw her paper, Pesko is offering an after-the-fact justification for the cl...
Pesko’s central argument is that it does not matter that Friedman’s assessment of the effect of San Francisco’s ban on the sale of flavored tobacco products is not based on any data collected after the ban actually went into force. In particular, Friedman’s “after” data were collected in fall 2018, before the ordinance was enforced on January 1, 2019.[1] Pesko incredibly argues that Friedman’s “before-after” difference-in-difference analysis is valid despite the fact that she does not have any “after” data.
Pesko justifies this position on the grounds that the effective date of the San Francisco ordinance was July, 2018. While this is true, it is a matter of public record that the ordinance was not enforced until January 1, 2019 because of the need for time for merchant education and issuing implementing regulations.[2]
Friedman is aware of the fact that the enforcement of the ordinance started on January 1, 2019 and used that date in her analysis. In her response[3] to critiques[4] of her paper, she stated “retailer compliance jumped from 17% in December 2018 to 77% in January 2019 when the ban went into effect.” Friedman thought the YRBSS data was collected in Spring 2019; she only learned that the “2019” San Francisco YRBSS data she used were in fact collected in fall 2018 from our paper.[1]
Rather than simply accepting this as an honest error and suggesting Friedman withdraw her paper, Pesko is offering an after-the-fact justification for the claim that Friedman’s conclusion is still valid despite not being based on any data after the ordinance actually took effect.
In addition to this central issue, Pesko raised some other minor points that we address below.
Pesko criticised the CDC for providing unequal access to data. This is false. We simply used the request form on the CDC public website (https://www.cdc.gov/healthyyouth/data/yrbs/contact.htm) and were directed to reach the San Francisco School District that conducted the YRBSS to confirm these dates.
Pesko argued that our discussion of the tobacco industry promoting Friedman’s study is irrelevant. We disagree. The tobacco industry and its allies and front groups have widely used Friedman’s conclusion “that reducing access to flavored electronic nicotine delivery systems may motivate youths who would otherwise vape to substitute smoking”[5] to oppose local and state flavored tobacco sales restrictions.
References:
1 Liu J, Hartman L, Tan ASL, et al. Youth tobacco use before and after flavoured tobacco sales restrictions in Oakland, California and San Francisco, California. Tob Control 2022;:tobaccocontrol-2021-057135. doi:10.1136/tobaccocontrol-2021-057135
2 Vyas P, Ling P, Gordon B, et al. Compliance with San Francisco’s flavoured tobacco sales prohibition. Tob Control 2021;30:227–30. doi:10.1136/tobaccocontrol-2019-055549
3 Friedman AS. Further Considerations on the Association Between Flavored Tobacco Legislation and High School Student Smoking Rates-Reply. JAMA Pediatr 2021;175:1291–2. doi:10.1001/jamapediatrics.2021.3293
4 Maa J, Gardiner P. Further Considerations on the Association Between Flavored Tobacco Legislation and High School Student Smoking Rates. JAMA Pediatr 2021;175:1289–90. doi:10.1001/jamapediatrics.2021.3284
5 Friedman AS. A Difference-in-Differences Analysis of Youth Smoking and a Ban on Sales of Flavored Tobacco Products in San Francisco, California. JAMA Pediatr 2021;175:863–5. doi:10.1001/jamapediatrics.2021.0922
NOT PEER REVIEWED
In their response to my reply, the authors appear to not address mistakes in their analysis. It's important that any inaccurate statements be corrected for the benefit of other researchers trying to learn from this conversation. 1) The authors say in their response (and the paper) that there is no "after" period in the Friedman study. However, as reported by Gammon et al. (2022), there was an immediate decline in e-cigarette sales in San Francisco at the effective date. The authors need to explain how they can say there is no "post" period if other research clearly shows that e-cigarette sales declined starting July 2018. This is a central part of their argument and the paper unravels if there actually is a reduction in July 2018 as has been documented previously. The authors mention in their reply that they are aware of changes beginning in July 2018 ("merchant education and issuing implementing regulations"). The press may also have widely covered the effective date, which led to changes in youth's demand for e-cigarettes. Many retailers may have wished to become compliant immediately rather than wait until enforcement. All of these are valid potential mechanisms explaining why e-cigarette sales declined starting July 2018. So for the authors to say that Friedman doesn't have a "post" period is ignorant of both the literature and many valid reasons explaining why e-cigarette sales declined at...
NOT PEER REVIEWED
In their response to my reply, the authors appear to not address mistakes in their analysis. It's important that any inaccurate statements be corrected for the benefit of other researchers trying to learn from this conversation. 1) The authors say in their response (and the paper) that there is no "after" period in the Friedman study. However, as reported by Gammon et al. (2022), there was an immediate decline in e-cigarette sales in San Francisco at the effective date. The authors need to explain how they can say there is no "post" period if other research clearly shows that e-cigarette sales declined starting July 2018. This is a central part of their argument and the paper unravels if there actually is a reduction in July 2018 as has been documented previously. The authors mention in their reply that they are aware of changes beginning in July 2018 ("merchant education and issuing implementing regulations"). The press may also have widely covered the effective date, which led to changes in youth's demand for e-cigarettes. Many retailers may have wished to become compliant immediately rather than wait until enforcement. All of these are valid potential mechanisms explaining why e-cigarette sales declined starting July 2018. So for the authors to say that Friedman doesn't have a "post" period is ignorant of both the literature and many valid reasons explaining why e-cigarette sales declined at the effective date. 1a) The authors state in their abstract: "We also found that 2019 YRBSS data from San Francisco, California cannot be used to evaluate the effect of the sales restriction on all flavoured tobacco products in San Francisco as the YRBSS data for this city were collected prior to enforcement of the sales restriction." This is undercut by the above finding that the policy effective date led to declines in e-cigarette sales. Additionally, for other researchers in this space, I highly recommend the use of effective date in these types of policy evaluation efforts. Only one thing can change the effective date: legislation. In contrast, any number of things can change enforcement dates including government resources and willpower to enforce the laws. Further, enforcement intensity can change over time for many reasons. For these reasons, enforcement is a messy source of variation subject to all kinds of endogeneity concerns. For this reason, the vast majority of quasi-experimental research uses effective date, and I recommend that continue. However, it's reasonable to consider alternative timing points (such as enactment date and/or enforcement date) as sensitivity analyses. 2) The authors state: "Following the sales restriction, high school youth vaping and cigarette use declined between 2017 and 2019 in Oakland. These observations of patterns are purely descriptive and observational and are not statistically significant changes." The authors cannot say that cigarette use 'declined' between 2017 and 2019 if this change is not statistically significant. 3) The authors say in their paper that they received the YRBSS survey collection date from the CDC. In their reply, they appear to acknowledge that this was false and they actually received the data from the San Francisco School District. The reference should be corrected so that people know where to go for this type of information in the future. 4) This statement is not completely accurate: "If youth smoking rates increased similarly in Oakland following that city’s sales restriction, this would lend credence to the call for caution against flavoured tobacco sales restrictions. However, if the patterns differ, we should identify alternate explanations for the rise in San Francisco’s youth smoking prevalence." It's entirely possible smoking rates could continue to fall, just by less than in control groups as a result of flavor bans. That would still be evidence that flavor bans are increasing smoking (by reducing smoking cessation). The loose language the authors use here could lead people to make the wrong conclusion in other contexts.
NOT PEER REVIEWED
After seeing the response from the authors of “Youth tobacco use before and after flavored tobacco sales restrictions in Oakland, California and San Francisco, California” to the Rapid Response, “Scientific Concerns,” I was dismayed by the reply of the authors that dismissed the efforts of fellow scientists to rigorously discern the effects of flavored tobacco sales restrictions. The central point of their critique of Friedman’s paper is that it only contains pre-flavored tobacco product sales ban datapoints. Hence, a pre-post difference-in-differences design is inappropriate. Friedman most certainly had post-data in her sample. Despite the criticisms from Liu et al, they have not unseated her primary contribution; after a policy change, youth tobacco use behavior in San Francisco changed. Liu et al. provide no rigorous counter-analysis on this point. The author’s argument that no behavior had changed in San Francisco during YBRSS data collection in late 2018 falls apart at close inspection.
First, Liu et al. claim the flavored tobacco sales ban was not yet affecting retailer behavior in late 2018. This question is binary; it can either be answered yes or no. As of July 21, 2018, it was not legal to sell flavored tobacco products in San Francisco. No grace period was in place. Sales of all prohibited flavored products plummeted in the months after the policy became effective (Gammon et al., 2021 ; Table S1). However, sales did not reach zero,...
NOT PEER REVIEWED
After seeing the response from the authors of “Youth tobacco use before and after flavored tobacco sales restrictions in Oakland, California and San Francisco, California” to the Rapid Response, “Scientific Concerns,” I was dismayed by the reply of the authors that dismissed the efforts of fellow scientists to rigorously discern the effects of flavored tobacco sales restrictions. The central point of their critique of Friedman’s paper is that it only contains pre-flavored tobacco product sales ban datapoints. Hence, a pre-post difference-in-differences design is inappropriate. Friedman most certainly had post-data in her sample. Despite the criticisms from Liu et al, they have not unseated her primary contribution; after a policy change, youth tobacco use behavior in San Francisco changed. Liu et al. provide no rigorous counter-analysis on this point. The author’s argument that no behavior had changed in San Francisco during YBRSS data collection in late 2018 falls apart at close inspection.
First, Liu et al. claim the flavored tobacco sales ban was not yet affecting retailer behavior in late 2018. This question is binary; it can either be answered yes or no. As of July 21, 2018, it was not legal to sell flavored tobacco products in San Francisco. No grace period was in place. Sales of all prohibited flavored products plummeted in the months after the policy became effective (Gammon et al., 2021 ; Table S1). However, sales did not reach zero, not even after the January 1, 2019 enforcement date that Liu et al. purport as the critical date for a pre-post analytical design. This pattern is normal in sales data analyses of policy change. For example, even after Washington state had temporarily banned sales of flavored e-cigarettes in October 2019, sales of menthol-flavored e-cigarettes in November 2019 were still at 10% of pre-ban volumes. Sales crashed after the policy went into effect but never reached zero. Enforcement was incomplete. But to argue that the policy was not in effect in San Francisco or Washington after it was implemented is flat wrong. By late 2018, as measured in sales, retailer behavior had been affected by the policy.
Second, Liu et al. relying on work from Vyas et al. , argue that the policy was not truly affecting real-life outcomes in late 2018 because there was a low measured compliance rate with the flavored tobacco policy among retailers. Interestingly, in this case, Liu et al. judge whether retailers were affected by the flavored sales ban in a binary manner, favoring an interpretation that any retailer being out of compliance by selling one flavored product counts as not changing behavior at all. They assume that those 82% of retailers who violated the sales ban in San Francisco in December 2018 had not altered their behavior or wares since the policy came into effect in July of that year. Vyas points out that many retailers had questions about which products were covered by the ban, such as capsule cigarettes and cigars with “Sweet” descriptors. Vyas et al. frustratingly do not provide evidence about what it meant for retailers to be out of compliance in December 2018. But, judging from the details of the enforcement survey conducted, selling just one flavored tobacco product, even unknowingly, would make a retailer non-compliant. Further, given the importance of flavored tobacco sales in the US tobacco market, it would be reasonable to assume almost all tobacco retailers sold flavored products before the policy was in effect. So, at least 18% of retailers had changed their behavior to become fully compliant with the policy before the enforcement date, and I strongly suspect that many more reduced the number of non-compliant products on their shelves before enforcement (judging by changes in sales). Real-life changes in retailer behavior were in effect by late 2018.
For Friedman’s pre-post design to be inappropriate, as Liu et al. claim, the flavored tobacco sales ban must have had no effect on any person’s behavior before January 2019, when YBRSS data collection finished. The authors have repeatedly claimed that Friedman is not measuring what they think they are measuring. Still, her rejoinders that she meets the requirements to use a pre-post differences-in-differences analytical design with her chosen data are correct. Friedman should not retract her study.
Liu et al. should continue to look into important policy questions raised by Friedmans’ study. They and the rest of our field should use rigorous and appropriate analytical methods. We should learn as much as we can using all tools and data available. And the answers we find will depend as much as possible on the data and not on the convenience of findings of advocacy groups.
Finally, this case highlights the need for including more precise date of data collection identifiers in publicly available datasets. Had CDC included some of these data in the original YRBSS, this controversy could have been averted.
NOT PEER REVIEWED
I would like to make three comments by way of a brief post-publication review.
1. The impacts of vaping tax on smoking have been completely overlooked
For a study of e-cigarette taxation to have any public health relevance, it must consider the impact of e-cigarette prices on *cigarette* demand. Cigarettes and e-cigarettes are economic substitutes. The demand for one responds to changes in the price of the other, an idea well understood in economics and quantified through the concept of cross-elasticity. The paper appears to pay no regard to the impact of vaping taxes on cigarette demand, Yet such effects might easily overwhelm any benefits from reduced e-cigarette use - in fact, impact on demand for other tobacco products and the development of informal markets are by far the most important impacts of a vaping tax. By way of example, a 2020 paper by Pesko et al. [1] concluded:
"Our results suggest that a proposed national e-cigarette tax of $1.65 per milliliter of vaping liquid would raise the proportion of adults who smoke cigarettes daily by approximately 1 percentage point, translating to 2.5 million extra adult daily smokers compared to the counterfactual of not having the tax."
2. The case for reducing adult vaping by taxation has not been made
The authors have based their paper on an unexamined assumption that it is a justifiable goal of policy to lower rates of adult e-cigarette use. Why should...
Show MoreNOT PEER REVIEWED
I have a number of concerns with the paper as currently written.
1) The authors write: “Besides, none of the previous studies except Pesko et al (15) that examined the associations between vaping product excise tax adoption and ENDS use has accounted for the clustering of respondents within the same localities…” This is not accurate, as citation 19 also clusters standard errors at the locality level in all specifications.
2) The authors write: "A working paper reported reduced ENDS sales, but not ENDS use prevalence or behaviours, after implementation of a vaping product excise tax policy. (19)” This is not accurate, as the cited study uses the magnitude of e-cigarette tax values, rather than an indicator variable for tax implementation. States have adopted e-cigarette taxes of different magnitudes and a number of them (such as California) have changed the magnitudes of these taxes after adoption. All of this variation is used in citation 19, contrary to the current study’s description. It's also unclear from the sentence whether citation 19 studied use and found imprecise estimates, or did not study use. It's the latter and this should be clarified. It's also unclear why the authors did not use magnitude of e-cigarette taxes themselves in the current paper, as has been commonly done in the referenced literature.
3) Authors write they use a “nationally representative sample of US young adults.” I do not beli...
Show MoreNOT PEER REVIEWED
We appreciate the comments from Bates and the opportunity for us to respond and clarify.
First, Bates' argument heavily relies on the assumption that e-cigarettes and combustible cigarettes are substitutes, which is theoretically possible as some consider vaping as a harm reduction alternative to combustible cigarettes. Empirically, however, there have been mixed findings about whether e-cigarettes and combustible cigarettes are substitutes (or complements). Bates cited Pesko et al. (2020) that concludes e-cigarettes and combustible cigarettes are substitutes, whereas other studies have shown that they are complements. For example, Cotti et al. (2018) found that higher cigarette excise taxes, in fact, decrease sales of both e-cigarettes and combustible cigarettes, suggesting that they are complements. Such mixed results abate Bates' argument that taxing ENDS could lead to more use of combustible cigarettes.
Second, Bates might have ignored that our study focused on young adults aged 18-24 years rather than general adults when examining the effect of vaping product tax on e-cigarette use. Although Pesko et al. (2020) suggests that e-cigarettes and combustible cigarettes are substitutes, the findings are based on the general adult population (average age: 55 years) which may not be generalizable to the young adult population. In fact, one study conducted by Abouk and Adams (2017) indicates that e-cigarettes and combustible ci...
Show MoreNOT PEER REVIEWED
We thank Pesko for his comments and the opportunity for us to respond and clarify.
First, we appreciate Pesko’s clarification that Cotti et al. (2020) clustered standard errors to account for clustering. In the present study, we used multilevel analysis not only to account for clustering of respondents (i.e., design effects) but also to incorporate different error terms for different levels of the data hierarchy which yields more accurate Type I error rates than nonhierarchical methods where all unmodeled contextual information ends up pooled into a single error term of the model.
Second, we understand that Cotti et al. (2020) evaluated the magnitude of e-cigarette tax values, which does not contradict to our statement because our study focused on the effects of e-cigarette excise tax policies on individual e-cigarette use and prevalence rather than aggregated sales at state or county levels. We also clearly described the reason why we examined the e-cigarette excise tax policy implementation indicator rather than its magnitude in our paper’s discussion section.
Third, our study used a nationally representative sample of young adults (rather than a nationally representative sample of general adult population). While we understand Pesko’s concern that a sample’s representativeness might be lost when subgroups are explored, we believe our use of sampling weights in analysis has reduced such a concern.
Fourth, in Table 3,...
Show MoreNOT PEER REVIEWED
Clive Bates’ commentary on our paper repeats claims we previously addressed [1]. Here, we address seven points, the first is contextual and the remaining are raised in his letter.
1. We note the failure of the author to acknowledge Māori perspectives, in particular their support for endgame measures, concerns in relation to harm minimisation [2] as outlined in his “all in” strategy, and ethical publishing of research about Indigenous peoples. [3]
2. We reject the assertion that the basis of our modelling is “weak”. While there is uncertainty around the potential effect of denicotinisation, as this policy hasn’t been implemented, there are strong grounds to believe that it will have a profound impact on reducing smoking prevalence. This is based on both theory and logic (i.e., nicotine is the main addictive component of cigarettes and why most people smoke), and the findings of multiple randomized controlled trials (RCTs) showing that smoking very low nicotine cigarettes (VLNCs) increases cessation rates for diverse populations of people who smoke [4-7].
Our model’s estimated effect on smoking prevalence had wide uncertainty, namely a median of 85.9% reduction over 5 years with a 95% uncertainty interval of 67.1% to 96.3% that produced (appropriately) wide uncertainty in the health impacts. The derivation of this input parameter through expert knowledge elicitation (EKE) is described in the Appendix of our paper. Univariate se...
Show MoreNOT PEER REVIEWED
I have published a summary critique of this modelling exercise on PubPeer. [1] This summarises concerns raised in post-publication reviews of this paper while it was in pre-print form by experts from New Zealand and Canada, and me. [2][3]
By way of a brief summary:
1. All the important modelled findings flow from a single assumption that denicotinisation will reduce smoking prevalence by 85% over five years. Yet the basis for this assumption is weak and disconnected from the reality of the market system being modelled.
2. The central assumption is based partly on a smoking cessation trial that bears no relation to the market and regulatory intervention that is the subject of the simulation. Even so, the trial findings do not support the modelling assumption.
3. The central assumption also draws on expert elicitation. Yet, there is no experience with this measure as it would be novel, and there is no relevant expertise in this sort of intervention. Where experts have been asked to assess the impacts, their views diverge widely, suggesting that their estimates are mainly arbitrary guesswork.
4. The authors have only modelled benefits and have not included anything that might be a detriment or create a trade-off. The modelling takes no account of the black market or workarounds. These are inevitable consequences of such 'endgame' prohibitions, albeit of uncertain size. Though it may be challenging to mo...
Show More¶ The authors make some points in their article that are reasonable: 1) the generalizability of San Francisco's flavor ban compared to other places is an open question, and 2) the original study uses the San Francisco ban effective date rather than enforcement date. The original author (Friedman), who does not accept tobacco industry funding and is a well-respected scientist in the field, had pointed to both facts in her original article. So that information isn’t new.
Show More¶ The current authors appear to construct a straw man argument claiming that Friedman argued that she was studying the effect of San Francisco enforcing its flavor ban policy. Friedman specifically wrote in her original article that she was studying, “a binary exposure variable [that] captured whether a complete ban on flavored tobacco product sales was in effect in the respondent’s district on January 1 of the survey year.” She specifically uses effect in the above sentence, so there is no ambiguity that she is studying effective date. San Francisco’s flavor ban effective date was July 2018 (Gammon et al. 2021).
¶ The authors found new information that the San Francisco YRBSS survey was collected between November to December of 2018. Gammon et al. 2021 (Appendix Figure 1) shows that flavored e-cigarette sales declined in San Francisco between the effective date and the end of August 2018 (compensating for a 30-day look-back period for the YRBSS question wording), even though the flavor ban...
Pesko’s central argument is that it does not matter that Friedman’s assessment of the effect of San Francisco’s ban on the sale of flavored tobacco products is not based on any data collected after the ban actually went into force. In particular, Friedman’s “after” data were collected in fall 2018, before the ordinance was enforced on January 1, 2019.[1] Pesko incredibly argues that Friedman’s “before-after” difference-in-difference analysis is valid despite the fact that she does not have any “after” data.
Pesko justifies this position on the grounds that the effective date of the San Francisco ordinance was July, 2018. While this is true, it is a matter of public record that the ordinance was not enforced until January 1, 2019 because of the need for time for merchant education and issuing implementing regulations.[2]
Friedman is aware of the fact that the enforcement of the ordinance started on January 1, 2019 and used that date in her analysis. In her response[3] to critiques[4] of her paper, she stated “retailer compliance jumped from 17% in December 2018 to 77% in January 2019 when the ban went into effect.” Friedman thought the YRBSS data was collected in Spring 2019; she only learned that the “2019” San Francisco YRBSS data she used were in fact collected in fall 2018 from our paper.[1]
Rather than simply accepting this as an honest error and suggesting Friedman withdraw her paper, Pesko is offering an after-the-fact justification for the cl...
Show MoreNOT PEER REVIEWED
Show MoreIn their response to my reply, the authors appear to not address mistakes in their analysis. It's important that any inaccurate statements be corrected for the benefit of other researchers trying to learn from this conversation. 1) The authors say in their response (and the paper) that there is no "after" period in the Friedman study. However, as reported by Gammon et al. (2022), there was an immediate decline in e-cigarette sales in San Francisco at the effective date. The authors need to explain how they can say there is no "post" period if other research clearly shows that e-cigarette sales declined starting July 2018. This is a central part of their argument and the paper unravels if there actually is a reduction in July 2018 as has been documented previously. The authors mention in their reply that they are aware of changes beginning in July 2018 ("merchant education and issuing implementing regulations"). The press may also have widely covered the effective date, which led to changes in youth's demand for e-cigarettes. Many retailers may have wished to become compliant immediately rather than wait until enforcement. All of these are valid potential mechanisms explaining why e-cigarette sales declined starting July 2018. So for the authors to say that Friedman doesn't have a "post" period is ignorant of both the literature and many valid reasons explaining why e-cigarette sales declined at...
NOT PEER REVIEWED
Show MoreAfter seeing the response from the authors of “Youth tobacco use before and after flavored tobacco sales restrictions in Oakland, California and San Francisco, California” to the Rapid Response, “Scientific Concerns,” I was dismayed by the reply of the authors that dismissed the efforts of fellow scientists to rigorously discern the effects of flavored tobacco sales restrictions. The central point of their critique of Friedman’s paper is that it only contains pre-flavored tobacco product sales ban datapoints. Hence, a pre-post difference-in-differences design is inappropriate. Friedman most certainly had post-data in her sample. Despite the criticisms from Liu et al, they have not unseated her primary contribution; after a policy change, youth tobacco use behavior in San Francisco changed. Liu et al. provide no rigorous counter-analysis on this point. The author’s argument that no behavior had changed in San Francisco during YBRSS data collection in late 2018 falls apart at close inspection.
First, Liu et al. claim the flavored tobacco sales ban was not yet affecting retailer behavior in late 2018. This question is binary; it can either be answered yes or no. As of July 21, 2018, it was not legal to sell flavored tobacco products in San Francisco. No grace period was in place. Sales of all prohibited flavored products plummeted in the months after the policy became effective (Gammon et al., 2021 ; Table S1). However, sales did not reach zero,...
Pages