Article Text
PDF
Abstract
Background To prioritise tobacco control interventions, simulating their health impacts is valuable. We undertook a systematic review of tobacco intervention simulation models to assess model structure and input variations that may render model outputs non-comparable.
Methods We applied a Medline search with keywords intersecting modelling and tobacco. Papers were limited to those modelling health outputs (eg, mortality, health-adjusted life years), and at least two of cancer, cardiovascular and respiratory diseases. Data were extracted for each simulation model with ≥3 arising papers, including: model type, untimed or with time steps and trends in business-as-usual (BAU) tobacco prevalence and epidemiology.
Results Of 1911 papers, 186 met the inclusion criteria, including 13 eligible simulation models. The SimSmoke model had the largest number of publications (n=46), followed by Benefits of Smoking Cessation on Outcomes (n=12) and Tobacco Policy Model (n=10). Two of 13 models only estimated deaths averted, 1 had no time steps, 5 had no future trends in BAU tobacco prevalence, 9 had no future trends in BAU disease epidemiology and 7 had no time lags from quitting tobacco to reversal of health harm.
Conclusions Considerable heterogeneity exists in simulation models, making outputs substantively non-comparable between models. Ranking of interventions by one model may be valid. However, this may not be true if, for example, interventions that differentially affect age groups (eg, a tobacco-free generation policy vs increased cessation among adults) do not account for plausible future trends. Greater standardisation of model structures and outputs will allow comparison across models and countries, and for comparisons of the impact of tobacco control interventions with other preventive interventions.
- prevention
- economics
- litigation
Statistics from Altmetric.com
Introduction
Policymakers are faced with numerous options and decisions on tobacco control interventions that have to be made with finite budgets and resources, limited timeframes and often sudden and short political windows of opportunity. To strengthen the case for choosing one tobacco control intervention over the other, or choosing a package of multiple tobacco control intervention over another, and even comparing tobacco control interventions with other public health interventions (those for nutrition, physical activity and reducing carbon emissions), robust methods to compare the expected public health impact of interventions are necessary. To allow additional comparison with other preventive programmes, output metrics comparable across intervention and disease domains are needed (eg, life years, not tobacco prevalence), as well as comparable model structure and assumptions. Policy decision-making informed by comparable and plausible estimation of health gains of a range of interventions that include tobacco control will lead to better utilisation of finite health and economic resources.
Empirical data are not available for the future; to estimate future impacts of tobacco control interventions we have to rely on projections and simulation models. These models leverage off existing retrospective data on tobacco use and disease epidemiology, ideally for generating plausible future projections rather than assuming the future will be the same as today, and allow comparisons of tobacco control interventions using a number of public health metrics (eg, morbidity rates, mortality rates, life years and health-adjusted life years (HALYs) gained, health equity and cost-effectiveness). Different simulation modelling approaches (eg, decision trees, comparative risk assessment, Markov models, system dynamics models, Markov chain models, discrete event simulation and agent-based simulation models) have different sets of advantages and disadvantages with regard to feasibility, data requirements, ability to model time steps and demands on computational power.1 Epidemiology is core to simulation modelling of tobacco control interventions, being required for both conceptualising and specifying model structure, and deriving accurate and coherent input parameters. For example, ignoring the decline in trends in tobacco use reported across many high-income countries2 can lead to overestimating the intervention impacts—assuming this decline will continue into the future absent the intervention of interest. Assumptions that quitting tobacco leads to immediate health benefits and ignoring time lags between quitting and accruing health benefits can be misleading and also overestimate prospective health impacts.3
The application of tobacco simulation models has increased substantially over the last three decades. But, there is a lack of clarity on variations in model structure and parameterisation. Three systematic reviews have collated the evidence from simulation models applied in tobacco control.4–7 Bolin4 reviewed health economic evaluations of smoking-cessation interventions and critically examined six different simulation models (Benefits of Smoking Cessation on Outcomes (BENESCO), Health Economic Consequences of Smoking (HECOS), PREVENT, Chronic Disease Model (CDM), Quit Benefits Model (QBM) and Tobacco Policy Model) for methodological properties. Although the Bolin review comprehensively examined the models’ structures and assumptions (although for cessation interventions only), other new models and model variations in reviewed models have emerged after this review.8 9 The Bolin review also excluded a widely published tobacco simulation model.10 Another review that was also published in 2012, examined studies only on cost-effectiveness of nicotine replacement therapies, varenicline and bupropion.6 The most recently published systematic review in 2016 examined the evidence on mathematical modelling in tobacco control research, but only reported preliminary results from their research on different model types and public health outcomes modelled5; models were not examined critically. Hence, there is a gap in knowledge on the epidemiological characteristics and comparability of simulation models in tobacco control that we aimed to address.
Our systematic review aimed to collate and examine the epidemiological characteristics of widely applied simulation models that quantify the population health impacts of tobacco control interventions, focusing on model structure variations that may render model outputs non-comparable.
Methods
A review was performed to identify, collate and synthesise evidence from simulation models applied to model effectiveness of tobacco control interventions. As we were reviewing models, rather than publications of effect sizes usually the focus of a ‘systematic review’, our protocol does not exactly accord with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement. Nevertheless, we were systematic in our literature search and data extraction.
The research question was:
‘What are the key model characteristics of widely applied tobacco simulation models?’
Search strategy
An initial search of MEDLINE (PubMed platform) was undertaken to identify index terms and Medical Subject Headings terms. The actual literature search strategy was then specified. Search terms were specified to identify papers combining simulation models (‘mathematical model’ (tiab) OR ‘mathematical models’ (tiab) OR Simulat* (TIAB) OR Markov (TIAB) OR Multistate (TIAB) OR Multi-state (TIAB) OR Forecast* (TIAB) OR Computer Simulation (MH)) and tobacco products (Smoking (TIAB) OR Tobacco use (TIAB) OR Cigarette* (TIAB) OR Smokeless tobacco OR Chewing tobacco OR Tobacco Products (MH) OR Tobacco (MH) OR Tobacco Use (MH) OR Tobacco, Smokeless (MH) OR Tobacco Smoking (MH) OR Tobacco, Waterpipe (MH) OR Smokers (MH) OR Smoking, Non-Tobacco Products (MH) OR ‘E-cigarette’ (TIAB) OR ‘E-cigarettes’ (TIAB) OR ‘E-cig’ (TIAB) OR ‘e-cigs’ (TIAB) OR ‘Electronic cigarette’ (TIAB) OR ‘Electronic cigarettes’ (TIAB) OR ‘electronic nicotine delivery system’ (TIAB) OR ‘electronic nicotine delivery systems’ (TIAB) OR Vaping (TIAB) OR Vaper (TIAB) OR Vapers (TIAB) OR Vape (TIAB) OR ((Vapour (TIAB) OR vapours (TIAB) OR vaporised (TIAB) OR vapour (TIAB) OR vapours (TIAB) OR vapourized (TIAB) OR Electronic (TIAB) OR aerosol (TIAB) OR aerosols (TIAB) OR aerosolized (TIAB)) AND (nicotine (TIAB) OR tobacco (TIAB) OR smoking (TIAB))) OR ‘e-juice’ (TIAB) OR ‘e5 juices’ (TIAB) OR ‘e-liquid’ (TIAB) OR ‘e-liquids’ (TIAB)). The search was performed on, and includes results until, 13 May 2019.
Model selection process
We applied a two step-strategy to, first, identify eligible simulation studies, then second to group these studies by the simulation model they used, the unit of review for our research question.
Criteria for selection of studies
The study had to be published in English using a model that simulated:
the effect of actual or hypothesised tobacco control intervention(s);
the health impact in terms of a summary health or mortality measure (mortality, morbidity, life years (LYs), HALYs (subsuming quality-adjusted life years (QALYs) and disability-adjusted life years (DALYs)), life expectancy (LE) or health-adjusted life expectancy (HALE);
either:
directly estimated all-cause mortality impacts;
or estimated impacts through at least two of three major tobacco-related disease groupings: cardiovascular, respiratory and cancer. (Given tobacco causes many diseases, we required models simulating more than just one disease domain. As cardiovascular, respiratory and cancer diseases represent the three largest disease domains affected by tobacco, we required at least two of these domains included.)
Studies simulating tobacco prevalence changes alone, without a summary health metric, were therefore excluded.
Criteria for selection of models
Studies were grouped by the simulation model used, which was often easily determined due to clear naming (eg, SimSmoke). Of the remaining studies, we searched for common authors, grouping all remaining studies into mutually exclusive simulation models.
Full-text review was then conducted on the three (or all if less than three) most recent publications using each model, and all other studies where it was not possible to determine the actual simulation model used from the abstract. Studies were excluded if full-text review disclosed more information relevant to inclusion and exclusion criteria, and adjustments made of allocation of studies to model groupings as necessary. Additional relevant publications were sourced from the reference lists of included studies.
Screening of studies and models
Title and abstract screening, based on set inclusion and exclusion criteria, was conducted by AS for all publications and cross-checked by TB independently.
Data extraction on model characteristics
From the above finalised groupings of papers by underlying simulation model used, we selected simulation models used in three or more published studies for detailed analysis. (This a priori decision was based on efficient use of researcher time, and that the more commonly published models are more influential and relevant.) For data extraction, we used the three most recent papers published for that model and other ‘source’ publications identified from citations (eg, the commonly cited methodological papers or reports that describe the simulation model in detail).
A data extraction form was developed by all three coauthors, piloted on 10% of selected models, then modified to adequately capture model characteristics that (a priori) we hypothesised to be important for both understanding the model structure, and comparability of model outputs:
model type (decision trees, comparative risk assessment, Markov models, system dynamics models, Markov chain models, discrete event simulation and agent-based simulation models);
epidemiological characteristics of the model:
presence of time component (time steps and time horizon);
specification of future trends under business as usual (BAU) for:
tobacco prevalence/use;
all-cause mortality and cause-specific epidemiological parameters (incidence rates, case fatality rates and disease mortality rates);
characterisation of tobacco exposure (eg, current smoking status only, more sophisticated measures of smoking history and intensity, years since quit);
the types of tobacco or nicotine products included (eg, cigarettes, smokeless tobacco use, electronic or alternative nicotine delivery systems);
diseases included;
specification of time lags from cessation to changing disease rates;
health metrics estimated (eg, deaths averted, mortality rates, morbidity rates, LYs, HALYs, LE, HALE);
if the model also estimated health expenditure impacts (ie, cost offsets);
data sources for cause-specific and all-cause mortality relative risks by level of tobacco use;
quantification of uncertainty (scenario as well as probabilistic using Monte Carlo simulation);
model calibration and validation;
whether the model had been applied:
for cross-country comparisons;
in both high-income and low-income and middle-income countries;
to compare subpopulation or heterogeneity within countries (eg, socioeconomic inequalities).
We recorded information with specific citations when only one or a few papers reported a specific model characteristic (eg, time lags were included in only 2 of the 46 selected SimSmoke studies).
Evidence synthesis
We used a narrative synthesis of evidence by prevailing themes under logical headings, consistent with the objectives.
Results
Search results
Overall, 1911 papers were identified in the search, and 186 (10%) were eligible for full-text review on title and abstract screening (figure 1). Of these, 114 (61%) and an additional 8 studies sourced through reference lists of selected studies and a previous systematic review (n=122) were found to be grouped in 13 models with 3+ publications, based on author-named models or common authors across publications (Abridged SimSmoke, BENESCO, Burden of Disease Epidemiology, Equity & Cost-Effectiveness Programme (BODE3) EQUIPTMOD, Common author (CA) Higashi, CA Hoogenveen, Dynamic Model of Health Impact Assessment (Dynamo-HIA), Philip Morris International (a tobacco company) (PMI), SimSmoke, Tobacco Policy/CA Ahmad, CA Vos, CA Warner & Mendez and WHO-CHOosing Interventions that are Cost Effective (WHO-CHOICE)) (table 1). Fifty-three studies were reviewed in depth to characterise the 13 simulation models (table 1).
PRISMA flowchart.
Studies informing the selected tobacco simulation models
How the models have been applied
To give context, table 2 summarises the applications of the 13 models. The models have been mostly used for interventions targeting cigarette smoking, although (for example) SimSmoke has also modelled smokeless tobacco.11 A large variety of tobacco control interventions have been simulated, ranging from public policies (taxation, smoke-free workplaces), mass media campaigns to smoking cessation treatments. By interventions, EQUIPTMOD12 and SimSmoke13 included the largest number of tobacco control interventions. The models have been mostly used for estimating the impacts of single interventions; exceptions include Abridged SimSmoke,8 SimSmoke,13 EQUIPTMOD,12 BODE3 14 CA Higashi15 and WHO-CHOICE,16–18 where the impact of two or more interventions in combination were assessed.
Summary of how the models have been applied
The majority of simulation models have been applied to high-income countries; exceptions included adaptations of the SimSmoke, Abridged SimSmoke,8 19 20 CA Higashi15 and WHO-CHOICE16–18 models which were applied in low-income and middle-income countries. Multicountry comparisons are carried out by at least 3 of the 13 models (eg, Abridged SimSmoke,8 EQUIPTMOD12 and Dynamo-HIA21). Of the 13 models, 1 was developed by funding through a drug company (BENESCO22–24) and 1 through a tobacco company (PMI25 26).
Outcomes modelled, population targets and tobacco characterisation
Nine of 13 included models generated QALYs or DALYs as summary measures of population health impact compared with 4 (Abridged SimSmoke, PMI, SimSmoke and CA Warner & Mendez) onto just mortality, smoking-attributable deaths averted and cumulative LYs (table 3). Eight models estimated cost impacts (BENESCO, BODE3, EQUIPTMOD, CA Higashi, Tobacco Policy, CA Vos, CA Warner & Mendez and WHO-CHOICE). Among these, the CA Higashi and WHO-CHOICE models only included intervention costs and the Tobacco Policy modelled only health system cost offsets (ie, the future cost savings from averted disease), while others modelled both intervention costs and health system cost offsets.
Tobacco characterisation, tobacco control intervention types and specification and outcomes modelled
Heterogeneity of intervention impacts by population subgroups were reported in all but two models (Tobacco Policy and WHO-CHOICE). While WHO-CHOICE modelled intervention impacts by population subgroups, the papers did not report intervention impacts by subgroups. Most commonly, heterogeneity by age and sex was reported. Additional reporting of heterogeneity of impacts included ethnicity (BODE3, SimSmoke), country-level economic development (Abridged SimSmoke), country-level geographic region (DYNAMO-HIA), workers (CA Warner & Mendez) and a psychiatric population (BENESCO) (table 3). Two of the 13models (BENESCO and EQUIPTMOD) modelled the impact of tobacco control interventions only among smokers (table 3).
Model built and variations in epidemiological characteristics of selected models
All models except for Abridged SimSmoke were reported or inferred to be Markov-based models. Models mainly included age-sex cohorts of individuals going through annual cycles or time steps. None of the models were full microsimulation models (eg, simulating individuals through both their smoking and disease histories), although two models (Tobacco Policy/CA Ahmad and PMI) include microsimulation of smoking history that was then linked to aggregate estimation of health impacts. Nearly half of the models (7/13) simulated the impact of tobacco interventions over the full lifetime of the simulated populations (table 4).
Epidemiological and model structure characteristics of the simulation models
Nine of the 13 models quantified the impact of tobacco interventions by modelling mortality due to smoking through diseases. Lung cancer, chronic obstructive pulmonary disease (COPD), stroke and coronary heart disease (CHD) featured across the majority of these models, but explicit mention of incorporating time trends in epidemiological measures including incidence rates and case fatality rates were only included in the BODE3 and CA Vos models. Except WHO-CHOICE, all models treated diseases as independent of each other; WHO-CHOICE used the underlying PopMod model27 that only allows two diseases (thus IHD and COPD were treated as one combined disease, cerebrovascular disease treated separately and lung cancer appended to all-cause mortality) but does allow for disease dependency (here, combined IHD and COPD with cerebrovascular). The remaining four models (Abridged SimSmoke, SimSmoke, Tobacco Policy/CA Ahmad and CA Warner & Mendez) estimated impacts directly onto mortality, bypassing changing disease epidemiology due to tobacco use. Time trends in all-cause mortality were incorporated in SimSmoke, CA Warner & Mendez and WHO-CHOICE (table 4).
Eight of the 13 models explicitly included projected (and therefore changing) future tobacco use prevalence in their BAU projections; the five exceptions were Abridged SimSmoke, BENESCO, EQUIPTMOD, Dynamo-HIA and WHO-CHOICE (which used smoking impact ratios). Six of the 13 models included time lags for decay in relative risks (RR) of disease incidence or all-cause mortality after cessation (BODE3, CA Hoogenveen, Dynamo-HIA, PMI, SimSmoke and CA Vos). The RR for the strength of association (and therefore implied causation) of smoking with disease or mortality were most commonly taken from the American Cancer Prevention Study II and Doll et al.28 Ten of the 13 models explicitly stated that heterogeneity in the RRs was included (age, sex, country, ethnicity and country-level economic development) (table 4).
Probabilistic uncertainty around impact was modelled and reported in half (7/13) of the models (BENESCO, BODE3, EQUIPTMOD, CA Higashi, CA Vos, CA Warner & Mendez and WHO-CHOICE) (table 4).
We were able to determine that some form of validation or calibration of models was also carried out by nearly half (6/13) of the models (Abridged SimSmoke, BODE3, CA Higashi, SimSmoke, Tobacco Policy/CA Ahmad and CA Warner & Mendez) (table 4).
Discussion
There was marked variability in many aspects of model structures and parameterisation. Of the 13 simulation models: 2 only estimated deaths averted; 1 had no time steps; 5 had no future trends in BAU tobacco prevalence; 9 had no future trends in BAU disease epidemiology and 7 had no time lags from quitting tobacco to reversal of health harm. These issues all highlight the desirability for more standardisation of models. Also, despite the importance of resource constraints for health sectors around the world, only just over half considered some cost aspect.
Despite providing an up-to-date review of the status of tobacco control models, this review has several limitations. First, we limited our review to models with three of more publications, so as to report more on commonly used models and models with a greater likelihood of sufficient documentation to inform our analysis. It is possible that including models published only once or twice may have altered our conclusions, although we suspect not. (We provide a full list of studies, including those with models only published once or twice, in an online supplement for interested readers.) Second, we deliberately undertook a targeted approach to data extraction by identifying and using the most recent and relevant methodological publications; it was not possible, or efficient, to read all publications and technical documentation for each relevant model. Therefore, the results in tables 2 and 3 should be treated as context and scene setting, as (for example) we may not have captured all interventions for each model. The main analytical thrust of this paper, however, was around the data presented in table 4 on model structure and parametrisation, for which a targeted extraction from selected papers should suffice. Third, we have not attempted to summarise the magnitude of change for varying assumptions; we encourage future research to (say) examine the influence of varying time lag and tobacco prevalence trend assumptions. Our search was limited to one electronic database (Medline using the PubMed platform) and was restricted to studies published on or before 31 May 2019. However, given we included models with three or more publications it is less likely to exclude any relevant model. Finally, we included models that estimated impacts through at least two of three major tobacco-related disease groupings: cardiovascular, respiratory and cancer. While this risks exclusion of quality models that estimate impact through only one disease, we stand by our position that the public health benefits of tobacco control interventions is best captured comprehensively and comparably through including multiple disease groupings.
Our findings are consistent with the findings from the existing systematic reviews on tobacco simulation models.4–6 Bolin4 in their review (which was limited to a few of the selected models), found that studies failed to sufficiently report model structures, report input data, analyse uncertainty and test validity of the models used. Similar concerns related to inadequate reporting of model characteristics were also raised by another systematic review.7 While we concur model documentation could and should be better, compared with these two systematic reviews (and aided by focusing on the models with three or more publications), we were able to extract a more comprehensive picture of variations between model structures of interest: time trends in BAU, lag time, heterogeneity and probabilistic as well as discrete uncertainty. However, this was not straightforward as seldom these characteristics were mentioned explicitly and often needed to be inferred.
Does variability in tobacco simulation model structure and output metrics matter that much? Perhaps sometimes not, for example, if one is comparing the estimated impacts of interventions assessed by one model, and there is a wide spread of impact (eg, large variation in HALYs), then the ranking of interventions by health impact is probably valid. However, even this fall-back position will come under challenge in some circumstances. For example, a tobacco-free generation intervention accrues health gains to younger cohorts compared with a cessation intervention. A model that does not allow for (likely) reductions in future tobacco use8 29 will relatively overestimate health gains for a tobacco-free generation (as they were going to have lower smoking rates in the future anyway) compared with a cessation intervention.
We suggest that variability in models will often matter. First, variability means that some models are likely to simulate future impact of tobacco control interventions more accurately than others. Given the models are estimating the future, it is impossible to have certainty that their estimates are ‘correct’ but increasing levels of plausibility and comparability are desirable. Second, if the goal of tobacco intervention modelling is to allow additional comparison with other preventive programmes (eg, interventions to reduce the obesogenic environment, screening programmes, etc), then accurate or at least comparable model structure and assumptions are necessary.7 Consider time lags from quitting to reduced disease incidence and mortality.3 If the tobacco simulation model assumes instantaneous reversal of risk on cessation, it will substantively overestimate the health gains. (We note the same criticism can be levelled at dietary intervention simulation—it probably takes some years for cancer risk to fall after body mass index reduction.) Third, if used for cost-effectiveness (and putting aside costing accuracy in this paper), variation in the HALYs and QALYs will have a major impact on the cost-effectiveness ratios.
We contend that decision-making, and research prioritisation, should be informed by comparable and plausible estimation of health gains (and costs) of a range of interventions. We do not pretend this is an easy task, but we do argue that there are some relatively straightforward steps that researchers and analysts can take to improve comparability or at least confidence in simulation modelling. First, clear documentation of models is required. This issue has also been underscored by the systematic review on models by Bolin.4 This could be assisted by reporting guidelines for epidemiological modelling for journals, for example, building on guidelines for health economic modelling by ISPOR.30 Future studies can examine disagreements between tobacco simulation models and reporting guidelines (eg, ISPOR). Second, models that report deaths averted or postponed should ensure that the defined time window appropriately reflects projected changes in disease outcomes expected. For a model projecting out into the future, everyone is still going to die. The deaths averted and postponed metric includes deaths averted by a matter of hours to decades. LYs gained, and preferably HALYs gained, is more meaningful. Recognising, however, that tobacco control advocates and policy makers find the deaths averted statistic useful, we strongly recommend that a defined and meaningful short-term and long-term time window is used (e.g. deaths averted in the next 10 years, which is the difference in number of deaths in BAU and under intervention in the next 10 years). Third, allowing for likely future trends in tobacco consumption and epidemiological parameters is obligate, even if models use built-in trends for the next decade or so and then hold these parameters constant (while also testing these assumptions with scenario analyses). Fourth, time lags should be explicitly incorporated—as they should too for other intervention models (eg, physical activity, diet, etc).
What this paper adds
Tobacco simulation models can facilitate health and health economic comparisons between both different tobacco control interventions as well as multiple other health sector interventions; such information can potentially facilitate evidence-informed policymaking.
There is a lack of knowledge around the methodological diversity of different tobacco simulation models and around how this can be reduced.
In this review, we found that considerable heterogeneity exists in epidemiological structure of published tobacco simulation models.
There is a need for greater methodological standardisation across such models to maximise meaningful comparisons between tobacco control interventions and with other health sector interventions.
Supplemental material
Ethics statements
References
Footnotes
Twitter @drankursingh99
Contributors AS contributed to design, analysis, interpretation of results and prepared draft of the manuscript. NW contributed to interpretation of results and providing critical feedback on the manuscript. TB led the conception of the project and design, analysis, interpretation of data and oversaw the drafting of the manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests TB and NW have led the development of the BODE3 tobacco control model, and AS has applied BODE3 model.
Provenance and peer review Not commissioned; externally peer reviewed.