Objective To examine the generalisability of findings from clinical trials of individuals with nicotine dependence to a large general population sample.
Methods Eligibility criteria were drawn from typical criteria of clinical trials for nicotine dependence. The National Epidemiological Survey on Alcohol and Related Conditions (NESARC), a large national sample of the US population, was used to assess how many potentially eligible people would fulfil the eligibility criteria. NESARC interviewed more than 43 000 adults aged 18 years and older. We applied a standard set of eligibility criteria representative of smoking cessation clinical trials to all the 4962 adults with nicotine dependence in the past 12 months, and then to a subgroup of participants motivated to quit (n=4121).
Results We found that approximately six out of 10 participants (65.89%) with nicotine dependence were excluded by at least one criterion. In the subgroup of nicotine-dependent participants motivated to quit, more than half (58.60%) were excluded by at least one criterion. For the overall sample, smoking 10 cigarettes per day or less and lack of motivation to quit were the two criteria leading to exclusion for the greatest percentage of individuals (32.02% and 17.60%, respectively). For the sample motivated to quit, smoking 10 cigarettes or fewer per day and current depression led most frequently to exclusion (33.79% and 15.71%, respectively).
Conclusions Further studies and interventions should explore the efficacy of tobacco treatment interventions in a larger segment of the population, notably in the subpopulations of people with nicotine dependence who smoke fewer than 10 cigarettes per day or who have comorbid depression.
Statistics from Altmetric.com
Clinical guidelines are developed based on the evidence obtained using clinical trials.1–4 In smoking cessation trials, exclusion and eligibility criteria are often used in order to maximise treatment efficacy and safety.5 However, they may impair the external validity of the study, since they often exclude a substantial proportion of participants, resulting in a selection bias,5 and extending the gap between research and clinical practice.6 Common exclusion criteria include age, current or past psychiatric/drug disorder, minimal levels of tobacco use and medical conditions.7 There is a risk that this selection of participants affects the results of the treatment trial for nicotine dependence.8 9 The impact of eligibility/exclusion criteria on the generalisability of clinical trials has been described for antidepressant efficacy trials,5 10–14 antipsychotic efficacy trials15–17 and clinical trials for alcohol dependence18–21 and cannabis dependence.22 The percentage of subjects excluded by these criteria ranged between 50.5% and 75.8% in these studies.10 18
The impact of eligibility criteria in smoking cessation trials has been discussed in the literature.7 23–29 As called for by CONSORT guidelines, several studies reported the reasons for ineligibility.7 28 For example, Robinson et al screened 1347 adolescents for a nicotine replacement treatment trial, and found that only 24.4% were eligible for inclusion in the trial.28 The main reason for ineligibility was a failure to meet minimum requirements regarding the number of cigarettes smoked per day and/or a low level of nicotine dependence (criterion present in 39.1% of ineligible individuals).28 More recently, Kamholtz et al assessed 97 non-eligible and 201 eligible participants in a laboratory research on smoking.7 They reported that the main reasons for ineligibility were current alcohol and substance use disorders (present in 23.7% and 11.3% of ineligible individuals, respectively) and failure to meet minimum requirement regarding cigarettes smoked per day (24.7%). However, when comparing eligible and non-eligible participants, they found no difference in levels of nicotine dependence as assessed by the Fagerström Test for Nicotine Dependence Questionnaire.30
A limitation of the clinical trials reported in the literature is that they rely on a sample of participants, and therefore cannot be extrapolated to individuals with nicotine dependence in the community. As suggested by Robinson et al,28 and in order to understand the impact of eligibility criteria in the population, an analysis of the application of eligibility criteria to a representative sample of the general population of individuals with nicotine dependence is required.
With that in mind, we assessed the effect of exclusion criteria commonly applied in clinical trials in a large, nationally representative sample, the National Epidemiological Survey on Alcohol and Related Conditions (NESARC). NESARC is a survey conducted in the United States, including a broad range of psychiatric disorders as well as measures of various medical conditions. We used a method previously described by Blanco et al in clinical trials for major depression 10 and alcohol dependence.18 We wanted to estimate the population generalisability of clinical trials for nicotine-dependent individuals. We applied common clinical trial eligibility criteria to all individuals with a current diagnosis of nicotine dependence, and then to a subsample of individuals who were motivated to quit, to examine the proportion who would have been excluded in treatment trials for nicotine dependence.
Subjects were participants in NESARC, a nationally representative face-to-face survey of 43 093 respondents aged 18 years and older (response rate 81%), conducted by the National Institute on Alcohol Abuse and Alcoholism (NIAAA) in 2001–2002.31 32 NESARC assessed the civilian non-institutionalised population residing in the United States. African-Americans and Hispanics were oversampled, as were young adults. The research protocol, including informed consent procedures, received full ethical review and approval from the US Census Bureau and the Office of Management and Budget. Data were adjusted for oversampling and household-level and person-level non-response. The weighted data were then further adjusted to represent the civilian population in the United States based on the 2000 census.
Measure of nicotine dependence
NESARC used the National Institute on Alcohol Abuse and Alcoholism's Alcohol Use Disorder and Associated Disabilities Interview Schedule DSM-IV version (AUDADIS-IV), a structured diagnostic interview made for non-clinician interviewers.33 Algorithms were designed to produce diagnoses of nicotine dependence consistent with the final DSM-IV criteria. For example, the ‘using nicotine to relieve or avoid withdrawal symptoms’ criterion was defined by the following four items: (1) the use of nicotine on waking up, (2) the use of nicotine after being in a situation in which use was forbidden, (3) the use of nicotine to decrease nicotine withdrawal symptoms, and (4) waking up in the middle of the night to use tobacco.34 Several studies have documented good to excellent retest reliability.35
Exclusion criteria commonly applied in clinical trials of treatments for nicotine dependence (see below in Clinical Trial Exclusion Criteria) were applied to individuals from the general population to determine the proportion of these individuals with current nicotine dependence according to DSM-IV criteria who would be eligible for the clinical trials. The same criteria were applied to the subset of individuals with current nicotine dependence motivated to quit, and examining potential differences in eligibility between motivated and less motivated individuals, using a pattern of analysis described elsewhere.10 18 In these studies, Blanco et al used attempts to quit a substance in the last 12 months as a proxy variable for motivation to quit in the future.10 18
The appropriate statistical weight was employed when mentioned to ensure the data were representative of the population.
Clinical trial exclusion criteria
We examined eligibility criteria from clinical trials included in a recent meta-analysis comparing the effectiveness of pharmacotherapies for smoking cessation.36 We collected all eligibility criteria from 54 randomised clinical trials,37–92 and ranked them according to their frequency. Criteria included in more than 10% of the studies are listed in table 1. The median of the number of eligibility criteria used in a study was 12 (considering not only criteria included in table 1 but also criteria present in less than 10% of the studies). We thus applied the 12 most frequently used criteria to the NESARC sample.
The percentages of individuals excluded by criteria 1, 3, 5, 6, 7, 8, 11 and 12 were estimated from data collected by the AUDADIS-IV. Information to approximate criterion 4 (use of psychotropic medications), criterion 9 (use of bupropion or nicotine replacement therapy) and criterion 10 (history of eating disorder) was not available in the NESARC.
Criterion 1 (pregnancy status) was assessed with a single question (“Were you pregnant at any time during the past year?”).
The presence of a recent cardiac event (criterion 2) was assessed by series of questions on chest pain, angina pectoris, heart attack, myocardial infarction or any other form of heart disease in the last 12 months, and whether the diagnosis was confirmed by a physician.
Criterion 3 (‘Smoking 10 cigarettes or fewer per day on average’) was applied using a 12-month time frame (as it is assessed in the NESARC).
Criterion 5 (‘Alcohol dependence’) was defined having a diagnosis of alcohol dependence within the last 12 months.
Criterion 6 (‘Being not motivated to quit smoking’) was assessed by two questions: “In your entire life, did you ever, more than once, want to stop or cut down your tobacco use?”), and “Did this happen in the last 12 months?” Participants who respond positively to both questions were classified as being motivated to quit smoking. Other participants were classified as being not motivated to quit smoking. This assessment is therefore at variance with standard questions about motivation in research trials, that usually asked whether participants want to cut down/attempt to stop in the future rather than if they have done so in the past.
Criterion 7 (‘Dependence on other drugs’) was defined having a diagnosis of dependence on an illicit substance (sedatives, tranquillisers, opiates, stimulants, hallucinogens, cannabis, cocaine (including crack cocaine), inhalants/solvents, heroin or other drugs) within the last 12 months.
Criterion 8 (‘Having current depression’) was assessed using the criteria for major depressive disorder within the last 12 months.
Criterion 11 (‘Having a current psychosis’) was assessed by two questions: “Did a doctor or other health professional ever diagnose you with schizophrenia or psychotic illness or episode?” Participants who respond positively to this were classified as having a ‘psychosis’.
Participants with a lifetime history of mania were classified as having a bipolar disorder (criterion 12). We chose to consider only bipolar type I disorder because hypomania, the hallmark of bipolar type II disorder, is a more subtle form of the disorder and therefore not likely to be screened routinely in eligibility assessments of clinical trials for nicotine-dependent individuals. For the same reason, we considered participants as having bipolar disorder if they had a history of mania even if manic episodes were induced by a substance or an illness, and did not restrict our analysis to independent bipolar disorders. As a control, we did a sensitivity analysis to examine how the results would change if (i) substance-induced and illness-induced mania were ruled out, and (ii) if bipolar type II disorder was also included in the eligibility criteria (with substance-induced and illness-induced disorders being ruled out).
We first determined the number and percentage of nicotine-dependent participants in the NESARC who would be excluded by individually applying each of the 12 most frequent eligibility criteria reported previously. Because individuals might have been excluded by more than one criterion, we also calculated the overall percentage of subjects who would have been excluded by the simultaneous application of all the measurable criteria. We conducted these analyses for all individuals with a current DSM-IV diagnosis of nicotine dependence (n=4962), and for the subsample of individuals who want to stop or cut down on tobacco use in the last 12 months (n=4121). Weighted prevalence estimates and 95% CIs were computed using SUDAAN, version 10.01 (Research Triangle Park, NC, USA). This software implements a Taylor linearisation to adjust for complex survey sampling design effects including clustering data.
The percentage of subjects excluded by at least one criterion was 65.89% among respondents who met DSM-IV criteria for nicotine dependence and 58.60% for those motivated to quit smoking in the past year (table 2).
The percentage of respondents excluded owing to the application of a single criterion ranged from 2.14% (lifetime diagnosis of psychosis) to 32.02% (smoking fewer than 10 cigarettes per day) in the overall sample of respondents with nicotine dependence, and 1.95% (lifetime diagnosis of psychosis) to 33.79% (smoking fewer than 10 cigarettes per day) among those motivated to quit smoking.
For the overall sample, smoking 10 or fewer cigarettes per day and lacking motivation to quit were the two criteria including the highest percentage of individuals. For the treatment-seeking sample, having current depression and smoking 10 or fewer cigarettes per day were the criteria comprising the greatest percentage of individuals who would not be eligible. Current alcohol dependence and a history of bipolar disorder also excluded a notable proportion of individuals in both samples (table 2).
A history of bipolar disorder (type I) was present in 10.33% of the participants with nicotine dependence (95% CI 8.16 to 10.50). As a control, ruling out illness-induced and substance-induced mania only slightly decreased to 9.26% the percentage of participants excluded because of these criteria (95% CI 8.16 to 10.50). When bipolar type II disorder was also included in this eligibility criteria (substance-induced and illness-induced disorder still ruled out), the percentage of participants excluded because of this criterion increased to 14.70% (95% CI 13.55 to 15.93). The overall exclusion rate was 65.58% when considering bipolar I disorder after ruling out illness-induced and substance-induced mania, and 66.8% when considering bipolar I and II after ruling out illness-induced and substance-induced mania, compared to an overall exclusion rate of 64.13% when considering only bipolar I disorder even if manic episodes were induced by a substance or an illness. This suggests that the criteria used to define bipolar disorder have little or no impact on the overall inclusion rate.
More than six out of 10 respondents from the full nicotine-dependent sample and more than half of the subsample of individuals motivated to quit smoking would have been excluded by one or more of the study criteria.
This study ascertains the proportion of community-dwelling adults with nicotine dependence who would have been eligible for a typical nicotine-dependence treatment study. The results of this study suggest that traditional criteria used in nicotine-dependence trials tend to exclude from participation half of individuals with nicotine dependence who are likely to seek out treatment. These results are in line with previous findings, suggesting that a majority of individuals who were screened for a nicotine-cessation trial were not eligible to participate in the trial. For example, among the 54 randomised clinical trials assessed in the present paper,37–92 the ineligibility rates varied widely, ranging from 12.9%37 to 85.31.5 6
Our study has several limitations
First of all, our exclusion criteria are somehow arbitrary. We considered eligibility criteria from 54 randomised clinical trials included in a recent meta-analysis,36 but the use of another methodology could have led to other results. An important point is that the exclusion criterion based on alcohol consumption varies widely across studies. It has been emphasised that an alcohol-related exclusion criterion appears frequently in smoking cessation pharmacotherapy trials.29 95 A recent review showed that 41.6% of trials (45 of 125 nicotine-replacement trials, 15 of 22 bupropion trials and three of three varenicline trials) involved exclusion of participants with either current or recent alcohol problems, leading to a lack of information on the effects of alcohol use disorders on smoking cessation.29 95
A second restriction is that three of the 12 exclusion criteria initially included could not be used in our study, because the relevant information was not assessed in the NESARC sample, including (1) participants currently taking psychotropic medication, (2) participants “currently taking bupropion or nicotine replacement therapy”, and (3) participants having an eating disorder. This may theoretically lead to an underestimation of the proportion of patients excluded in clinical trials. However, these criteria are rarely met in the general population. For example, the estimated percentage of smokers in Australia who used bupropion in a year was only 3.6% in 2005.96 Eating disorders have a low prevalence, affecting less than 4.5% 97 of the general population. While an investigation of the impact of these exclusion criteria on the generalisability of clinical trials is required in a future study, they are not likely to exclude a significant proportion of smokers.
A third limitation is that the NESARC sample included only individuals aged 18 years or older. Information was unavailable for adolescents, who maybe have a lower level of comorbidities, and may therefore be more likely to be eligible for clinical trials.
Some of the criteria have been implemented for safety reasons (eg, pregnancy, potential interaction with psychotropic drugs or with alcohol) while some others may contribute to stigmatise a significant proportion of the population (eg, having a history of substance abuse but with no use within the last 12 months should not be considered as a valid exclusion criterion in a clinical trial).
The exclusion of participants with alcohol dependence is particularly relevant, since nicotine dependence is a major issue in alcohol-dependent patients. For example, smokers with a lifetime history of alcohol dependence are more likely to die from smoking-related diseases rather than from alcohol-related diseases.98 Moreover, alcohol-dependent subjects with nicotine dependence have a higher prevalence of nearly all psychiatric and addictive disorders,99 making treatment for smoking cessation in this specific population an unmet need.
In summary, we found that the current criteria of eligibility applied in clinical trials involving nicotine-dependent individuals are highly restrictive and exclude a majority of participants, thus limiting the generalisability of their findings. Particularly, our findings suggested that (1) individuals smoking few cigarettes in a day or (2) having a current or past history of mood disorders (major depressive disorder or bipolar disorder) are under-represented in clinical trials. These two related groups should be the focus of further investigations.
What this paper adds
Clinical trials for treatment of nicotine dependence often exclude sizeable segments of the general population with nicotine dependence. This article quantifies the lack of generalisability by using a large representative US general population survey. It was found that the majority of nicotine-dependent subjects would have been excluded from participating in clinical trials.
Funding The National Epidemiologic Survey on Alcohol and Related Conditions was sponsored by the National Institute on Alcohol Abuse and Alcoholism and funded, in part, by the Intramural Program of DHHS-NIH-NIAAA. YLS is funded by a grant from the Société Française de Tabacologie and the Addiction Program of CAMH.
Competing interests None.
Ethics approval This study was conducted with the approval of the US Census Bureau and the Office of Management and Budget.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.