Article Text

Download PDFPDF

Testing warning messages on smokers’ cigarette packages: a standardised protocol
  1. Noel T Brewer1,2,
  2. Marissa G Hall1,
  3. Joseph G L Lee1,
  4. Kathryn Peebles1,
  5. Seth M Noar2,3,
  6. Kurt M Ribisl1,2
  1. 1Department of Health Behavior, Gillings School of Global Public Health, University of North Carolina, Chapel Hill, North Carolina, USA
  2. 2Lineberger Comprehensive Cancer Center, University of North Carolina, Chapel Hill, North Carolina, USA
  3. 3School of Journalism and Mass Communication, University of North Carolina, Chapel Hill, North Carolina, USA
  1. Correspondence to Dr Noel T Brewer, Department of Health Behavior, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, 325 Rosenau Hall CB7440, Chapel Hill, NC 27599, USA; ntb{at}


Purpose Lab experiments on cigarette warnings typically use a brief one-time exposure that is not paired with the cigarette packs smokers use every day, leaving open the question of how repeated warning exposure over several weeks may affect smokers. This proof of principle study sought to develop a new protocol for testing cigarette warnings that better reflects real-world exposure by presenting them on cigarette smokers’ own packs.

Methods We tested a cigarette pack labelling protocol with 76 US smokers ages 18 and older. We applied graphic warnings to the front and back of smokers’ cigarette packs.

Results Most smokers reported that at least 75% of the packs of cigarettes they smoked during the study had our warnings. Nearly all said they would participate in the study again. Using cigarette packs with the study warnings increased quit intentions (p<0.05).

Conclusions Our findings suggest a feasible pack labelling protocol with six steps: (1) schedule appointments at brief intervals; (2) determine typical cigarette consumption; (3) ask smokers to bring a supply of cigarette packs to study appointments; (4) apply labels to smokers’ cigarette packs; (5) provide participation incentives at the end of appointments; and (6) refer smokers to cessation services at end of the study. When used in randomised controlled trials in settings with real-world message exposure over time, this protocol may help identify the true impact of warnings and thus better inform tobacco product labelling policy.

Clinical trial number NCT02247908.

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.


For much of the world, tobacco product packaging is a key part of marketing efforts to make tobacco use appealing.1–3 In contrast, warnings on tobacco product packages accurately convey the many health risks of tobacco to discourage smoking initiation and increase cessation.4 Information on the effectiveness of these warnings can guide policymakers as they consider new warnings and develop regulations. In the more than 100 countries that already have cigarette pack labelling policies in place,5 testing the impact of potential cigarette pack warnings can guide selection of warnings with the greatest impact.

Research on cigarette pack warnings currently focuses on population-based observational studies of smoking behaviour and on laboratory experiments. Longitudinal observational studies have found increased cessation behaviour after countries introduced new health warnings on cigarette packs.6–9 These observational studies have high external validity, but they do not rule out alternative explanations for the association of the changes in labelling and behaviour change. Experiments on exposure to cigarette pack warnings in laboratory settings can offer much stronger evidence that warnings cause observed changes in beliefs and behaviour.10 ,11 While a large experimental literature has developed, these experiments use very brief exposure to warnings; assess non-behavioural short-term outcomes, such as attitudes or quit intentions, instead of smoking behaviour; and, yield results that need corroboration outside of a controlled research setting.

It will be helpful to develop a way to deliver new messages on cigarette packs that smokers use daily in order to expand the external validity of experimental research on cigarette warnings. This will ensure a meaningful warning ‘dose’ that replicates the frequency and duration of warning exposure in the real world. For example, a pack-a-day smoker sees a cigarette pack at least an estimated 7300 times per year (>20 views/day×365 days/year).

Several studies have tested warnings by providing mocked-up cigarette packs to participants, but most of these studies used a one-time, brief exposure that does not replicate message dose in the real world.10 ,12–18 At least five studies have used mocked-up packs for a longer period of time.19 ,20 A 2001 study conducted in France, Switzerland and Belgium mailed 485 adult smokers cardboard boxes with one of four antismoking messages selected by the smoker at recruitment. Study materials instructed participants to place their cigarette packs in the boxes for 4 weeks, but few did so.20 A 2010 study in Scotland provided 140 adult smokers with 14 plain cigarette packs that had warnings but no brand information or logos, asking participants to transfer their own cigarettes into the supplied packs for 2 weeks. Only 34% of participants completed the study as intended.19 A recent US study provided smokers with free cigarettes and labelled the packs with health warnings for 4 weeks. During the study, participants collected their cigarette butts and brought them to study appointments. This study used an innovative method for labelling cigarette packs, but the results may have been influenced by receipt of free cigarettes and saving the cigarette butts.21 Two additional recent studies in the US had smokers apply warning labels to their own cigarette packs, which may be an intervention in its own right, and both studies had short follow-up periods (3–7 days).22 ,23 While these studies’ methods are promising because they more closely replicate real-world message dose, we need strategies with high protocol adherence and longer follow-up periods. Tobacco control research would benefit from a way to test health warnings on cigarette packs that allows for random assignment to warning conditions, adds realism and maintains high protocol adherence.

To that end, we tested a new cigarette pack labelling and carrying protocol with adult smokers in two proof-of-principle pilot studies. We aimed to assess whether the pack labelling and carrying protocol (1) is feasible and has high adherence for assessing smokers’ reactions to cigarette pack warnings in a real-world setting and (2) is sensitive to changes in psychosocial outcomes.

Materials and methods


Pilot study 1

From February to March 2013, we recruited 30 smokers ages 18 or older who we observed smoking cigarettes in public places in North Carolina, USA, or who received referrals from study participants. We defined current smoking as having smoked at least 100 cigarettes during one’s lifetime and now smoking every day or some days, and we excluded pregnant women, people who smoke only roll-your-own cigarettes and cigarillo-only smokers.

Pilot study 2

In July and August 2014, we recruited an additional 48 smokers using the same eligibility criteria as Pilot Study 1. We used a variety of recruitment methods, including newspaper ads, flyers and email lists.


Pilot study 1

A research assistant explained the study to participants and obtained written informed consent. We provided participants with an $80 financial study incentive in cash and accompanied them to a nearby store, where we asked them to purchase the amount of cigarette packs they would normally smoke over 2 weeks as determined in the initial screening. We randomly assigned participants to receive one of nine graphic warnings. In 2011, the US Food and Drug Administration (FDA) proposed these nine warnings for implementation, but the warnings were later challenged in court and are not currently in use.24 At the time of the study, US cigarette packs also included one of four text-only warnings on one side of the pack; these warnings have been in circulation since 1985. Then, participants completed a baseline survey on a tablet computer while staff removed the package cellophane and applied the same graphic warning labels to the top half of the front and rear panels of participants’ cigarette packs, in accordance with the proposed FDA requirements (figure 1). As applying the labels on the top half of the front panel sealed the flip-top box shut, staff cut through the label to allow the box to again open freely. Participants returned to the study offices to complete a follow-up computer survey 2 weeks later and received an additional $50 cash incentive. After completing the follow-up visit survey, each participant received information about a local smoking cessation programme.

Figure 1

Example of graphic warning label on pack.

Pilot study 2

Procedures for Pilot Study 2 were similar to Pilot Study 1. After obtaining written informed consent, we randomly assigned participants to receive one of five graphic warnings, four of which were also used in Pilot Study 1. Participants visited our study offices at baseline and then weekly for 4 weeks, completing a survey on the computer at each visit. Smokers brought 8 days’ worth of cigarettes to the first four appointments. While participants were taking the survey, we labelled their packs using the same procedures as in Pilot Study 1. Participants received a cash incentive at the end of each visit in an envelope marked ‘payment for survey completion.’ Cash incentives totalled $185. At the final appointment, each participant received information about a local smoking cessation programme. Two of the 48 participants withdrew from the study. The University of North Carolina Institutional Review Board approved the procedures for both pilot studies.


Prior to the studies, we refined the survey instruments by conducting cognitive interviews with 17 adult smokers.25 Baseline and follow-up surveys applied measures adapted from prior studies, including knowledge of smoking health risks (3 items),26 perceived likelihood of harm from smoking (2 items),27 worry about the harms of smoking (1 item),28 subjective norms (3 items),29 positive smoker prototypes (7 items),30–32 negative smoker prototypes (5 items),30–32 self-efficacy to quit smoking (3 items)29 and quit intentions (3 items).33 Cronbach's α for multiitem scales was 0.70 or higher in both studies, except for self-efficacy in Pilot Study 1 (α=0.69). We also measured whether participants attempted to quit smoking (defined as not smoking for at least 24 h because they were trying to quit smoking) or successfully stopped smoking during the study (defined as not smoking for at least 7 days).

The follow-up surveys assessed noticing the warning, whether people talked to others about the warning, avoidance of the warning, emotional reactions to the warning,34 participant satisfaction with study procedures, the number of cigarettes smoked from labelled and unlabelled packs and unprompted recall of the warning image. Two raters independently coded whether each participant correctly recalled the image in their assigned warning, with perfect agreement.

Finally, in Pilot Study 1, several open-ended questions assessed attitudes toward the study protocol. Two raters independently coded whether open-ended responses indicated that smokers thought the study was giving them free cigarettes. A coauthor resolved all coding discrepancies. The survey instruments are available online:

Data analysis

We calculated descriptive statistics to summarise demographic information and process measures. We assessed changes between baseline and follow-up using paired t tests or McNemar χ2 tests. For continuous variables with non-normal distribution (by Q–Q plots and Kolmogorov-Smirnov tests of normality), we used Wilcoxon signed-rank tests, a non-parametric equivalent of the paired t test. Analyses used SPSS Statistics V.20 and Stata/IC V.13.1. The two Pilot Study 2 participants who withdrew from the study were excluded from analyses. We set test critical α value to 0.05 and used two-tailed tests.


Smokers’ mean age was 30 years in Pilot Study 1 and 43 years in Pilot Study 2 (table 1). Fewer than half of participants were white and about one-third were African-American. Most smokers (69% in Pilot Study 1 and 59% in Pilot Study 2) were low-income, defined as being at or below 200% of the US federal poverty level. At baseline, participants in Pilot Study 1 smoked an average of 12 cigarettes per day (SD=9) and participants in Pilot Study 2 smoked an average of 11 cigarettes per day (SD=8).

Table 1

Participant demographics

Exposure to labels

Study participants reported high rates of adherence to smoking cigarettes only from labelled packs. Around 90% of smokers reported that at least three quarters of the cigarettes smoked during the study were from labelled packs (figure 2). Ninety per cent of smokers in Pilot Study 1 and 93% of smokers in Pilot Study 2 correctly recalled the image on their assigned labels. At follow-up, most (93% in Pilot Study 1 and 89% in Pilot Study 2) smokers reported noticing the warning sometimes, often, or all of the time (table 2). About half of smokers tried to avoid thinking about the labels sometimes, often, or all the time. Similarly, 43% of smokers in Pilot Study 1 and 46% of smokers in Pilot Study 2 tried to avoid looking at the labels sometimes, often, or all of the time.

Table 2

Outcomes at follow-up

Figure 2

Process evaluation measures for cigarette pack labelling proof of principle studies.

Outcomes at follow-up

About half (50% in Pilot Study 1 and 41% in Pilot Study 2) of participants reported that the warnings caused them to think about the harmful effects of smoking somewhat, quite a bit, or very much. Almost all participants (97% in Pilot Study 1 and 93% in Pilot Study 2) reported talking to someone about the warnings during the study.

Changes in outcomes

Endorsement of positive smoker prototypes decreased between baseline and follow-up in both studies (p<0.05, table 3). Likewise, quit intentions increased in both studies (p<0.05). Worry about the harms of smoking increased in Pilot Study 1 (p=0.01) but not in Pilot Study 2. Knowledge and perceived likelihood of harm increased in Pilot Study 2 (p<0.05), but not in Pilot Study 1. The 2-week outcomes for Pilot Study 2 showed the same pattern as the 4-week outcomes (data not shown), except for cigarettes smoked per day which dropped to a mean of 9.10 (SD=6.81) from 11.11 (SD=7.44) at baseline (p=0.02).

Table 3

Changes in outcomes

The mean number of cigarettes smoked per day did not change between baseline and follow-up in either study. Ten per cent of Pilot Study 1 participants and 20% of Pilot Study 2 participants stopped smoking for at least 24 h during the study. No participants successfully quit smoking during Pilot Study 1. During Pilot Study 2, two participants had quit smoking by week 2 but they resumed by week 4, and one participant quit between weeks 2 and 4.

Process measures

Retention was very high in both studies (100% in Pilot Study 1 and 96% in Pilot Study 2). Nearly all participants said that they would probably or definitely participate in the study again, and 100% said they would probably or definitely recommend the study to a friend (figure 2). Most smokers (90% in Pilot Study 1 and 78% in Pilot Study 2) found participating in this study to be easy or very easy.

About a third (11/30) of Pilot Study 1 smokers said in exit interviews that they thought the study was buying them cigarettes as our staff had prepaid them and escorted them to the store so they could buy cigarettes at the baseline interview. To address these concerns, we asked Pilot Study 2 participants to bring an eight-day supply of cigarettes with them to the study visits every week for 4 weeks. Most participants did not find this task to be burdensome: 15% of participants reported that it was difficult to bring in 8 days’ worth of cigarettes, 30% said it was neither difficult nor easy, and 54% said it was easy or very easy.


Our proof of principle studies of placing warnings on real-world cigarette packages for an extended period of time suggests that our proposed protocol is feasible. The protocol had low attrition, high adherence and allowed assessment of impact over time. Most smokers complied with the study protocol and reported high levels of satisfaction with the procedures. Thus, this new pack carrying protocol offers the potential for both experimental control and greater real-world dose than exposures in brief lab studies.

The original protocol (Pilot Study 1) provided a financial incentive immediately before the smoker purchased cigarettes because we were concerned that low-income smokers could not afford the up-front costs of buying 2 weeks’ worth of cigarettes at one time. However, we believe that our revised protocol (Pilot Study 2) may pose fewer risks to smokers. In the USA, lower-income populations smoke at higher rates than their higher-income counterparts,35 and, accordingly, research shows that most smokers do not purchase cigarettes in bulk, but rather by the pack.36 Losing large numbers of smokers who cannot afford to purchase cigarettes in bulk would, in turn, limit the generalisability of cigarette pack labelling studies using such a protocol. However, many Pilot Study 1 participants believed that the study gave them free cigarettes, which could potentially increase smoking. Studies suggest lower prices encourage cigarette consumption.37 Fortunately, cigarette consumption did not increase, while quit intentions did increase.

Another potential concern is whether having more cigarettes on hand might make participants smoke more, especially if they are not accustomed to purchasing packs in bulk. The empirical impact of stockpiling cigarettes on smoking behaviour is unknown. Smokers can save money by purchasing cigarettes in bulk (eg, cartons) rather than by the pack, but many smokers elect to ration their purchase quantities and impose additional transaction costs to limit their smoking.38 ,39 We suggest keeping the length of time between study appointments at 1 week, which will limit the number of cigarette packs that participants will have on-hand at one time, thus reducing the potential risk that participants will smoke more. Future research should try to assess unintended consequences of stockpiling packs.

Having participants bring their own cigarettes to study appointments decouples cigarette purchasing from receiving payment. Moreover, it appears that low-income smokers in our study were not deterred by having to purchase cigarettes in advance as part of the protocol as more than half of Pilot Study 2 participants were at or below 200% of the US federal poverty level. It remains possible that other low income smokers were unable to participate in the study.

Our findings point to a new pack labelling protocol with six steps: (1) schedule appointments 1 week apart; (2) determine daily cigarette consumption through smokers’ self-report; (3) ask smokers to purchase and bring to the study appointments the amount of cigarette packs they would normally smoke between appointments, with an extra supply to buffer against cancellations and allow for greater than expected use; (4) during study appointments, apply labels to smokers’ cigarette packs in-person by removing the top of the cellophane wrapping from the cigarette packs and applying self-adhesive labels with warnings directly to the packs, to prevent smokers from inadvertently removing the label with the wrapping; (5) provide study participation incentives at the end of the appointments and clearly communicate that the payment is for survey completion to reduce the possibility that participants will believe that the payments equate to receiving free cigarettes; and (6) refer smokers to cessation services at study completion or directly help smokers who want to quit. Table 4 gives the rationale for each protocol step. We implemented the six steps of this protocol successfully in Pilot Study 2.

Table 4

Recommended pack labelling protocol for studies that place warnings on smokers’ cigarette packs

Strengths of the study include establishing the feasibility of a new pack labelling protocol in a field study with current smokers, followed over time. As we studied a convenience sample recruited in a single location, the generalisability of our findings remains to be established. By labelling cigarette packs that smokers were currently using, we tested novel warnings alongside the standard US text-only warnings that we did not obscure. Depending on the brand, the warnings may have covered some of the branding or colour of the cigarette packs. In these cases, we may have overestimated the impact of the warning, as obscuring the branding may make cigarettes less appealing to smokers.40 ,41 We relied on self-report to measure protocol adherence, but the extent to which self-reported adherence reflects actual adherence is unknown. We tested only methods for labelling cigarette packs, and we are unable to determine the utility of this protocol for other tobacco products. Finally, the small sample size limited statistical power to detect changes in psychosocial and behavioural outcomes.

Given the compelling need for better strategies to test the efficacy of warnings on tobacco products,42 ,43 this feasible protocol holds promise for improving our ability to determine the utility of warnings on cigarette packages. Our recommended protocol developed through this proof of principle study mimics real-world exposure to tobacco product packaging while allowing for experimental control needed in randomised trials. The protocol could be used to test the impact of other cigarette pack warning characteristics such as size and format, as well as the impact of modified-risk tobacco product warnings.44 ,45 Moreover, researchers can adapt the protocol to test graphic or text warnings for non-cigarette tobacco products. Future research should examine the utility of our pack labelling protocol in randomised controlled trials.

Rigorous evidence is required to identify the best ways to communicate the risks of tobacco use, de-normalise tobacco products and reduce the extent to which tobacco product packaging glamorises and promotes the deadly product inside. Our proof of principle studies provide a feasible strategy for testing warning messages that ensures high adherence, and can generate stronger evidence of the real-world impact of warnings on smoking-related outcomes.

What this paper adds

  • Experimental studies on graphic cigarette pack warnings typically use a brief one-time exposure, often in a laboratory setting.

  • Naturalistic pack labelling studies hold promise for replicating the frequency and duration of warning exposure in the real world while maintaining experimental control.

  • We developed and tested a six-step protocol for labelling smokers’ actual cigarette packs. The protocol was feasible to implement, was acceptable to smokers, and had high retention over four weeks. Using the protocol can yield stronger evidence on the impact of warnings to better inform the development of cigarette pack warning policies around the world.


The authors thank the research participants for taking part in this study. The authors also thank and appreciate Katie Byerly for her help with participant recruiting and data collection.



  • Contributors KMR and NTB originated the study. NTB and MGH drafted the manuscript. JGLL and MGH implemented the protocols and conducted statistical analyses. KMR, MGH, SMN and NTB developed the measures and protocols. KP assisted with analysis and writing and edited multiple drafts of the paper. All authors provided critical feedback on drafts of the manuscript and approved the final manuscript.

  • Funding This work was supported by 4CNC: Moving Evidence into Action grant number U48/DP001944, and by the National Institutes of Health/National Cancer Institute grant numbers U01 CA154281 and P30 CA016086-38S2.

  • Competing interests None.

  • Ethics approval University of North Carolina Institutional Review Board.

  • Provenance and peer review Not commissioned; externally peer reviewed.