Background Decisions about which antismoking advertisements should be aired are often guided by audience ratings of perceived effectiveness (PE). Given that the usefulness of PE measures depends on their ability to predict the likelihood that a message will have a positive impact on outcomes such as behaviour change, in the current study we used pre-exposure, postexposure and follow-up measures to test the association between PE and subsequent changes in quitting intentions and smoking behaviours.
Methods Daily smokers (N=231; 18 years and older) completed baseline measures of quitting intentions before watching an antismoking advertisement. Immediately following exposure, intentions were measured again and PE was measured using six items that factored into two scales: ad-directed PE (ADPE) and personalised PE (PPE). A follow-up telephone survey conducted within 3 weeks of exposure measured behaviour change (reduced cigarette consumption or quit attempts).
Results From pre-exposure to postexposure, 18% of smokers showed a positive change in their intentions. Controlling for baseline intentions, PPE independently predicted intention change (OR=2.57, p=0.004). At follow-up, 26% of smokers reported that they had changed their behaviour. PPE scores also predicted the likelihood of behaviour change (OR=1.93, p=0.009).
Conclusions Audience ratings of PPE, but not ADPE, were found to predict subsequent intention and behaviour change. These findings increase confidence in the use of PE measures to pretest and evaluate antismoking television advertisements, particularly when these measures tap the extent to which a smoker has been personally affected by the message.
- Social marketing
Statistics from Altmetric.com
Variations in the effectiveness of antismoking mass media campaigns are due in large part to differences in the reach, intensity and duration of campaign exposure.1 However, campaign success may also be related to the content and executional characteristics of the messages used.1 ,2 For this reason, there is a need for methods that can efficiently and effectively determine which messages are most likely to have the desired effects.3 Audience ratings of perceived effectiveness (PE)—which typically tap the extent to which a message has been favourably received and evaluated—may provide one such method.
Confidence in the utility of PE measures has increased recently, as the body of evidence linking PE with measures of actual effectiveness such as attitude4 ,5 and intention change3 ,4 ,6 has grown. In one recent study, the PE ratings given to the four antismoking advertisements to which an individual was exposed (from a range of 100 messages total) were found to predict quitting-related intentions.3 In another recent study, Davis et al6 found that PE ratings predicted several campaign outcomes measured 2 weeks after initial exposure to the message, including greater intentions to quit. However, Davis et al6 did not find PE ratings to be associated with the likelihood that smokers had actually attempted to stop smoking. While evidence of a predictive relationship between PE and changes in behavioural intentions is undoubtedly an important finding, and the link between intentions and behaviours is well established,7 it remains the case that demonstrating an association between PE ratings and subsequent behaviour change would greatly enhance confidence in the use of these measures. As such, in the current study we aimed to provide a further test of the association between the PE ratings given to antismoking television advertisements and the likelihood of subsequent changes in quitting intentions and smoking behaviours.
In explaining why they did not find a significant association between PE and behaviour change, Davis et al6 emphasised that participants received only a low dose of media exposure, and that for many smokers, a 2-week follow-up may not have allowed sufficient time for implementation of their decision to quit. In addition, baseline rates of quitting over a 2-week period are typically low (eg, 5%8), so the Davis et al study may have lacked power to detect differences in the rates of quitting activity. Therefore, given that many smokers relapse within the first week of quitting,9 and some smokers try to reduce their cigarette consumption before making an actual attempt to quit,10 in the current study we more broadly defined behaviour change to include both successful and unsuccessful quit attempts, as well as reductions in cigarette consumption, thereby capturing the activity most likely to occur soon after exposure to an effective antismoking message.
Some studies have found that smokers who are intending to quit tend to give higher PE ratings to antismoking advertisements than those who are uninterested in quitting,11 ,12 a finding which could be taken to indicate that antismoking messages will be most effective for this group.11 On the other hand, it is also possible that PE may not be all that important in determining whether those who are intending to quit are influenced by an advertisement, given that the arguments made in the message are likely to be consistent with their current attitudes. In contrast, those who are not yet planning to quit are likely to require convincing of the merits of the antismoking argument and may therefore only be affected by the advertisements that they perceive to be credible, relevant and motivating. To our knowledge, no previous studies have examined whether the association between PE and campaign outcomes is the same for smokers who are and are not intending to quit (note however that the two recent studies did control for baseline interest in quitting when testing the association between PE and postexposure intentions3 ,6). Therefore, given the practical implications of any such differences for the use of PE measures to pretest antismoking advertisements, in the current study we also examined whether the association between PE ratings and subsequent intention and behaviour change was moderated by baseline intentions to quit.
Sample and procedure
Data were collected as part of a larger study in which current smokers and non-smokers participated in friendship pairs. In this study, a two (advertisement: Pam Laffin or Rick Stoddard—46 Years Old) by two (conversation: no conversation or conversation) experimental design assessed the influence of interpersonal conversations on campaign effectiveness. For the purposes of the current study, however, participants from the four experimental conditions were combined, and the sample was restricted to the 231 current smokers. Additional information about the design, sample and procedures of the experimental study can be obtained from the authors.
One member of each friendship pair was recruited through a market research company's existing database of research participants, and the second participant was a friend of this recruit. All participants were at least 18 years old and resided in the state of Victoria in Australia. One experimental session was conducted per friendship pair; however, each participant was exposed to the advertisement and completed all measures in isolation from, but at the same time as, their partner. Participants were exposed to one of two antismoking advertisements, both of which had not aired in Victoria before (the Pam Laffin and Rick Stoddard—46 Years Old advertisements can both be found online at http://apps.nccd.cdc.gov/MCRC). Prior to exposure, participants completed measures of quitting intentions and individual characteristics (Time 1 measures); after watching the advertisement, they completed a postexposure questionnaire (Time 2 measures). All participants were then independently followed up within 3 weeks by telephone (Time 3 measures). Participants were reimbursed $60, and this study was approved by the University of Melbourne's Human Research Ethics Committee.
At Time 1, participants reported their sex, age, highest level of education completed and daily cigarette consumption. Socioeconomic status (SES) was measured using the Australian Bureau of Statistics’ Index of Socio-Economic Disadvantage, using 2006 census data of the postcode area in which respondents resided.13 Sample characteristics are presented in table 1. Preliminary logistic regression analyses determined whether each of these individual characteristic variables was included as a covariate in the models predicting intention and behaviour change.
At Time 2, six items drawn from previous studies11 ,14–16 measured PE, and factor analysis was used to determine if these items represented one or more underlying factors. On a 5-point scale (1—strongly disagree to 5—strongly agree), participants indicated whether the advertisement: (a) made me stop and think; (b) made a strong argument for quitting; (c) taught me something new; (d) was relevant to me; (e) made me feel concerned about my smoking and (f) made me feel motivated to try to quit smoking. Inspection of the polychoric correlation matrix17 and the Kaiser-Meyer-Olkin value (0.82) indicated that the data were suitable for factor analysis,18 and a principal factors extraction with promax rotation revealed a two-factor structure. Three items loaded on the first factor: stop and think (0.58), strong argument for quitting (0.76) and taught me something new (0.61). This factor was labelled ad-directed PE (ADPE), and the three items were averaged together (α=0.74). Three items loaded on a second factor: relevant to me (0.49), concerned about my smoking (0.86) and motivated to try to quit (0.67). This factor was defined as personalised PE (PPE) (α=0.75). The ADPE and PPE scales were moderately correlated (r=0.66).
Intentions to quit
At Time 1 and Time 2, participants were grouped into one of six perspectives on change based on the transtheoretical model's stages-of-change.8 ,19 Precontemplators (not considering quitting within the next 6 months) were divided based on whether they (1) were happy to continue smoking forever or (2) planned to quit sometime. Contemplators (considering quitting within the next 6 months) were divided according to whether (3) quitting was just a possibility or (4) they were actually thinking about it, and preparers (planning to quit within the next month) were further divided based on whether they (5) had not set a date to quit within the next 2 weeks or (6) had set a date to quit within the next 2 weeks.8 In analyses examining interactions between baseline intentions and PE ratings, those who had plans to quit (PTQ) at Time 1 were captured using a binary variable (no PTQ—perspectives 1, 2, 3 and 4; PTQ—perspectives 5 and 6).
Participants were categorised as having changed their intentions to quit if they showed any forward movement on the perspectives on change measures between Time 1 and Time 2 (eg, from perspective 1, to perspectives 2, 3, 4, 5 or 6; from perspective 5 to 6, etc), or if they remained in perspective number 6. At the beginning of the follow-up survey (Time 3), participants were asked whether they had changed, or had thought about changing, their smoking behaviour in the past week. Participants were dichotomised into those who had not changed their behaviour (no change; thought about quitting, but did not make an attempt; or decided to quit, but did not make an attempt) and those who had made an actual change (tried to cut down the number of cigarettes smoked; attempted to quit but relapsed to smoking; or attempted to quit and are still quit).
Models predicting intention change used the full analytic sample (N=231; note that 1 current smoker (from an original sample of 232) was excluded from the analytic sample due to outlier values on several variables). Models predicting behaviour change were restricted to the 208 current smokers (90%) who completed the follow-up survey (days to follow-up: M=8.5; SD=3.5; range=5–21). t Tests and χ2 compared the characteristics of those who were and were not followed up, and these analyses indicated that the two samples differed significantly in the distribution of men and women (χ2 (1, N=231)=4.02, p=0.045), and that the difference in the proportion of participants who had watched the Pam Laffin and Rick Stoddard—46 Years Old advertisements was approaching statistical significance (χ2 (1, N=231)=3.67, p=0.055). However, because 90% of the initial sample was retained at follow-up, these different distributions did not produce noticeable differences in the characteristics of the initial and followed-up samples (table 1). Across all Time 1 measures, only two missing values for the education variable and one missing value for SES were identified, and these were replaced using the sample median. There were no missing data for any Time 2 or Time 3 variables.
A preliminary set of logistic regression models tested the association between each potential covariate and the two outcome variables (for categorical covariates, a χ2 test assessed the overall effect of the variable), and only those that were associated with the outcome at p<0.25 were included as covariates in subsequent models.20 In the models predicting intention change, daily cigarette consumption and conversation condition were therefore included as covariates (age OR=0.98, 95% CI (0.72 to 1.33), p=0.893; daily cigarette consumption OR=0.68, 95% CI (0.44 to 1.03), p=0.067; sex (female) OR=1.27, 95% CI (0.63 to 2.55), p=0.504; conversation condition (yes) OR=0.64, 95% CI (0.31 to 1.31), p=0.218; education χ2=0.34, p=0.843; SES χ2=2.35, p=0.308). In the models predicting behaviour change, sex and age were included as covariates (age OR=1.44, 95% CI (1.08 to 1.94), p=0.014; daily cigarette consumption OR=0.87, 95% CI (0.61 to 1.23), p=0.424; sex (female) OR=1.83, 95% CI (0.94 to 3.60), p=0.078; conversation condition (yes) OR=0.92, 95% CI (0.49 to 1.74), p=0.792; education χ2=0.21, p=0.902; SES χ2=2.64, p=0.268). The association between PE and intention change was then examined using three logistic regression models in which ADPE and PTQ were the predictor variables (unadjusted models, multivariable model, and multivariable model with two-way interaction), and three additional models in which PPE and PTQ were predictors. In addition, a final multivariable model included both ADPE and PPE (and PTQ) as predictors, and the two-way interaction between ADPE and PPE was also tested. In the same way, two sets of logistic regression models, and a final multivariable model (plus a test of the two-way interaction between ADPE and PPE), tested the association between ADPE and PPE (and PTQ) and behaviour change.
Models predicting intention change and behaviour change all controlled for the advertisement to which participants were exposed, and models predicting behaviour change also controlled for the number of days between the experimental session and the follow-up survey. In addition, during the data-collection period (April–June 2010), the Australian Government implemented a 25% increase in tobacco excise, which immediately increased the price of a pack of 30 cigarettes by around $2.16AUD. They also announced plans to introduce plain packaging for cigarettes in Australia. Therefore, all models also controlled for the number of days since the tax increase and announcement. All models adjusted for dependency within friendship pairs, and robust SEs were used. Continuous predictor variables were standardised. Analyses were conducted using Stata V.12.0.
At Time 1, 88.7% of smokers were not planning to quit. Overall, the ADPE scale had a mean of 3.6 (SD=1.0), and the PPE scale had a mean of 3.4 (SD=1.0). Preliminary linear regression models indicated that those who had no PTQ at Time 1 (M=3.6; SD=1.0) and those who were planning to quit (M=3.8; SD =0.9) gave similar ADPE scores, β=0.21, 95% CI (−0.16 to 0.57), p=0.267. However, compared with those who were not planning to quit (M=3.4; SD=1.0), those who were (M=3.9; SD=0.8) gave significantly higher PPE scores, β=0.50, 95% CI (0.14 to 0.86), p=0.007. From Time 1 to Time 2, 18.2% of participants changed their intentions to quit, and at follow-up, just over a quarter (26.4%) reported that they had changed their smoking behaviour.
Models predicting changes in quitting intentions and smoking behaviour
Table 2 presents the results from analyses predicting intention change. ADPE and PPE were both significantly associated with intention change in the unadjusted and multivariable models. The two-way interaction between PTQ and ADPE was not significant, and the interaction with PPE was only approaching statistical significance. In the final multivariable model, only PPE was significantly associated with intention change. A two-way interaction between ADPE and PPE was also tested (not shown in table 2), but this interaction term was not significant, OR=1.40, 95% CI (0.91 to 2.16), p=0.126.
Table 2 also presents the findings from analyses predicting changes in smoking behaviour. ADPE was not associated with the likelihood of behaviour change, and the two-way interaction with PTQ was also not significant. However, PPE was significantly associated with the likelihood of behaviour change (the interaction between PTQ and PPE was not significant; table 2). Consistently, PPE was the only significant predictor in the final multivariable model predicting behaviour change, and the two-way interaction between ADPE and PPE was not statistically significant (not shown in table 2; OR=1.04, 95% CI (0.73 to 1.50), p=0.819). One additional model examined the association between intention change and behaviour change. After adjusting for PTQ, smokers who changed their intentions between Time 1 and Time 2 were significantly more likely to make a subsequent change to their behaviour (40.0%, compared with 23.7% for those who did not change their intentions), OR=2.41, 95% CI (1.10 to 5.31), p=0.029.
Audience ratings of PE have been used in studies to compare the effectiveness of different types of antismoking messages,6 ,11 ,12 ,16 ,21–23 and have been recommended as a hurdle requirement that should be satisfied before messages are broadcast to a population.24 Building on recent findings of an association between PE and changes in message-relevant attitudes4 ,5 and intentions,3 ,6 in the current study we found that antismoking advertisements were more likely to produce a change in intention when smokers perceived the advertisement to be effective. We also found, for the first time, that perceptions of the personalised effectiveness of a message predicted the likelihood of individuals making positive changes in their smoking behaviour. As such, these findings further endorse the use of PE measures to pretest and evaluate antismoking advertisements.
Just under one-fifth (18%) of smokers showed a positive change in their intentions to quit (or maintenance of a preparation stage) from before to after seeing the advertisement. ADPE was a significant predictor, with the odds of change in intention increasing by around 70% for every 1-unit increase on the standardised ADPE scale. However, PPE was an even stronger predictor, as the odds of change in intention increased by more than 100% for every 1-unit increase on the standardised PPE scale, and in the multivariable model in which ADPE and PPE were both included as predictors, only PPE remained significantly associated with change in intention. Of those who were followed up, 26% reported that they had changed their smoking behaviour—meaning that in the time since exposure to the antismoking advertisement, they had cut down the number of cigarettes smoked or had made an attempt to quit. While ADPE ratings were not associated with behaviour change, smokers were at least 50% more likely to have changed their behaviour with every 1-unit increase on the standardised PPE scale, and it is notable that this effect remained significant even after adjusting for the smoker's baseline interest in quitting. Demonstrating that smokers who perceived that they had been personally affected by the advertisement were then more likely to change their intentions and smoking behaviour, these findings have important implications for those involved in the development and evaluation of antismoking campaigns. Assessing changes in smoking behaviour requires that studies include a follow-up component, a requirement that may be beyond the resources and time available to many of the organisations that develop and evaluate antismoking campaigns. Therefore, it is useful for these groups to know that audience perceptions of message effectiveness are likely to indicate how successful a message will be at producing actual changes in intentions and behaviours. That is not to say that measures of intention and behaviour change should not remain the gold standard for assessing campaign effectiveness. However, when messages are being developed or compared against one another in formative research, or when sufficient resources or time are unavailable, the current findings indicate that audience perceptions of the personalised effectiveness of the message can be used with some confidence.
Somewhat consistent with previous studies,11 ,12 we found that smokers who were planning to quit at baseline gave higher ratings on the PPE scale than did non-intenders. While this finding could indicate that antismoking messages may be most effective for those who are already planning to quit,11 the results from the interaction analyses did not provide any evidence that the associations between PE and changes in intentions and behaviours were moderated by baseline interest in quitting. As such, the current findings indicate that, while baseline interest in quitting should be used as a control variable when assessing the association between PE ratings and quitting outcomes, it may not be necessary to consider the baseline quitting intentions of the sample when using PE ratings to decide which messages should be aired in a given population.
In this study, we measured PE using six different items, and the factor analysis indicated that these represented two underlying dimensions of audience perceptions of message effectiveness. Comprising items measuring whether the advertisement taught the individual something new, made a strong argument for quitting, and made them stop and think, the ADPE scale captured perceptions of the general effectiveness of the message, and these ratings were associated with intention change but not with behaviour change. In contrast, the PPE scale measured whether the individual believed that the advertisement was relevant to them and made them feel concerned about their smoking and motivated to try to quit. These ratings predicted the likelihood of both intention and behaviour change. As such, these findings suggest that it may be necessary for individuals to perceive that they have been personally affected by a message, if that message is to have an impact on their smoking behaviour. These findings indicate that future formative studies may wish to prioritise the collection of PPE measures over ADPE measures, if they are facing limited resources and/or questionnaire space. Broadly consistent with the suggestion from Dillard and Ye25 that PE measures are most accurate when the individual thinks only about themselves when evaluating the message, these findings may also help guide future efforts to refine the conceptualisation of PE and to resolve the inconsistency in these measures within the literature.
One of the limitations of this study is the use of an experimental design, which exposed participants to antismoking advertisements in an artificial setting. On the other hand, the inclusion of a follow-up component that retained the majority of participants (90%) is a particular strength. Although all follow-up surveys were conducted within 3 weeks of the experimental session, some participants were recontacted as early as 5 days postexposure and so had less time to change their behaviour than others. In addition, the self-report behaviour change measure referred to behaviour within the past week only. Therefore, it is possible that participants were not bringing to mind all recent behaviour change attempts when responding to this question. However, any effects of variation in the time to follow-up were minimised by adjusting for the number of days between exposure and the follow-up survey (which was positively and significantly associated with behaviour change in all models). An additional limitation is that individuals who remained in perspective number six from Time 1 to Time 2 were categorised as having changed their intentions to quit, even though they may not have been persuaded by the advertisement. However, this was only the case for four participants. Also, the absence of a control group means that we cannot attribute all of the observed behaviour change to the advertisement exposure. However, the associations between individual responses to the advertisement and the likelihood of behaviour change, and the inclusion of baseline interest in quitting in the models, provides some confidence that message exposure contributed to the behaviour change that we observed. This reasonably small sample of participants was recruited from a database of individuals who had agreed to be contacted about future research studies (or through snowballing from these participants). Future research should therefore aim to replicate the findings with a larger, representative sample of smokers. It is also important to note that the two advertisements evaluated in this study used a testimonial format to impart information about the negative consequences of smoking with the intention of motivating smokers to quit. Therefore, the generalisability of these findings to other styles of antismoking messages may be limited, particularly given that why-to-quit messages have previously been found to receive higher PE ratings than other types of messages.11
Antismoking mass media campaigns are increasingly being recommended as a key component of comprehensive tobacco control programmes,26–28 and evidence of their effectiveness at the population level continues to grow.1 With this increased use comes the need for methods that can guide decisions about which antismoking messages should be aired. By demonstrating that PE measures—and in particular, measures that tap the extent to which a smoker has been personally affected by a message—may predict subsequent changes in quitting intentions and smoking behaviours, the current study provides further evidence supporting the use of PE ratings to assess the potential effectiveness of antismoking television advertisements.
What this paper adds
Perceived effectiveness measures may provide an efficient method by which to pretest and evaluate antismoking messages, but only to the extent that these measures are associated with indicators of actual effectiveness, such as intention and behaviour change. Consistent with the findings from two recent studies, in this study we found that changes in intentions to quit following exposure to an anti-smoking advertisement were predicted by audience ratings of personalised perceived effectiveness. We also found, for the first time, that changes in smoking behaviour were also predicted by personalised perceived effectiveness ratings. By demonstrating that smokers who perceived that they had been personally affected by the advertisement were then more likely to change their quitting intentions and smoking behaviour, these findings provide important evidence supporting the use of perceived effectiveness ratings to guide decisions about which antismoking messages should be aired.
Correction notice This article has been corrected since it was published Online First. The *** p value indicators in the Table 2 body have been changed to *, and the * p value indicators have been changed to ***.
Contributors EB designed the study, analysed and interpreted the data and led the writing of all drafts. SJD, MAW and YK assisted in the design of the study, oversaw data analysis and contributed to the interpretation of the data and all drafts. All authors gave final approval of the manuscript.
Funding This research was supported by an Australian Research Council Linkage Grant LP0882363 awarded to YK at the University of Melbourne, and SJD and MAW at the Cancer Council Victoria. The Australian Research Council had no involvement in the study design; in the collection, analysis and interpretation of data; in the writing of the report; or in the decision to submit the paper for publication.
Competing interests All authors have completed the Unified Competing Interest form at http://www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: EB, SJD, MAW and YK had support, in the form of grant funding, from the Australian Research Council for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous 3 years; and no other relationships or activities that could appear to have influenced the submitted work. This study was conducted while EB was a PhD candidate at the University of Melbourne and the Cancer Council Victoria.
Ethics approval University of Melbourne's Human Research Ethics Committee.
Provenance and peer review Not commissioned; internally peer reviewed.
Data sharing statement Readers are encouraged to contact the authors for further information regarding the data.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.