Article Text

Download PDFPDF

A review of risk perception measurement in tobacco control research
Free
  1. Annette R Kaufman1,
  2. Alexander Persoskie2,
  3. Jenny Twesten3,
  4. Julie Bromberg3
  1. 1 Tobacco Control Research Branch, Behavioral Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute, National Institutes of Health, Rockville, Maryland, USA
  2. 2 Office of Science, Center for Tobacco Products, US Food and Drug Administration, Silver Spring, Maryland, USA
  3. 3 BLH Technologies, Inc., Rockville, Maryland, USA
  1. Correspondence to Dr Annette R Kaufman, Tobacco Control Research Branch, Behavioral Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute, National Institutes of Health, Rockville, MD 20850, USA; kaufmana{at}mail.nih.gov

Abstract

Objective To describe the characteristics of risk perception measures used in tobacco control research and to evaluate whether these measures incorporate measurement suggestions put forward by risk perception measurement scholars.

Data sources Three databases (PubMed, PsycINFO and Web of Science) were searched in March 2015 for published English language peer-reviewed articles measuring tobacco risk perceptions (n=2557). The search string included terms related to tobacco products, perceptions and risk.

Study selection Three coders independently coded abstracts for initial inclusion. In total, 441 articles met the initial inclusion criteria, and 100 were randomly selected for a full-text review.

Data extraction A codebook was developed and tested through a training phase. Three coders independently coded the characteristics of each article (eg, population), multi-item measure (eg, validity) and item (eg, likelihood, affect, health outcome). Fifty-four articles, 33 measures and 239 items were coded.

Data synthesis Twenty-one articles had a multi-item risk perception measure, and 12 articles had one risk perception item. Many of the items asked about general health outcomes (36%), did not specify the person for whom risk was being judged (44%; eg, self, average person) or did not specify the conditions of use (27%; eg, the product used, intensity of use).

Conclusions There is little consistency across risk perception measures in tobacco research. There may be value in developing and disseminating best practices for assessing tobacco risk perceptions. A set of risk perception consensus measures may also benefit researchers in the field to help them consistently apply measurement recommendations.

  • prevention
  • surveillance and monitoring
  • smoking caused disease
View Full Text

Statistics from Altmetric.com

Introduction

Risk perceptions—subjective judgements about the potential harm to health related to a hazard—are important constructs in health behaviour and decision-making theories.1–4 Risk perceptions can motivate people to avoid health hazards and to engage in health-protective behaviours.5 The strength of these motivational effects may depend on other factors such as whether risk appraisals, response efficacy and self-efficacy are high.6 Risk perceptions play a role in the experimentation with and initiation of tobacco use,7 8 cessation of tobacco use,9–11 the frequency and intensity of tobacco use,12 and switching from one tobacco product to another.13 14 In tobacco control research, risk perceptions are often evaluated to examine the impact of advertising (eg, descriptors15 16), packaging,17–20 warning labels,21 22 flavours (eg, menthol23), educational interventions (eg, for water pipe24) and product design (eg, filters25) on users and non-users of tobacco products. Perceptions of risk of tobacco products are also relevant to how physicians address the treatment of tobacco use among their patients (eg, recommending cessation and delivering the five As13).

Despite the theoretical importance of risk perceptions, measures of perceived risk are not always associated with tobacco use behaviours. For example, among current cigarette smokers, some studies have found no association between risk perceptions and quit intentions and subsequent quitting behaviour, and others have found only weak associations.9 10 26–30 Moreover, some studies have observed a lack of association, small associations, and even reverse associations between risk perceptions and smoking behaviour.31–33 For instance, one study concluded that ‘the ability of these variables to explain individual variation is small’, with risk perceptions explaining only 5% of the variance in smoking behaviour beyond sociodemographic factors (Cutler and Glaeser, p241).31

One explanation for why risk perceptions are not always associated with tobacco use behaviours is that this association may be moderated by other factors. For example, one study found that the association between risk perceptions and smoking behaviour in youth and young adults was moderated by addiction beliefs and immediacy of health effects.7 Perceived health risk was a strong deterrent for those who believed that smoking was addictive and that smoking’s health effects occur rather immediately but not for those who viewed the addiction risk as low and the health effects as not immediate.7 The link between risk perceptions and smoking behaviour has also been shown to vary based on other factors, such as the extent to which people worry about their risk,34 the extent to which people have high response efficacy and self-efficacy for protective behaviours,6 and the complexity of the smoking behaviour (eg, planning to quit vs sustained quitting10).

Another potential explanation for the variability of findings is that some measures of risk perception may be more or less valid than others. Early research in the field of risk perception revealed that risks are more likely to motivate protective actions depending on their characteristics, such as whether the hazard evokes dread (eg, by being uncontrollable) and whether the hazard is unknown (eg, by being new and not yet understood by science).35 Dual-process models of decision-making propose that humans have two parallel systems for processing and responding to risk36system 1 uses heuristics and automatic associations to make rapid, low-effort judgements, whereas system 2 uses controlled, deliberative reasoning to make rule-based, effortful judgements. In general, system 1 is the default processing system but can be overridden with conscious and effortful system 2 processing. A key component of system 1 is affect (ie, the immediate, reflexive feeling of the goodness or badness of a stimulus37), which plays an important role in guiding decision-making.38 Research has uncovered an important bias known as unrealistic optimism, which is the tendency for people to minimise the extent to which they see themselves as at risk even when they recognise risks to other people.39

Based on advances in risk perception research and the field of survey design, researchers have identified important considerations for assessing risk perceptions in applied contexts. Brewer and colleagues1 made suggestions for designing risk perception measures using influenza as the example health risk, including (1) focusing on specific risks and outcomes rather than vague ones, (2) identifying the person for whom the risk is being evaluated, (3) making the question contingent on behaviour (ie, specifying the level of risk exposure) and (4) identifying a risk timeframe. These recommendations are aimed at standardising the risk that people are evaluating (ie, removing ambiguity) and increasing the likelihood that people provide perceptions of the same risk when responding to risk perception questions. For example, when asked a non-specific question about one’s own risk of developing lung cancer, a current smoker may estimate her risk as low, because she is expecting to quit in the near future. In contrast, another current smoker may estimate her own risk as high because she does not expect to quit. This inconsistency can be resolved by instead asking people to evaluate their risk of developing lung cancer if they continue smoking at their current rate. Failure to adhere to such measurement suggestions may lead to weak and inconsistent associations between risk perceptions and behaviour. Indeed, Brewer and colleagues1 found that risk perceptions for getting influenza were a stronger predictor of influenza vaccination behaviour when the measures adhered to their measurement suggestions.

A report by the Institute of Medicine (IOM) on studies of modified risk tobacco products also offers suggestions for how to measure risk perceptions.40 Consistent with Brewer and colleagues,1 the IOM suggested using conditional risk scenarios rather than unconditional risk assessments and assessing perceptions of specific tobacco-related outcomes rather than general perceptions of harm.40 The IOM report also suggested assessing affective reactions to risk40 consistent with research demonstrating the importance of affect in judgement and decision-making.38 41 Moreover, this suggestion is supported by research indicating that risk perception measures are more predictive of health-protective behaviours when measures ask about people’s feelings of risk rather than simply their cognitive probability judgements.42–44

Finally, general scientific principles also stress the need to empirically test the reliability and validity of measures of psychological constructs such as risk perception.45 46 Measures should be reliable over time, internally consistent (in the case of multi-item scales) and valid for assessing perceived risk. In general, there are benefits to using multi-item scales rather than single items to measure psychological constructs, such as risk perception, including reduced measurement error and the ability to assess internal consistency. Also, given that multi-item scales tend to outperform single-item scales in terms of their predictive validity,47 developing measures with multiple items for assessing risk perceptions is advisable.

Despite the importance of perceived risk and the breadth and volume of tobacco control research assessing risk perceptions, no studies have been conducted to review how this construct is measured across tobacco studies. Thus, the extent to which advances in the scientific understanding of risk perception have been incorporated into the field of tobacco control research remains unclear. Moreover, given that risk perception researchers have suggested ways to improve risk perception measures, there is an opportunity to evaluate whether tobacco studies typically reflect these considerations by incorporating the emerging guidance. This literature review is an effort to fill this gap by describing and evaluating the characteristics of risk perception items and measures used in tobacco control research.

Methods

Data sources

In March 2015, we conducted a search of three databases (PubMed, PsycINFO and Web of Science) for published, peer-reviewed articles measuring tobacco risk perceptions with no date restrictions. The search string (see online supplementary file: Supplement A) included terms related to tobacco products, perceptions and risk.

Study selection

Figure 1 shows the article inclusion flowchart. The initial search produced 5478 articles from the three databases, of which 2927 (53.4%) were duplicates and 2551 were unique. Three reviewers independently coded abstracts for initial inclusion, double coding approximately 10% of the abstracts (n=251), with 89.6% agreement among reviewers on average. Articles were included for further review if they appeared to describe research assessing people’s perceptions of risk in the context of tobacco products or tobacco use. Articles were excluded if they reported only qualitative data (n=67) or tests of tobacco-related knowledge (n=69) or were not assessing risk perceptions related to tobacco products or their use by an individual (n=1974). A total of 441 (17.2%) articles met the initial inclusion criteria.

Figure 1

Risk perception literature review flow chart.

Next, given the large number of articles (n=441), we randomly selected 100 (22.7%) articles for full-text review. We liken this approach to recruiting a random sample of human participants or taking a random sample of media or web content48–50 to draw inferences about a larger population. We included an article for coding if it stated that it assessed people’s perceptions of risk or if any of the items met our definition of tobacco risk perception as a participant’s subjective judgement about the potential harm to health related to tobacco products or their use. Examples of potential harms to health included cancer, heart disease, respiratory effects, oral effects and the risk of becoming addicted. After full-article review, 29 articles were excluded (see footnote in figure 1). Thus, 71 articles were identified as being related to tobacco risk perceptions and included in the review. If an article did not provide the risk perception survey item(s) or an adequate description of the item(s), the authors of the publication were contacted to provide this information. For measure and item-level coding, an additional 17 articles were excluded because of insufficient information.

Data extraction

We coded at the article, multi-item measure and item levels. Multi-item measures and items were coded only if they were about risk perceptions. Within a multi-item measure, items that did not meet the inclusion criteria were excluded from coding (eg, an item assessing harm from secondhand smoke).

We developed a codebook and tested it through a training phase in which all authors coded the first 10 articles. Three reviewers then independently coded 20% (n=11) of the articles, and 90% agreement among reviewers was achieved across all codes. Discrepancies were resolved through discussions among the reviewers and lead authors.

Data synthesis

Population

Articles were coded based on the study populations and included youth (ages younger than 18), adults (ages 18 and older) or both.

Research design

Articles were coded for being prospective (same participants assessed more than once over time), cross-sectional (assessed at a single point in time), experimental (random assignment, including both prospective and cross-sectional designs) or other.

Citation of source

Items and measures were coded based on whether the article cited the source of the item or measure (eg, a prior study) or stated that it had been adapted or used verbatim from a previous study.

Validity and reliability

Items and measures were coded as valid or reliable if the study completed validity or reliability testing or if the authors stated that the items or measures were previously shown to be valid or reliable.

Exact wording

Items were coded based on the amount of information provided about the questions and response options. (For example, did the article include the exact wording of the item, partial wording or no wording? If the latter two, we coded the item if the author sent exact wording after being contacted for the information.)

Risk perception characteristics

Items were coded based on their risk perception characteristics. These characteristics included whether the items assessed absolute or comparative likelihood; were contingent on behaviour (ie, particular product, frequency of use, timeframe or some combination of these); identified the risk target; and assessed risk severity, affective responses to risk, the controllability of the risk, the extent to which the risk is known to experts or others and general perceptions of harmfulness. Table 1 provides descriptions for each of these characteristics.

Table 1

Risk perception item characteristics

Response options

The response options for each item were coded based on whether they used a numeric judgement scale (eg, “out of 100 smokers, how many do you think will get lung cancer due to their smoking?”) or a Likert-type rating scale (ie, any ordinal scale response with verbally labelled options). ‘Other’ response options included dichotomous (eg, yes/no) or non-ordinal categories (eg, males, females, both equally likely). We also created a response transformation code to capture whether the response options from one or more items were combined by taking the difference or product between items. This code did not include collapsing response options or instances in which items were simply combined into a risk perception index, which we refer to as a measure.

Additional analyses

Analyses were conducted to examine different combinations of item characteristics. To describe items that assessed product perceptions, we examined items that were coded as both conditional on product and general health outcome harm. We examined severity items that were also coded with the following item characteristics: likelihood, health outcome and risk target. We examined the items assessing affect and their overlap with health outcome and risk target.

Results

Study characteristics

Fifty-four studies published between 1981 and 2015 were included in the review and can be found in online supplementary file: Supplement B.12 51–103 Sixty-nine per cent (n=37) of the articles focused only on adults, 19% (n=10) of the articles focused only on youth, and 13% (n=7) of the articles included both youth and adults. Seventy-four per cent (n=40) of the studies were cross-sectional, 18.5% (n=10) were prospective, 5.5% (n=3) were experimental and 1.9% (n=1) were other (repeated measures).

Multi-item measure descriptive statistics

Of the 54 articles, 21 (39%) reported using a multi-item risk perception measure (ie, two or more items combined into a single scale or index). Across the 21 articles that used multi-item risk perception measures, a total of 33 measures were coded. Six of these measures (18%) were used exactly and four (12%) were adapted from a previously published study. For the other 23 measures (70%), the article did not provide a citation or source for the origin of the measure. Only four of the measures (12%) in two separate articles were described as being previously validated; neither reported any validity testing. Twenty-three measures (70%) across 14 studies assessed the reliability of the measure using Cronbach’s alpha.

Characteristics of risk perception items

Table 2 shows the number of risk perception items included across articles. Many articles included multiple risk perception items, such that 239 items were coded across the 54 articles. The number of items per article ranged from 1 to 18, with the average number of risk perception items being 4.4 per article. Forty-one items (17%), including those within and not within multi-item measures, were used exactly as used in another cited study.

Table 2

Number of risk perception items across articles

Table 3 presents the risk perception item descriptive statistics. More than half (57%) of the risk perception items included a specific health outcome. Slightly less than half (48%) of the items assessed risk for the self, and 42% (n=100) assessed risk generally (eg, ‘It seems like almost everything causes cancer’). Three quarters of the items (n=179) assessed absolute risk and 22% (n=52) assessed comparative risk. Few items assessed severity (n=27, 11%), affect (n=22, 9%), controllability (n=19, 8%) or known risk (n=8, 3%).

Table 3

Risk perception item descriptive statistics

Nearly 73% (n=173) of the items included consideration of specific (often hypothetical) behaviours, such as the use of a particular product, frequency of use, timeframe or some combination of these. Seventy per cent (n=167) of the items asked about risk conditional on using a particular product (eg, “If I smoke cigarettes, I will live for a long time”). Only 13% (n=31) of the items specified a frequency of use (eg, “I can smoke a couple of cigarettes a day and still not harm my health”) and 8% (n=20) specified the timeframe (eg, “Compared with others your same age and sex, how would you rate your risk of having a heart attack within the next 10 years?”or “There’s no risk of getting cancer if someone only smokes a few years”). Only 5% (n=12) of items specified all of the above (ie, product, frequency and timeframe) (eg, “Now imagine that you continued to smoke about 2 OR 3 cigarettes each day for the rest of your life. What is the chance that you will get lung cancer?”).

Most (n=183, 77%) of the item responses used a Likert-type rating scale and few items (n=15, 6%) used a numeric judgement scale. All numeric risk estimates were for absolute risk perception items. Of these, more than half (n=8, 53%) were for risk estimates for the self, whereas 20% (n=3) were for specific others and 27% (n=4) were for general risk. In addition, 73% of the numeric risk estimate response options were for specific risks rather than general risks (n=4, 27%). Few articles reported combining responses across different items (eg, by taking the difference or product between items) (n=8, 15%). Among those items that were transformed (n=28), 57% (n=16) used a rating scale whereas 11% (n=2) used a numeric estimate.

Further analyses revealed that 32% (n=82) of the items assessed perceived product harm (ie, items that were coded as both conditional on product and general health outcome: ‘Do you think smoking tobacco is harmful to your health?’) and that 55% (n=132) of items assessed risk perceptions of health outcomes (ie, items that were coded as self-risk, specific other risk, or average other risk and all health outcomes). Seventy-eight per cent (n=21) of the severity items assessed absolute risk, whereas 15% (n=4) assessed comparative risk. Of the articles that assessed severity (n=27), 63% (n=17) assessed a specific risk, 26% (n=7) assessed a general risk and 11% (n=3) assessed a hybrid of both. In addition, 44% (n=12) of those that assessed severity were for risk for the self and 41% (n=11) assessed general risk. All items assessing affect were for the self. For those items assessing affect (n=22), 64% (n=14) assessed a specific risk, 23% (n=5) assessed a general risk and 9% (n=2) assessed a hybrid of both.

Discussion

To our knowledge, this is the first review examining how risk perceptions are measured in tobacco control research. It is clear that risk perceptions are indeed measured in a large number of studies and that there is wide variation in the types of risk perception items used. There is a wealth of scientific research identifying important aspects of how people perceive risk and the components of risk perceptions that motivate behaviour. This study revealed that, among the studies reviewed, many did not incorporate advances from the field of risk perception.1 40

More than half of the studies used single items to assess risk perception, even though using multi-item measures can increase statistical power.47 The reliability and validity of measures are integral to the interpretability of research involving psychological constructs such as risk perception. In studies of tobacco risk perception, when small or no effects are observed, the reason may be the use of single-item measures with low reliability and validity. In this review, we found almost no validity testing and minimal reliability testing. Items that assess different aspects of risk perception (eg, absolute vs comparative) should not be combined into measures without psychometric or other testing showing that they can be combined.

The items and multi-item measures used in the studies in this review generally did not comply with the recommendations put forward by Brewer and colleagues1 and the IOM,40 although some recommendations were more likely to be met than others. The extent to which items specified various aspects of the risk to be judged (eg, the heaviness of product use) was highly variable across items. The health risk or outcome was specified in 57% of the items analysed in this review. The tobacco product was specified in more than half of the items (70%), but 27% of the items were not conditional on any aspect of use including the product under examination, heaviness of use or timeframe. Almost half of the risk perception items (48%) assessed risk for the self, but 42% assessed general risk without specifying the person at risk. Clearly, it is not possible to specify all aspects of product use that could contribute to one’s risk, as this would cause risk perception items to become too cognitively burdensome. However, ideally, items should be specific about the key aspects of the product use behaviour under consideration (eg, frequency of use) and should assess the risk for the self.1 40 104–110 It may also be useful to assess whether particular risk outcomes are salient to individuals.111

The IOM report recommended using either response scales with numerical likelihood estimates (ie, 0%–100%) or comparative risk assessments (eg, ‘Compared to [another product], is this modified risk tobacco product (MRTP) more or less likely to cause [a specific outcome]?’) instead of scales with verbal labels (eg, ‘very likely’).40 This recommendation may have been aimed at reducing noise in responses (eg, differences in interpreting verbal labels) or increasing the interpretability of estimates by using an objectively defined response scale.112 However, a prior study comparing percentage scales with response scales with verbal labels found that the scale with verbal labels was a better predictor of behavioural intentions and individual preferences than the scale with numeric percentage labels.113 Also, it is not possible to infer the accuracy of risk perceptions by eliciting numeric estimates and comparing them with objective levels of risk.114–116 In our review, few measures included numeric response scales, with more than three-fourths using scales with verbal labels. The IOM report also recommended comparative response scales, but only 22% of the items reviewed here assessed direct comparative risk perception.40 The evidence is mixed regarding how and when absolute vs comparative risk perception measurement is most predictive of outcomes of interest, including behaviour.117 118 A recent review of the literature on perceptions of relative risk from various tobacco and nicotine products found variation in how the perceptions were measured.119 Future research examining these important issues can inform risk perception measurement in tobacco control research.

Recognising the role of affect in risky decision-making41 has allowed researchers to ‘explain a wide range of phenomena that have resisted interpretation in cognitive-consequentialist terms’ (Loewenstein et al, p267)38 In tobacco control, measures that focus only on cognitive types of perceptions may overlook affect-laden components that are strong motivators of behaviour and are sensitive to factors such as marketing and advertising. However, this review found that only 9% of risk perception items included affect. It is possible that including affect in risk perception measures might improve our ability to predict important tobacco outcomes, including use behaviour.

Although not necessarily recommended as part of the standard assessment of risk perceptions, researchers have also identified certain aspects of risk perceptions that play an important role in motivating protective behaviours.5 6 These include whether the hazard is dreaded and unknown. We found that few studies assessed components of these risk perception factors such as the perceived controllability of the risk, its severity, and the extent to which it is seen to be novel and unknown to science. Further research measuring these aspects of perceived risk may be informative.

This review must be viewed in light of several limitations. Although this was not a comprehensive review of all studies, we selected a random sample of studies for analysis to reduce bias. While some studies preceded publication of Brewer and colleagues’ paper, when we examined the subset of articles published afterwards, we found that the characteristics of the items were similar to what is reported here.

Conclusions

Suggestions for how to measure risk perceptions1 40 45 46 have not been consistently incorporated in tobacco research, which may make detecting significant associations and effects more difficult. These results indicate that the field may benefit from the development and dissemination of risk perception measurement best practices and specific guidelines for risk perception measurement in tobacco control research. These tobacco-specific guidelines can take into account critical nuances and special considerations. Most importantly, we hope to encourage a dialogue around risk perception measurement among tobacco control researchers. This review highlights the importance and potential benefits of risk perception measurement harmonisation in tobacco control research.

What this paper adds

  • In tobacco control research, risk perceptions are beliefs about the potential harms to health from using tobacco products and are widely assessed across the field.

  • Risk perceptions are associated with numerous tobacco-related health behaviours, such as initiation, cessation and product switching.

  • There is currently no consensus on how to best measure tobacco risk perceptions, but risk perception researchers have put forward some general guidelines for risk perception measures.

  • This study provides the first review of risk perception measurement in tobacco control research.

  • Previously developed suggestions for risk perception measurement have not been consistently incorporated in tobacco research, making detecting effects and associations concerning this important construct more difficult.

Acknowledgments

We thank Alexandra Stern for her contributions to the inclusion criteria, codebook development and initial article coding. We thank William Klein, Michele Bloch and Jerry Suls for their feedback on this manuscript.

References

View Abstract

Footnotes

  • Contributors ARK conceptualised the study and led the project and writing. All authors contributed to the development of the coding scheme. JB and JT conducted the coding and analyses and drafted the methods. ARK and AP reviewed the codes and results. All authors contributed to the writing and revision and approved the final version of the manuscript.

  • Disclaimer The views and opinions expressed in this manuscript are those of the authors only and do not necessarily represent the views, official policy, or position of the U.S. Department of Health and Human Services or any of its affiliated institutions or agencies.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.