You are here

  • Home
  • Archive
  • Volume 27, Issue 3
  • Evaluating the pictorial warnings on tobacco products in Arabian Gulf countries against other international pictorial warnings

Article Text

Article menu

Download PDFPDF

Research paper
Evaluating the pictorial warnings on tobacco products in Arabian Gulf countries against other international pictorial warnings
Free
  1. Nasser F BinDhim1,2,
  2. Kevin McGeechan3,
  3. Anwar K T Alanazi4,
  4. Hossam M S Alanazi4,
  5. Sasoun A J Alanazi4,
  6. Solaiman M Al-Hadlaq5,
  7. Hisham Aljadhey2,
  8. Tariq M Alhawassi4,6,
  9. Nadia A Alghamdi7,
  10. Ahmed M Shaman4,
  11. Meshari S Alquwayzani2,
  12. Mada H Basyouni8,9
  1. 1 Department of Health Informatics, Saudi Electronic University, Riyadh, Saudi Arabia
  2. 2 Saudi Food and Drug Authority, Riyadh, Saudi Arabia
  3. 3 Department of Public Health, University of Sydney, Sydney, Australia
  4. 4 College of Pharmacy, King Saud University, Riyadh, Saudi Arabia
  5. 5 College of Dentistry, King Saud University, Riyadh, Saudi Arabia
  6. 6 Medication Safety Research Chair, King Saud University, Riyadh, Saudi Arabia
  7. 7 Public Health Department, College of Health Sciences, Saudi Electronic University, Riyadh, Saudi Arabia
  8. 8 The Smart Health Project, Riyadh, Saudi Arabia
  9. 9 Investigational Drugs and Research Unit, King Khalid University Hospital, Riyadh, Saudi Arabia
  1. Correspondence to Dr Nasser F BinDhim, Saudi Food and Drug Authority, Riyadh, Saudi Arabia; nd{at}nasserdhim.com

Abstract

Background Few assessments of pictorial warnings (PWs) on cigarette packs implemented in Gulf Cooperation Council (GCC) countries have been done.

Methods This article includes two cross-sectional studies. In Study 1, convenience samples of adults from the Kingdom of Saudi Arabia (n=111) and USA (n=115) participated in a consumer survey to rate a total of nine PWs from the GCC, Australia and the UK. Outcome measures were affective responses to PWs and concerns about smoking. In Study 2, tobacco control experts (n=14) from multiple countries rated the same PWs on a potential efficacy scale and completed one open-ended question about each. The PWs were altered to mask their country of origin. Analyses compared ranking on multiple outcomes and examined ratings by country of origin and by smoking status.

Results In the consumer survey, participants from both countries rated the PWs from GCC lower than PWs from other countries on the two measures. The mixed-model analysis showed significant differences between the PWs from Australia and those from the GCC and between the PWs from the UK and those from the GCC (p<0.001) in the consumer and expert samples. The experts’ comments about the PWs implemented in the GCC were negative overall and confirmed previously identified themes about effective PWs.

Conclusion This study shows PWs originating from the GCC had significantly lower ratings than those implemented in Australia and the UK. The GCC countries may need to re-evaluate the currently implemented PWs and update them periodically.

  • Pictorial Warnings
  • Tobacco
  • Warning Messages
  • Public Health
  • Packaging and Labelling

https://doi.org/10.1136/tobaccocontrol-2016-053323

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Background

    The prevalence of adult daily cigarette smoking in Gulf Cooperation Council (GCC) countries ranges from 19% to 40%.1–5 In the Kingdom of Saudi Arabia (KSA), the prevalence is approximately 22.6% among adults (26.5% of males and 9% of females).1 Furthermore, according to the latest WHO youth tobacco survey, 25% of youths (ages 13–15 years) have experimented with smoking (34.6% of boys and 15.6% of girls).2 The World Bank estimates the following levels of smoking prevalence for other GCC countries: 40% in Bahrain and 19% in Oman.3 According to the WHO Framework Convention on Tobacco Control (FCTC) Implementation Database and Updates from Parties, Qatar’s latest report stated prevalence was 10.2%.4 However, the stated prevalence in United Arab Emirates (UAE) was 32.8%.5

    Article 11 of the FCTC mandates the inclusion of PWs on tobacco product packaging.6 Many research studies have supported the use of PWs.7–12 In summary, the studies show that PWs (a) are more likely to be noticed than text-only warning labels, (b) are more effective for educating smokers and non-smokers about smoking health risks and for increasing smokers’ thoughts about these health risks, (c) are associated with increased motivation to quit smoking and (d) may help prevent smoking initiation.7 8 13 Different PWs have different effects with regard to making smokers think about smoking-related dangers and consider quitting smoking.7–12 For example, PWs have the most impact when they are prominent and when they include emotionally engaging imagery that illustrates negative bodily impacts or human suffering due to smoking.8–11 Finally, studies have found differences between the PWs implemented in different countries in terms of salience and cognitive and behavioural impacts.11

    GCC countries have ratified the WHO’s FCTC and adopted a comprehensive ban on all tobacco advertising, promotion and sponsorship; a ban on the display of tobacco products at points of sales; and adopted PWs.14 The PWs currently implemented in the GCC were selected and approved by the GCC Health Ministers’ Council and the GCC Standardization Organization on 9 August 2011.15 16 Only three PWs were approved for use on cigarette packages in all GCC countries, and there have been no updates or new images added during the last five years. The authors could not find any relevant documentation regarding the GCC’s PW selection process.

    A study in Qatar evaluated the general public’s awareness, beliefs and perceptions on anti-tobacco PWs on cigarette packs prior to the GCC implementation.17 The study found that one-third of the respondents had no idea about any specific text warning messages on tobacco products sold, and about 45% of them did not know what a PW was.17 The current study will be one of the first in the Gulf region to evaluate the implemented PWs. The objective of this study is to evaluate the three PWs that originated from the GCC Health Ministers’ Council and that were implemented in the GCC by comparing them to similar PWs from other countries specifically Australia and UK.

    Overview

    This research gathered opinions of GCC PWs from different populations by conducting a consumer survey in the KSA and USA (Study 1) and by surveying tobacco control experts in multiple countries (Study 2). The research protocol was approved by the King Saud University Ethics Committee. Both consumer and expert samples were used to provide multi-dimensional data about the comparable assessments of the PWs. For the consumer and expert samples, an online questionnaire was developed; the consumer and expert questionnaires could only be completed after the user received a unique invitation link. Participants were unable to complete the questionnaire again or to share the link.

    Study 1 – Consumer survey methods

    Sample

    Only participants aged 18 year or older were invited to complete this study, which was conducted between 5 December 2015 and 14 January 2016. The rationale for recruiting the sample from the USA was mainly to obtain a response from a reference sample that had presumably never seen the PWs under evaluation in this study (as the sample from the KSA might recognise the PWs that originated from the GCC) and to add a sociocultural dimension to the study in terms of how the two samples will rate the PWs.

    For the participants from KSA, a convenience sample of 111 participants was recruited via direct contact with the participants. Participants were approached in public areas, such as shopping centres and recreational spaces; if they were interested in participating, a unique study participation link was sent via email or short messaging service. The web application developed for this study updates the researcher using sampling blocks based on age, gender and smoking status; this ensures the coverage of equal demographics during the recruitment period.

    For the sample from USA, 115 participants were recruited via an online research recruitment company (Find Participants), which was used to send the survey to a targeted list of participants via email. We verified the participants’ locations via their Internet protocols to confirm that they were from USA.

    Stimuli

    Nine PWs were selected from different origins (three from Australia, three from the GCC, and three from the UK) based on the similarity of the warning topics. The PWs from Australia and the UK were chosen for this study mainly because their evidence-based effectiveness was supported by various studies.18–20 The participants were blinded to the sources of the warnings, which were modified to have a standard design and displayed in random order (figure 1).

    Figure 1
    Figure 1

    The nine pictorial warnings (PWs) used in this research. High resolution images can be found in the supplementary file.

    Measures

    The participants were asked to rate each of nine PWs on two scales (asked repeatedly after each PW) – the Brief Worry Scale about Smoking (BWS) and the Self-Assessment Manikin (SAM). The participants were asked to look at each PW carefully and read the label then move on directly to the rating. The BWS is a four-item scale that measures worry about physical health as a consequence of smoking.21 The participants answered each question using 7-point scales ranging from ‘not at all’ to ‘very much’. For this study, the BWS was modified to include worry about the physical health of others the respondents cared about. The BWS was selected because of its strong predictive validity and strong reliability.21 22 The SAM is a three-item, non-verbal pictorial assessment method that directly measures the pleasure, arousal, and dominance associated with a person’s affective reaction to a wide variety of stimuli.23 The participants answered each question using 9-point scales ranging from ‘1=negative’ to ‘9= positive’ emotional effects. The SAM scale is frequently used in the food and beverage industry to study the consumers’ packaging, product experiences, and satisfaction levels with product packaging.24 The SAM has been included in this study to add a non-health-related aspect and to provide a more focused assessment of the PWs using an industry-standard evaluation scale.24 The questionnaire included demographic data on age, gender, educational level and smoking status that were collected before the rating.

    Data analysis and sample size

    Descriptive analysis was used to describe the participants’ characteristics. A Friedman test was used to rank the PWs. We used a mixed-model, repeated-measures analysis to investigate the differences among the PWs based on their origins and to examine rating differences between smokers and non-smokers. The repeated-measures design was used to provide a definitive evaluation of within-person changes over time or measurements.25

    The sample size calculations for the mixed-model analysis are affected by prior information about random effects, within-subject correlations, and other values that are often difficult to obtain without pilot data.26 27 With the three groups of PWs measured across nine observations an alpha of 0.05, a power of 0.80 and a medium effect size of 0.25, we estimated that 90 participants were needed to detect significant differences between the three PW groups.

    Study 1 – Consumer survey results

    demographics

    Table 1 shows the participant demographics for both consumer samples.

    View this table:
    Table 1

    Participants’ characteristics

    Image ranking

    shows the mean rankings of the PWs in this study, listed from least to greatest emotional effect. A Friedman test of mean rankings of the PWs indicated significant differences among some of the median consumer ratings (table 2) on the BWS scale (KSA sample: χ2[8, n=111]=141.19, p<0.001; USA sample: χ2[8, n=115]=92.29, p<0.001) and on the SAM scale (KSA sample: χ2[8, n=111]=152.53, p<0.001; USA sample: χ2[8, n=111]=115.79, p<0.001).

    View this table:
    Table 2

    Friedman test mean consumer rankings of the Pictorial Warnings

    Supplementary file

    Differences between PWs based on their origins

    In both samples, the mixed-model analysis showed significant differences between the PWs from Australia and those from the GCC and between the PWs from the UK and those from the GCC (table 3). For example, the PWs from the UK and Australia outperformed the PWs from the GCC on average by 12.5 and 10.1 rating points, respectively, on the SAM scale in the KSA sample. However, there were no significant differences between the PWs from Australia and those from the UK (table 3).

    View this table:
    Table 3

    Mixed-model analysis of the differences between PWs based on their origins

    When the two samples were combined, significant differences were still found between the PWs from Australia and those from the GCC and between the PWs from the UK and those from the GCC, with no interaction between the samples and the PWs’ origins. Overall, the PWs from the UK and Australia scored better than those from the GCC by an average 8.9 and 6.5 rating points, respectively, on the BWS scale. Furthermore, on the SAM scale, the PWs from the UK and Australia performed better than those from the GCC by, on average, with 7.8 and 5.9 rating points, respectively.

    Differences between smokers and non-smokers

    In the sample from KSA the mixed-model analysis showed no differences between smokers and non-smokers in how they rated the PWs from Australia, the GCC, and the UK on the SAM scale. However, on the BWS scale, a significant difference was found between smokers and non-smokers for PWs that originated from the GCC (estimate of fixed effect=−19.3, p<0.001).

    In the sample from the USA, the mixed-model analysis showed there were no differences in either the SAM or the BWS between smokers’ and non-smokers’ ratings of the PWs originating from Australia, the UK and the GCC.

    Study 2 – Expert survey methods

    Sample

    Between 10 October 2015 and 5 November 2015, 24 experts from different countries were invited via email to rate the same nine PWs; of them, 14 have completed the study. The experts were initially selected based on their previous scientific publications related to the implementation of PWs.

    Measures

    The experts were asked to rate each of nine PWs in Study 1 on a scale developed for this study. The experts rated each image based on the PWs’ potential ability to effectively provide the expected benefits of the WHO’s recommendations. The participants answered six question using five-point scales (supplementary file) ranging from ‘0= strongly disagree’ to ‘4= strongly agree’. The six attributes cover potential efficacy of the PW and text label, emotional engagement, potential persuasiveness, knowledge and awareness. The outcome measured was the average rating. In addition to filling out the questionnaire, each expert was asked to write optional comments describing her or his opinions about each of the nine images to capture more details about the experts’ perceptions of the images’ relative quality.

    Study 2 – Expert survey results

    The 14 experts were from New Zealand, Australia, India, Mexico, UK, Canada, Greece and USA.

    Image ranking

    Table 4 showed results of the Friedman test which indicated there were statistically significant differences in the experts’ ratings of the PWs (χ2[8, n=14]=50.32, p<0.001).

    View this table:
    Table 4

    Mean ranking of expert’s rating of the PWs on the expert’s scale

    Differences between PWs based on their origins

    The mixed-model analysis showed significant differences between the PWs from Australia and those from the GCC (estimate of fixed effect=16.3, p<0.001) and between the PWs from the UK and those from the GCC (estimate of fixed effect=18.2, p<0.001). There were no significant differences between the PWs from Australia and those from the UK (estimate of fixed effect=−1.9, p=0.57).

    Qualitative analysis of the experts’ comments on each pictorial warning

    Overall, eight of fourteen experts commented on the PWs. In particular, the experts criticised the three GCC PWs for being mostly unrealistic and for having weak connexions to the text warnings. Some experts perceived two (Image 4 ‘the burning hand’ and Image 6 ‘with skull shape’) of the three GCC PWs as being triggers for smoking instead of warnings. Image 5, ‘the apparently pregnant woman’, was described as equivocal and distracting.

    In addition, all experts who commented agreed that the following factors can weaken a PW’s effect: unrealistic images, inadequate expression of the impact smoking has on the human body, and lack of a strong connexion to the text warning. For example, regarding Image 7 from UK – ‘the one with baby back facing the consumer’ – all of the experts agreed the image was gentle and it would not have a strong emotional impact on smokers. The same image was among the lowest three PWs rated by the consumers.

    Discussion

    The results of both studies showed that the PWs implemented in the GCC countries were significantly less effective at evoking emotional feelings, emotions and worry about smoking, than the PWs from Australia and the UK. In addition, the qualitative analysis of the experts’ comments on each of the nine PWs corresponded with the quantitative analysis of the two studies. Finally, the study highlighted potential improvements for the implementation of PWs in the GCC.

    In terms of the ranking the PWs, there was complete agreement among the three samples that Image 6 and Image 4 were the least effective, with the mean rank of Image 6 – ‘the one with skull shape’ – skewed towards evoking positive rather than negative feelings. In terms of the differences between the PWs based on their origins, the mixed-methods analysis showed complete agreement among the three samples that the images from the GCC had significantly lower ratings than those from the UK or Australia. These results clearly show the images implemented in the GCC were less likely to achieve the intended goals of implementing PWs on tobacco products.

    This study’s results identified themes associated with the weak PWs, which included unrealistic images, inadequate expression of smoking’s impact on the human body, and the lack of a strong connexion to the text warning. The comparative effect of PWs between countries has been explored previously. In 2010, a survey of adult smokers in Brazil, Uruguay and Mexico showed that there was a lower impact of PWs in Uruguay than in the other countries.13 The authors suggested the ‘difference can be attributed to the relatively abstract imagery of PWs in Uruguay, which contrasts with the more gruesome illustrations of smoking consequences in the other countries’.13 This conclusion is in alignment with this study’s findings regarding the PWs from the GCC. The three PWs from the GCC are similarly weak in comparison to effective PWs from Australia and the UK.

    It is important for the GCC to include more powerful PWs to maximise their benefit. The GCC countries might need to consider implementing new PWs that are more prominent and more realistic and that have stronger text warnings, as identified above. Furthermore, the GCC countries may need to evaluate the implemented PWs regularly to improve their quality and effectiveness. There is also a lack of research on PWs in the GCC, which could also contribute to improving the implementation effectiveness and highlight areas of potential improvement.

    In addition, the GCC countries need to update their PWs periodically to avoid desensitisation, as there were significant differences in how smokers and non-smokers from the KSA sample rated the PWs that originated from the GCC countries; these differences, which were not found in the USA sample, might reflect desensitisation due to repeated exposure to the PWs over a long period. There were no significant differences between smokers’ and non-smokers’ ratings of PWs from the UK and Australia in either consumer sample. This may support the evidence that effective PWs can be beneficial for both smokers and non-smokers.13 28

    The KSA and USA samples showed agreement in that the images from the GCC had significantly lower ratings than did the images from the other countries. However, the USA sample had a more moderate rating for the PWs than did the KSA sample. The KSA participants’ stronger reaction to the prominent PWs might be due to the weak advertisement of anti-tobacco messages on local media in KSA29 or to the participants being exposed to more pro-smoking messages than anti-smoking ones.29 The implementation of more prominent PWs might play an important role towards more effectively educating smokers and non-smokers about the health risks of smoking and for increasing thoughts about these health risks.

    Finally, both studies were limited by the cross-sectional design, which prevented drawing inference about causation of the identified associations from being investigated. The use of a small convenience sample may also introduce bias in terms of how the participants rated the PWs in both studies. It may also weaken the generalisability of the results on the populations of interest.

    Conclusion

    This study showed the PWs implemented in the GCC countries are significantly less effective than images from Australia and the UK, as measured by consumers’ and tobacco-control experts’ ratings, with almost complete agreement between experts and consumers. The GCC countries should re-evaluate the PWs, they currently implement and update them periodically. In addition, this study confirmed previously identified themes associated with effective PWs, including the use of realistic images that show the impact of smoking has on the human body and the use of images with a strong connexion to the text warning.

    What this paper adds

    • The implementation of emotionally engaging pictorial warnings (PWs) on tobacco products has been associated with various positive outcomes among adults and adolescents in many countries. Previous studies found that PWs have the most impact when they are prominent and when they include emotionally engaging imagery that illustrates negative bodily impacts or human suffering due to smoking.

    • The Gulf Cooperation Council (GCC) countries approved only three PWs to be used on cigarette packages in those countries; these PWs have never been evaluated before.

    • This study assessed the three PWs implemented in the GCC for comparison to similar PWs from other countries.

    • This study shows consumers and tobacco–control experts rated the PWs originating from the GCC significantly lower than those implemented in Australia and the UK.

    Acknowledgments

    The authors would like to extend their sincere appreciation to the Deanship of Scientific Research at Saudi Electronic University for funding this research project. The authors would like to thank the College of Pharmacy Research at the King Saud University. Finally, the authors would like to acknowledge the suggestions provided by Professor Robert West to improve the study design.

    References

    1. ↵
      2. Bassiony MM
      . Smoking in Saudi Arabia. Saudi Med J 2009;30:876–81.
      OpenUrlPubMedWeb of Science
    2. ↵
      WHO. Global Youth tobacco survey (GYTS), Saudi Arabia, 2010.
    3. ↵
      The World Bank. Smoking prevalence. secondary smoking prevalence. http://data.worldbank.org/indicator/SH.PRV.SMOK.MA
    4. ↵
    5. ↵
      Fctc WHO. United Arab Emirates. Secondary United Arab Emirates. http://apps.who.int/fctc/implementation/database/parties/United_Arab_Emirates
    6. ↵
      2. Yach D
      . WHO Framework Convention on tobacco control. The Lancet 2003;361(9357):611.doi:10.1016/S0140-6736(03)12532-9
      OpenUrl
    7. ↵
      2. Hammond D
      . Health warning messages on tobacco products: a Review. Tobacco control 2011; 20(5):327-37.
      OpenUrlAbstract/FREE Full Text
    8. ↵
      2. Fong GT ,
      3. Hammond D ,
      4. Hitchman SC
      . The impact of pictures on the effectiveness of tobacco warnings. Bull World Health Organ 2009;87:640–3.doi:10.2471/BLT.09.069575
      OpenUrlCrossRefPubMedWeb of Science
    9. ↵
      2. Thrasher JF ,
      3. Hammond D ,
      4. Fong GT , et al
      . Smokers' reactions to cigarette package warnings with graphic imagery and with only text: a comparison between Mexico and Canada. Salud Publica Mex 2007;49:s233–40.doi:10.1590/S0036-36342007000800013
      OpenUrlPubMedWeb of Science
    10. ↵
      2. Hammond D ,
      3. Fong GT ,
      4. Borland R , et al
      . Text and graphic warnings on cigarette packages: findings from the international tobacco control four country study. Am J Prev Med 2007;32:202–9.doi:10.1016/j.amepre.2006.11.011
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      2. Thrasher JF ,
      3. Villalobos V ,
      4. Szklo A , et al
      ; Assessing the impact of cigarette package health warning labels: a cross-country comparison in Brazil, Uruguay and Mexico. salud pública de méxico 2010;52:S206–S15.
      OpenUrl
    12. ↵
      2. Muñoz MA ,
      3. Viedma-Del-Jesús MI ,
      4. Rosselló F , et al
      . The emotional impact of european tobacco-warning images. Tob Control 2013;22:123–9.doi:10.1136/tobaccocontrol-2011-050070
      OpenUrlAbstract/FREE Full Text
    13. ↵
      2. Noar SM ,
      3. Hall MG ,
      4. Francis DB , et al
      . Pictorial cigarette pack warnings: a meta-analysis of experimental studies. Tob Control 2016;25(3):341-54.doi:10.1136/tobaccocontrol-2014-051978
      OpenUrlAbstract/FREE Full Text
    14. ↵
      Fctc WHO. Parties to the WHO Framework Convention on tobacco control. World Health Organization website, 2012.
    15. ↵
      GCC Health Ministers’ Council. TOBACCO CONTROL PROGRAM. Secondary TOBACCO CONTROL PROGRAM. http://www.sgh.org.sa/en-us/technicalprograms/noncommunicablediseases/tobaccocontrol/overview.aspx
    16. ↵
      Tobacco Labelling Resource Center. Secondary 2012. http://www.tobaccolabels.ca/countries/Saudi-Arabia/
    17. ↵
      2. Awaisu A ,
      3. Hagi A ,
      4. Ashour MA , et al
      . Pictorial health warnings on cigarette packages in Qatar: preimplementation awareness and perceptions of ever-smokers versus never-smokers. Nicotine Tob Res 2013; 15(10):1765–72.doi:10.1093/ntr/ntt059
      OpenUrlCrossRefPubMed
    18. ↵
      2. Miller CL ,
      3. Hill DJ ,
      4. Quester PG , et al
      . Impact on the Australian quitline of new graphic cigarette pack warnings including the Quitline number. Tob Control 2009;18:235–7.doi:10.1136/tc.2008.028290
      OpenUrlAbstract/FREE Full Text
    19. ↵
      2. Hammond D ,
      3. Fong GT ,
      4. McNeill A , et al
      . Effectiveness of cigarette warning labels in informing smokers about the risks of smoking: findings from the International Tobacco Control (ITC) Four country survey. Tob Control 2006; 15(supp 3):iii19–iii25.doi:10.1136/tc.2005.012294
      OpenUrlAbstract/FREE Full Text
    20. ↵
      2. Miller CL ,
      3. Quester PG ,
      4. Hill DJ , et al.
      Smokers' recall of Australian graphic cigarette packet warnings & awareness of associated health effects, 2005-2008 BMC Public Health 2011; 11(1):1.doi:10.1186/1471-2458-11-238
      OpenUrlCrossRefPubMed
    21. ↵
      2. Dijkstra A ,
      3. Brosschot J
      . Worry about health in smoking behaviour change. Behav Res Ther 2003;41(9):1081–92.doi:10.1016/S0005-7967(02)00244-9
      OpenUrlCrossRefPubMedWeb of Science
    22. ↵
      2. Jensen JD ,
      3. Bernat JK ,
      4. Davis LA , et al
      . Dispositional Cancer worry: convergent, divergent, and predictive validity of existing scales. J Psychosoc Oncol 2010;28(5):470–89.doi:10.1080/07347332.2010.498459
      OpenUrlCrossRefPubMed
    23. ↵
      2. Bradley MM ,
      3. Lang PJ
      . Measuring emotion: the self-assessment manikin and the Semantic differential. J Behav Ther Exp Psychiatry 1994;25(1):49–59.doi:10.1016/0005-7916(94)90063-9
      OpenUrlCrossRefPubMedWeb of Science
    24. ↵
      2. Burgess P
      . Integrating the packaging and product experience in food and beverages: a road-map to consumer satisfaction. Woodhead Publishing series in food science, technology and nutrition, 2016.
    25. ↵
      2. Guo Y ,
      3. Logan HL ,
      4. Glueck DH , et al
      . Selecting a sample size for studies with repeated measures. BMC Med Res Methodol 2013;13(1):1.doi:10.1186/1471-2288-13-100
      OpenUrlCrossRefPubMed
    26. ↵
      2. Dang Q ,
      3. Mazumdar S ,
      4. Houck PR
      . Sample size and power calculations based on generalized linear mixed models with correlated binary outcomes. Comput Methods Programs Biomed 2008;91(2):122–7.doi:10.1016/j.cmpb.2008.03.001
      OpenUrlCrossRefPubMed
    27. ↵
      2. Morgan TM ,
      3. Case LD
      . Conservative Sample size determination for repeated measures analysis of Covariance. Ann Biom Biostat 2013;1:1.
      OpenUrl
    28. ↵
      2. Moodie C ,
      3. Mackintosh AM ,
      4. Hastings G
      . Adolescents' response to pictorial warnings on the reverse panel of cigarette packs: a repeat cross-sectional study. Tob Control 2015;24(e1):e93–e97.doi:10.1136/tobaccocontrol-2013-050999
      OpenUrlAbstract/FREE Full Text
    29. ↵
      2. Abdalla AM ,
      3. Saeed AA
      . Effect of tobacco advertisements on smoking habits among adolescents in Saudi Arabia.

    Footnotes

    • Contributors NFB designed the study and wrote the first draft of the manuscript. NFB and KMG analysed the data. AKTA, HMSA, and SAJA managed the data collection, MHB and NFB designed the web applications. All authors contributed to and have approved the final manuscript.

    • Funding This research was funded by the Deanship of Scientific Research at Saudi Electronic University.

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.