Article Text

Download PDFPDF
Crowdsourcing data collection of the retail tobacco environment: case study comparing data from crowdsourced workers to trained data collectors
  1. Annice E Kim1,
  2. Alicea J Lieberman2,
  3. Daniel Dench1
  1. 1Public Health Policy Research Program, RTI International, Research Triangle Park, North Carolina, USA
  2. 2Community Health Promotion Research Program, RTI International, Research Triangle Park, North Carolina, USA
  1. Correspondence to Dr Annice Kim, RTI International, 3040 Cornwallis Road, P.O. Box 12194, Research Triangle Park, NC 27709, USA; akim{at}


Objective To assess whether crowdsourcing is a viable option for conducting surveillance of point of sale (POS) tobacco marketing practices.

Methods We posted jobs to an online crowdsourcing platform to audit 194 Florida licensed tobacco retailers over a 3-week period. During the same period, trained data collectors conducted audits at the same retail locations. Data were collected on cigarette advertising, cigarette promotions and product availability (electronic cigarettes, snus and dissolvables). We compared data collected by crowdsourced workers and trained staff and computed frequencies, percent agreement and inter-rater reliability. Photographs of e-cigarettes and exterior cigarette advertisements submitted by crowdsourced workers were used to validate responses.

Results Inter-rater reliability between crowdsourced and trained data collectors was moderate to high for coding exterior cigarette advertisements, product availability and some tobacco promotions, but poor to fair when coding presence of sales and interior cigarette advertisements. Photos submitted by crowdsourced workers confirmed e-cigarette availability that was missed by trained data collectors in three stores.

Conclusions Crowdsourcing may be a promising form of data collection for some POS tobacco measures. Future studies should examine the cost-effectiveness of crowdsourcing compared with traditional trained data collectors and assess which POS measures are most amenable to crowdsourcing.

  • Advertising and Promotion
  • Surveillance and Monitoring
  • Environment

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.