Research Article
Reliability of a Retail Food Store Survey and Development of an Accompanying Retail Scoring System to Communicate Survey Findings and Identify Vendors for Healthful Food and Marketing Initiatives

https://doi.org/10.1016/j.jneb.2011.03.003Get rights and content

Abstract

Objective

To develop a retail grocery instrument with weighted scoring to be used as an indicator of the food environment.

Participants/Setting

Twenty six retail food stores in low-income areas in California.

Intervention

Observational.

Main Outcome Measure(s)

Inter-rater reliability for grocery store survey instrument. Description of store scoring methodology weighted to emphasize availability of healthful food.

Analysis

Type A intra-class correlation coefficients (ICC) with absolute agreement definition or a κ test for measures using ranges as categories.

Results

Measures of availability and price of fruits and vegetables performed well in reliability testing (κ = 0.681-0.800). Items for vegetable quality were better than for fruit (ICC 0.708 vs 0.528). Kappa scores indicated low to moderate agreement (0.372-0.674) on external store marketing measures and higher scores for internal store marketing. “Next to” the checkout counter was more reliable than “within 6 feet.” Health departments using the store scoring system reported it as the most useful communication of neighborhood findings.

Conclusions and Implications

There was good reliability of the measures among the research pairs. The local store scores can show the need to bring in resources and to provide access to fruits and vegetables and other healthful food.

Introduction

Professionals in the field of public health have become increasingly interested in how community environments support or hinder healthful behaviors, including behaviors related to the food environment,1, 2 physical activity barriers,3, 4, 5, 6, 7, 8 and marketing practices.9, 10, 11 The Centers for Disease Control and Prevention have put forth recommended community strategies and measures to prevent obesity that address access to healthful local food, incentives for food retailers to carry more healthful options, and limits on advertisements of less healthful food.12 Providing incentives for food retailers to offer more healthful food and beverages or to establish new locations in underserved areas is among strategies recommended by the Centers for Disease Control and Prevention.12 Identifying retailers suitable for interventions to increase and/or promote healthful options can present a challenge for community organizations, health departments, and other stakeholders ready to initiate change.

Evidence is increasing to establish the relationship between residential proximity to supermarkets and healthful eating.13, 14 Large chain grocery stores with the greatest variety and lower prices are more available in middle- and higher-income neighborhoods.14, 15, 16, 17, 18, 19 It has been well documented in low-income areas that there are fewer large food stores and more convenience and small markets.19 Point-of-sale marketing and product placement cannot be ignored as critical influences on consumer behavior.20, 21, 22 Data on the quality and types of retail food stores in low-income neighborhoods can provide crucial information that exposes inequities such as food deserts, the lack of access to healthful food by neighborhood residents, and level of egregious marketing of less-healthful food and beverages or to highlight vendors that contribute to a healthy neighborhood. Store-level data can provide useful information for intervention development or promotion of stores offering healthful choices and marketing practices.

Communities of Excellence in Nutrition, Physical Activity, and Obesity Prevention (CX3) was designed by the California Department of Public Health, Network for a Healthy California (Network) as a framework of healthy community indicators. Local health departments implemented CX3 to involve stakeholders and examine environmental conditions present in low-income neighborhoods that affect obesity. One of the goals of CX3 is for data from surveys of food retailers to advance local action through neighborhood improvement projects, redevelopment efforts, coalitions, nutrition educators, health advocates, and local governments. Documenting the availability of healthful food and marketing practices in retail stores is valuable for understanding the conditions that could positively or negatively influence consumer choices.

The Network is a social marketing initiative for nutrition education services to assist current and potential participants in the Supplemental Nutrition Assistance Program (SNAP), formerly known as Food Stamps. The Network serves California’s estimated 7.1 million parents and children from households with incomes below 185% of the federal poverty level through a wide range of local assistance contracts.23 Communities of Excellence in Nutrition, Physical Activity, and Obesity Prevention offers Network-eligible communities the opportunity to connect neighborhood environments to nutrition education activities. The presence or absence of particular conditions provides a way to target nutrition education delivered in conjunction with the Network Fruit, Vegetable, and Physical Activity Retail Program (Network Retail Program) or other consumer-based activities, especially when paired with community-based participatory research. Communities of Excellence in Nutrition, Physical Activity, and Obesity Prevention was designed to examine environments where people live, work, learn and play as a means of targeting nutrition education efforts and offering expanded methods that lead to community action.

Communities of Excellence in Nutrition, Physical Activity, and Obesity Prevention is implemented by local health departments to link residents and community organizations and to guide local planning. Health departments are trained with standardized CX3 methods using Geographic Information System mapping24 and field surveys to examine food sources including stores, fast food, food banks, emergency food outlets, and tracking methods to gather alternative sources of healthful food such as community gardens, produce stands, or community-supported agriculture in selected low-income neighborhoods. Health department staff members were encouraged to involve community members and youth in the data collection and/or interpretation process, build partnerships, and collaborate with a variety of other organizations and leaders to combine efforts that encourage healthful lifestyles. These partnerships are important to the success of obesity reduction efforts since Supplemental Nutrition Assistance Program Education (SNAP-Ed) resources to affect local policies and environments have been limited in scope.

Section snippets

Store Survey Instrument Reliability Testing

The CX3 Food Availability and Marketing Survey was pilot-tested among other instruments in 2006 by 6 local health departments in 21 low-income neighborhoods that ranged from dense urban to rural, with over 180 stores surveyed. Use in the field by health departments continued in 2007, which generated more refinements to the survey. The store survey was designed to be implemented by groups of varying abilities, specifically, youth or community members, making a reliable tool critical for moving

Reliability of the Store Survey

Results of inter-rater reliability between pairs of Network research staff (Table 2) showed findings that supported use of the majority of the measures. Measures showed generally better results for exterior ads for healthful products compared to ads for less healthful products. Overall methods to capture ads on exterior doors and windows showed low to moderate results, with a range of κ from 0.372 to 0.674. Methods to capture other exterior promotions or conditions performed well (ICC 0.857).

Reliability

Other reliable instruments to gather the availability of foods in stores have been developed,30, 31, 32, 33 however, many of them do not attempt to evaluate marketing practices. Despite the use of measures from tobacco studies that capture ads and promotions in the retail environment34 to inform development of CX3 survey items, determining size and categorization of ads for healthful or less healthful food and beverages appears to be difficult. Areas that are cluttered and contain visual

Implications for Research and Practice

The CX3 retail scoring system can be a catalyst for change. It translates store survey findings into a format that empowers community members and other stakeholders to foster meaningful and action-oriented approaches for environmental change. All participating health departments have reported current action or plans to start retail interventions. As a group, Network health departments that have participated in CX3 reported they have planned or started a Network Retail Program (16 of 22 health

Statement of Potential Conflict of Interest

This research project is funded in part by the United States Department of Agriculture (USDA) SNAP-Ed through a contract with the California Department of Public Health’s Network for a Healthy California (author VQ) administered by the Public Health Institute (authors AG and SS) for the data collection, analysis, interpretation of results, and manuscript preparation. The opinions expressed are those of the authors and do not necessarily represent the views or recommendations of their respective

Funding/Support

This study was funded by the USDA SNAP-Ed through a contract with the California Department of Public Health’s Network for a Healthy California administered by the Public Health Institute.

Acknowledgments

CX3 was produced by the California Department of Public Health’s Network for a Healthy California with funding from the USDA SNAP (formerly the Food Stamp Program). The authors thank the local health departments and their staff who contributed to the development of CX3 and the scoring methodology. The authors also thank Ellen Feighery, RN, MS, for contributions to CX3 and Jennifer Gregson, PhD; Barbara MkNelly, MS; Amanda Linares, MS; and Melissa McGuire, MPH, RD for data collection to test

References (40)

  • J.F. Sallis et al.

    Assessing perceived physical environment variables that may influence physical activity

    Res Q Exerc Sport

    (1997)
  • J.F. Sallis et al.

    Physical Activity and Behavioral Medicine

    (1998)
  • J.F. Sallis et al.

    San Diego surveyed for heart healthy foods and exercise facilities

    Public Health Rep

    (1986)
  • H. Frumkin

    Urban sprawl and public health

    Public Health Rep

    (2002)
  • H. Frumkin et al.

    Urban Sprawl and Public Health: Designing, Planning and Building for Healthy Communities

    (2004)
  • D. Bunn

    McDonald’s ads to focus on minorities

    Rocky Mountain News

    (December 28, 1997)
  • A. Perlik

    Minority Matters: survival of the fittest means rising to the opportunities of an evolving foodservice market

    Restaurants and Instutions

    (March 15, 2005)
  • K.D. Brownell et al.

    Food Fight: the Inside Story of the Fast Food Industry, America’s Obesity Crisis, and What We Can Do About It

    (2003)
  • L.K. Khan et al.

    Recommended community strategies and measurements to prevent obesity in the United States

    MMWR Recomm Rep

    (2009)
  • N.I. Larson et al.

    Neighborhood environments: disparities in access to healthy foods in the US

    Am J Prev Med

    (2009)
  • Cited by (42)

    • Development and Interrater Reliability of a Street Food Stand Assessment Tool

      2021, Journal of Nutrition Education and Behavior
      Citation Excerpt :

      Documenting whether an item was available was easier than documenting all the varieties of that item, especially among SFS that continuously sold items. This study's scores were similar to those of other interrater reliability studies assessing food and beverage availability.3,6,34,35 Using a retail store tool in Detroit, MI, Izumi et al35 found that most items had almost perfect reliability, whereas only 2% of items had fair to moderate reliability (0.21–0.40).

    • Development and Testing of the Healthy Food Pantry Assessment Tool

      2019, Journal of Nutrition Education and Behavior
      Citation Excerpt :

      To the authors' knowledge, this study led to the creation of a scored food pantry e-scan, the first to be tested across multiple states. The HFPAT may be applicable to a wider audience than might other tools tested in only 1 state.33,34,48 A resource guide was developed to accompany the HFPAT to increase the usability of the survey in FPE interventions.

    • Changes in Consumer Purchases in Stores Participating in an Obesity Prevention Initiative

      2018, American Journal of Preventive Medicine
      Citation Excerpt :

      The interview included structured response and open-ended short answer questions regarding perceived financial impacts, benefits, and challenges of the interventions. Changes in store environments were assessed by comparing baseline and follow-up CX3 scores based on rubrics developed by the California Department of Public Health.24 For Zone 1, which did not use the CX3 tool, change in food availability from baseline to follow-up was assessed by comparing checklists of food and beverage items that were part of their locally developed tool.

    View all citing articles on Scopus

    STATEMENT OF POTENTIAL CONFLICT OF INTEREST AND FUNDING/SUPPORT: See page S111.

    View full text