Cigarette smoking among workers is a significant cause of death and disability in the USA.¹ Several studies suggest that service and blue collar workers are at greater risk for smoking than white collar workers,^1–3 and worksite interventions are effective in reducing smoking among service and blue collar workers.^4–6 However, worksite interventions have largely ignored adolescents, despite the fact that 89% of adults begin smoking before the age of 18 years⁷ and adolescents comprise a significant part of the US workforce.

National studies indicate that employment and working long hours are associated with increased risk of smoking among adolescent workers,^8–11 and adolescents who work long hours are twice as likely to smoke than those who work fewer hours.¹² One study indicated that a third of adolescent smokers reported that they first started smoking regularly at work.¹³ Eighty per cent of teens work at least some point during their high school career,^14–16 with 78% working in retail or other service industry.¹⁷ Intervening with adolescents working in the service industry may have a significant impact on disease and disability caused by cigarette smoking.

No intervention studies have published data on the feasibility of a tobacco control intervention in worksites for employed adolescents. Feasibility testing in the retail service setting is an important step to developing strategies to reduce the risk and prevalence of smoking among working youth. However, the worksite setting presents unique challenges to collecting evaluation data on smoking behaviour. While adult workers are often employed full time,¹⁸ adolescents primarily work part time and work variable hours throughout the school year and summer.¹⁹ Adult worksite studies report participation rates from 2–88% with a median of 47%.⁴ Studies among adolescents in school and community settings have also found it difficult to recruit youth to participate in tobacco control interventions.⁷ Evaluating an intervention in the service retail setting requires a comprehensive system for monitoring teen employment rates and changes in smoking prevalence.

This paper is the first to describe the feasibility and challenges to implementing evaluation methods for a worksite based tobacco control intervention designed for employed adolescents. Indicators of feasibility include employment stability, response rates to multiple surveys, and self reported 30 day smoking rates. Testing the feasibility of evaluation methods provides valuable data for researchers who seek to intervene with youth in worksites.

METHODS

SMART study setting

Data for this study were collected as part of the SMART Teens Against the Risks of Tobacco Study. SMART, which began in 1997, is the first phase II methods development study designed to test the efficacy of a tobacco prevention and cessation worksite programme for employed adolescents. As specified by Greenwald and Cullen²⁰ and Flay,²¹ phase II methods development studies provide an important step in developing new approaches to health promotion, including exploratory research and pilot testing of new approaches. Details of the primary results, the feasibility of the behavioural intervention, and formative research are reported elsewhere.^22,23

Grocery stores in Massachusetts that were part of a single chain were selected to participate in the study. Grocery stores are the second largest retail employer of teens; during the school year, 13.6% of males and 9.9% of female workers aged 15–17 years are employed in this industry.¹⁹ Along with the corporate headquarter upper management of the company, research staff identified 12 of 52 grocery stores located within a 45 mile radius of Boston, Massachusetts. To be eligible for the study, stores were required to employ at least 40 teens between the ages of 15–18 years at the time of randomisation. Twelve stores were randomised to intervention and control stores (fig 1). Three were later dropped from the study because of excessive turnover of teen employees or store closing. SMART was completed in four intervention and five control stores over a 12 month period (table 1). Intervention activities included interactive (open houses, games, contests, and demonstrations) and non-interactive (bulletin board displays with educational messages, table tents, and home mailings) activities.

Data collection procedures

Study components were approved by the institutional review board at the Dana Farber Cancer Institute. Cross sectional data on smoking were collected during the randomised controlled trial at seven data points using the baseline, brief prevalence, and final surveys. Teens aged 15–18, employed full or part time, were eligible to voluntarily participate in the study. Final survey data also included 19–20 year olds who previously participated in one of the surveys, based on their age eligibility at a prior data collection point. Teens who were away at school or were on long term sick leave were ineligible for the study.

Research staff and adult store contacts were trained to administer the surveys following standard procedures. Teen advisors were selected by store managers and research staff to help facilitate survey administration. Teen advisors were provided $10 to $50 in cash for their assistance. Survey participants were recruited using fliers posted in the break room or placed on pay cheques and by word of mouth. Teens were scheduled to take the survey in groups or as individuals on work time. Written consent was provided by each teen before survey administration. Follow up data collection was conducted in every store. Individual respondent and store level incentives during survey administration included store, mall, and restaurant gift certificates, $5 in cash, and pizza parties to compensate them for their time.

Baseline survey

The baseline survey assessed the prevalence of tobacco use and factors influencing tobacco use among employed youth. This 36 item survey was administered to teens over a two month period before the delivery of the intervention.

Brief prevalence surveys

Brief prevalence surveys were designed to track new hires into the study, identify and track eligible teens who did not previously take the baseline survey, and assess 30 day smoking prevalence rates. These surveys were developed because data from the formative research suggested that teen employment rates fluctuated over time, and data from other studies and discussions with our scientific advisory board suggested that there might be high levels of variability in smoking rates among older adolescents.^24,25 This brief survey included two questions that asked teens to report the number of days smoked and the usual number of cigarettes smoked in a day in the past 30 days. These data were collected on a small index sized card with a consent form attached. The consent form was detached from the brief survey to disassociate the survey from the name on the consent form. Five brief prevalence administrations were conducted bimonthly following the baseline survey and before the administration of the final survey.

Final survey

The final survey assessed the prevalence of tobacco use, factors influencing tobacco use, and collected data on programme recognition and participation. This 45 item survey was administered over a two month period.

Teen tracking system

Data from the formative research suggested that teen employment was unstable, and thus at the onset of the study, we recognised that tracking employment stability was critical. Staff negotiated with the company’s upper management to obtain a centralised master list of employed teens. This “master list” provided multiple indicators to assess employment stability and eligibility and included date of hire, total hours worked, full time/part time status, store location, date of termination, pay cheque periods, date of birth, and sex. However, when the company was bought out, this list was unavailable. Individual lists were therefore collected from managers at the store level to monitor youth employment patterns and identify eligible teens. These lists contained only the name and date of birth or age of the teen. Research staff collected employee lists in coordination with the administration of the brief prevalence surveys. Managers were asked two weeks before each brief prevalence survey administration to identify teens on the list who were no longer working at each store or were ineligible to take the survey. Lists were combined by name and store in the teen tracking system to identify continuing, terminated, and new employees. Response rates and turnover rates were computed using the teen tracking system.

Measures

Survey measures included sociodemographic characteristics and an assessment of 30 day smoking prevalence. Sociodemographic characteristics were collected at baseline and final and included the measures age, sex, race/ethnicity, hours worked per week, and whether or not teens sold cigarettes at their job. Measures of smoking behaviour were collected on baseline, brief prevalence, and final surveys. Teens were asked to indicate, “How many days in the last month did you smoke?”, and “In the last month, how many cigarettes did you smoke per day?”. Current smokers were those who had smoked at least once in the last 30 days.

To obtain the mean smoking prevalence by store, smoking data from all seven surveys were combined and a mean smoking prevalence was computed for each store at each survey. Survey specific prevalence rates were averaged for each store. Using this combined data set, we linked the multiple responses for the teens and computed the number of completed surveys for each teen.

For teens who completed at least two surveys, a smoking trend variable was computed with four categories: no change non-smokers, no change smokers, increasing smokers, and decreasing smokers. No change non-smokers were teens who reported no smoking at either survey assessment. No change smokers reported smoking at both time 1 and time 2, but did not report an increase in the number of cigarettes smoked per day. Increasing smokers reported an increase in the number cigarettes smoked per day from time 1 to time 2. Decreasing smokers were teens who reported a decrease in the number of cigarettes smoked per day from time 1 to time 2. For each teen smoking at each time point, a slope coefficient for the number of cigarettes per day on the days smoked was computed. This was computed as a least squares estimate of change in number of cigarettes per 60 day interval.

The employee turnover rate was assessed using data collected from the teen tracking system. This rate was calculated by dividing the number of teens terminated at each administration by the total eligible number of teens currently employed at each administration. Turnover rates following the baseline were calculated at two month intervals to correspond to the brief prevalence surveys.

Analysis

Frequency distributions and descriptive statistics of teen sociodemographic characteristics were computed from the baseline survey. Teens in intervention and control stores were compared with regard to these characteristics using general linear modelling controlling for the nesting of teens within stores. We fitted a mixed model analysis for each characteristic with intervention condition as a fixed effect and grocery store as a random effect. For characteristics measured categorically, we computed a logistic regression analysis and used iteratively re-weighted likelihoods to fit the logistic regression model.²⁶ For continuously scaled measures, mixed model analysis of variance was used.

For changes in smoking prevalence, we cross tabulated our four category smoking trend measures and computed a χ² test of homogeneity for intervention and control stores. For these tests we did not control for the clustering of teens in grocery stores due to small cell sizes. The surveys and store specific smoking prevalence with the corresponding response rates were linked to investigate the correlation between smoking prevalence and response rate. A Pearson product moment correlation coefficient was computed. The personal computer version of SAS was used for all analyses.²⁷

RESULTS

Smoking and demographics of the study population

At baseline, there were no differences between intervention and control sites on sociodemographic factors. Fifty eight per cent of participants were aged 15–16, 45% female, and 66% were white, non-Hispanic. Twenty eight per cent worked more than 20 hours per week during the school year, 73% sold cigarettes in their store, and 28% of study participants were current smokers at baseline.

Employment stability among teens

Table 2 presents data on employment stability during the seven administrations and those meeting the eligibility criteria for the cross sectional surveys. A total of 824 eligible teens were employed at various times during the study. Eligible teens at any one survey administration ranged from 384–459. New hire rates ranged from 16–29% with rates highest in the fall and late summer. Turnover rates ranged from 14–32%. Turnover rates were highest in late summer.

Response rates for each administration and by intervention and control stores

As noted in table 2, overall response rates ranged from 43–77%. Other than the 43% response rate for interim 1, none of the response rates fell below 59%. Response rates by store were variable and ranged from 64–98% for the baseline, 0–100% for the brief prevalence, and 43–100% for the final survey (data not shown).

Table 3 reports the range and mean response rates by intervention and control stores. Mean response rates ranged from 41–91%. The mean response rate for the control stores, 71%, was somewhat greater than the rate in the intervention stores, 59%, p  =  0.06. Two control stores and one intervention site consistently yielded response rates greater than 50% for all administrations.

30 day prevalence rates

Table 3 reports the range and mean of 30 day smoking prevalence by intervention and control stores. The mean smoking prevalence by store ranged from 9–32%. Two intervention stores consistently reported current smoking at or above 18%, while two control stores consistently reported current smoking below 18%. Overall, there was a negative correlation between smoking prevalence and response rate by survey and store (–0.029, p  =  0.03).

Multiple survey assessments

Eighty seven per cent of teens completed at least one assessment. Table 4 presents the distribution of survey assessments completed by teen respondents only. Assessments were not necessarily consecutive. Among those who completed at least two assessments, there were no significant differences between intervention and control stores on the number of surveys completed (56.4% compared to 60.1%, p  =  0.58).

Changes in 30 day prevalence rates among those with two data points

We analysed trends in the frequency of smoking for teens who completed two or more surveys. The control stores reported a significantly greater percentage of no change non-smokers than the intervention stores while the intervention stores reported a significantly greater percentage of no change smokers and decreasing smokers (p  =  0.03) (table 5). When no change non-smokers were excluded from the analysis, there were no significant differences between intervention and control groups (p  =  0.91).

DISCUSSION

This paper is the first to report on the feasibility of implementing evaluation methods for a smoking prevention and cessation intervention among teens in the worksite. Major findings indicate that employment stability and teen eligibility were affected by the percentage of new hires and terminated teens. Survey response rates for the seven administrations ranged between 43–77% with baseline and final response rates at 77% and 72%, respectively. Although cohort data were limited, 58% of teens completed at least two or more surveys, which allowed us to assess changes in prevalence rates. The intervention group reported a greater percentage of no change smokers and decreasing smokers than the control group, although the changes were not significant when non-smokers were removed from the analysis. Current smoking rates were variable and as response rates increased, smoking rates decreased. These data suggest that we were able to implement evaluation methods, and researchers who seek to intervene with teens in worksites need to develop specific strategies to facilitate data collection and monitor employment stability.

Response rates suggest that there were challenges to collecting multiple cross sectional surveys. Research staff intended to collect the brief prevalence surveys monthly. However, following the first administration, it was not feasible to collect monthly data in the fast paced store setting. Despite the change to bimonthly administrations, brief surveys demanded a considerable amount of time from store contacts, research staff, and teen employees. The variable and part time work schedules of teens limited our ability to track teens in the store for survey follow up. Work schedules changed weekly and varied by weekend and after school hours. Data collection was difficult in stores with less staff resources and managerial support and high turnover rates. Stores with the highest response rates (consistently greater than 71%) were those that had enthusiastic and stable managers and a fairly stable teen population.

Future studies may consider identifying characteristics of teens who respond to multiple assessments and examine aspects of organisational culture that may facilitate the collection of evaluation data. Having permanent study staff working at the store site may help alleviate the burden on store staff and strengthen relationships with managers. Furthermore, because of variability in work schedules and turnover rates, researchers may consider following youth outside the worksite to assess smoking behaviour.

Current smoking rates were variable as suggested by other studies.^23,24 Although the data suggest that as response rates increased, smoking decreased, variability may be reflective of seasonal variations in smoking; youth who move in and out of experimentation or spontaneous quitting; smokers who select to respond non-sequentially to cross sectional surveys; or reflective of natural fluctuations in smoking among adolescents. Despite the variability, trend data on no change non-smokers suggest that the intervention may have had an effect on abstinence. Brief surveys can provide researchers with cohort data to tease out factors associated with variability and examine trend data on smokers and non-smokers.

It is not likely that social desirability influenced differences in current smoking reported by the intervention and control sites. The baseline survey was conducted before randomisation, therefore respondents were unaware of the study conditions. Community or store level demographics may explain differences in rates. We did not intend to match stores on demographics in this methods development trial; however, examining these factors in future studies may be important.

Individual and store level incentives were implemented to help increase response rates and compensate teens for their time. At baseline, we entered teens’ names into a raffle and provided gift certificates to the winners (for example, movies, shopping malls, athletic stores). After the first brief prevalence survey, we repeated this drawing and received a 43% response rate. We modified the incentive methods and awarded a pizza party to the stores with the highest response rates for each administration. At the final survey, participants received a $5 cash incentive. Although brief prevalence survey response rates increased gradually, it is not clear from the data if the increases were due to the type or amount of incentives or to the incentives at all. Although it is important to assess the effects of incentives on recruitment and survey responses, we did not design this study to appropriately examine these differences. One study indicated that few studies have examined the effects of monetary incentives and few are designed to examine their effects.²⁸ This study also found that monetary incentives did have a positive effect on adolescents’ response to a mailed survey. Given the adequate resources and staffing, we encourage researchers to develop both store level and individual level incentives and develop a study design which allows for the evaluation of their effects.

The data from the tracking system suggest that there are seasonal variations in employment. National data indicate that among youth aged 15–17 years, teens work on average 17 hours per week during the school months and 23 hours during the summer.¹⁹ Researchers may explore developing interventions during the summer months for teens at worksites. Furthermore, about 40% of youth participate in school-to-work programmes.¹⁹ These programmes may have longer employment tenure and may perhaps provide a cohort of youth to intervene with through multiple channels.

Other industries where teens work may have different managerial structures, different organisational cultures, and therefore provide opportunities to intervene with youth. For example, 28% of teens 14–17 years old are employed in eating and drinking retail setting, and 14–15 year olds have the longest tenure of employment in these settings.¹⁹ Furthermore, grocery store and other industry adult health promotion programmes should integrate smoking cessation as part of a larger health promotion programme and include specific components relevant to teens.

Worksite smoking restriction policies have been implemented to help adults quit smoking. One study indicated that teens who worked in smokefree workplaces were 68% as likely to be smokers as adolescents who worked in a workplace with no smoking restrictions.²⁹ However, workplace bans on smoking referred to indoor smoking policies. We observed teens smoking outside of the stores and perhaps indoor and outdoor workplace smoking bans may potentially influence smoking among adolescents. Additional research is needed to explore worksite policy restrictions on smoking among youth.

There were several limitations to the study. Although we intended to follow a cohort of youth, because of employment stability, cohort analyses were limited. Brief prevalence surveys were limited to two questions, and thus we were unable to assess initiation or quit attempts, except using the baseline and final surveys. Self report data were not biochemically validated in the worksites. The study scope was limited to 15–18 year olds, which did not allow us to collect smoking data on the entire worksite.

In conclusion, despite the numerous challenges and limitations, this phase II methods development study provided data on novel approaches to evaluating a tobacco control intervention in the worksite. The teen tracking system and brief prevalence surveys were critical to monitoring teen employment patterns, collecting data on smoking, and determining responses of individual teens to multiple surveys. Future studies should consider developing a comprehensive system for evaluating an intervention and be prepared to adapt to changes in the service retail sector.