Study design and sample

The goal of 2 Smart 2 Smoke was to reduce the psychosocial risk of smoking among elementary students by changing their intentions to smoke in the future. Secondarily, the programme was designed to change normative expectations, functional meanings, expected outcomes, and attitudes about smoking.

A pretest–posttest design with two sequential groups was used to evaluate the effectiveness of the “2 Smart 2 Smoke” productions. This design is outlined below, where group 1 = intervention, group 2 = delayed intervention, 0₁ and 0₂ represent the pretest and posttest surveys, respectively, and X represent the theatre production:

 0₁ X 0₂

       0₁ X 0₂.

Seventeen elementary schools in the Twin Cities metropolitan area participated in the evaluation of 2 Smart 2 Smoke. These schools were a random sample of 160 schools which staged the play during the 1997–1998 school year in Minnesota. The assignment of the date of the play to a given school was a random process.

Of the 17 schools, because of different grade level configurations, 16 schools provided pretest data from students in grades 1–3, and 15 (of the 16) schools provided posttest data from students in grades 1–3. Fourteen schools provided pretest and posttest data from students in grades 4–6.

Schools that participated in the play and the pretest and posttest surveys before December 1997 were designated as intervention schools. Schools which participated in the play and the surveys after December 1997 were designated as delayed intervention (control) schools. There were eight intervention and eight control schools for grades 1–3, and seven intervention and seven control schools for grades 4–6.

Data were collected by teachers of the students before the play’s performance and after the completion of the postplay activities. The time interval between surveys was one to two months. The surveys from students were anonymous and confidential.

Students who reported that they did not see the play were eliminated from posttest analyses. Students did not see the play if they were absent from school or if part of a special programme that prevented them or their class from attending. Likewise kindergarten students were eliminated from analyses, as reliable data would be difficult to obtain from such young children. About 71% of the sample were white, 7.5% Asian American, 6.0% African American, 1.6% Hispanic, 4.3% American Indian, and 9.6% from other racial/ethnic groups.

The total sample size was 2524 for the pretest survey, grades 1–3; 1833 for the posttest survey, grades 1–3; 1730 for the pretest survey, grades 4–6; 1355 for the posttest survey, grades 4–6. There were no significant differences in gender or racial/ethnic group composition between the pretest and posttest samples.

The 2 Smart 2 Smokeintervention

The intervention consisted of two 30-minute plays that were shown at each school with actors from the National Theater for Children, classroom activities, and take-home materials for parents, designed for kindergarten through third grade (K–3), and fourth through sixth grade (4–6) students, respectively.

The K–3 play was an adaptation of the Three Little Pigs. The Big Bad Wolf started to smoke, because it “would make him look cool” and because “he thought that everyone smoked”. As a result, he was unable to blow down the three little pigs’ houses. The three little pigs offered a range of advice to the wolf, from firing him from his job as the Big Bad Wolf, to telling him that his clothes stink, to sympathising with him, to explaining the consequences of smoking, to helping him to quit. Students were recruited from the audience to “replace” the Big Bad Wolf, and were able to blow down the pigs’ house, because they were non-smokers.

The 4–6 grade play was about Planet Tramsos. The chairman of the board of Snarlboro cigarettes on Earth is looking for a new market, as children on Earth are not starting to smoke as much as they did in the past. There is a television special about Planet Tramsos, which describes the people of Tramsos, and their “addictive” personalities (people from Tramsos have a hard time stopping whatever they start). The chairman of the board decides it is a ideal opportunity and flies his rocket ship to Tramsos. One of the Tramsonian teenagers, Eerf Ekoms, overhears the chairman’s plans and tries to figure out how to save the planet. She does this by getting all the young people of the planet to say “no”, a phrase that they become “addicted” to. The executive tells them, as he heads back to Earth, that they are all just backward, and Eerf agrees, explaining that Tramsos is “So smart” backwards and Eerf Ekoms is “Smoke free” backwards.

After the plays, the teachers worked with the students on three or four follow-up activities, as classroom time permitted, and materials were sent home with the children to work on with their parents.

Measures

Two questionnaires were developed for the two age groups. The questionnaires were designed based on previous work with these age groups, such as the Child and Adolescent Trial for Cardiovascular Health,14 and assessed intentions to smoke, and factors that were associated with smoking onset as well as factors targeted in the theatre production.1 The questionnaire for grades 1–3 students had 17 questions and measured intentions to smoke in the future, normative expectations (perceptions of the number of adults and teenagers who smoke), the expected outcomes of smoking, attitudes and knowledge about smoking, the functional meanings of smoking (the meanings young people place on smoking), and self-reported behaviour.

The questionnaire for grades 4–6 students had 46 questions and measured intentions to smoke in the future, normative expectations, the outcomes of smoking, attitudes, functional meanings, self-efficacy to refuse a cigarette (confidence to be able to say “no”), and self-reported behaviour. The posttest surveys also included items assessing the play. The questionnaires were piloted with same-age students before implementation. The surveys are available from the first author by request, and abbreviated questions are shown in tables1 and 2.

Analysis

To evaluate the impact of 2 Smart 2 Smoke, two sets of analyses were conducted. These two sets of analyses were conducted separately for students in grades 1–3 and 4–6.

The first set of analyses tested for differences between the pretest and posttest surveys among all students in 15 schools for grades 1–3 and 14 schools for grades 4–6. These analyses used repeated measures analysis of variance, implemented using SAS PROC MIXED and specifying school as a nested random effect, with each variable analysed separately as a dependent measure.15 16 Threats to the validity of a one-group, pretest–posttest design include history, statistical regression, maturation, and testing.17

The second set of analyses tested for differences between the intervention and control (delayed intervention) schools, using analysis of variance, implemented using SAS PROC MIXED and, again, specifying school as a nested random effect, with each variable analysed separately. There were eight intervention and eight control schools for the grades 1–3 comparisons, and seven intervention schools and seven controls schools for the grades 4–6 comparisons. As the intervention group (group 1) received the play and pretest and posttest surveys between October and December 1997, and the control group (group 2) received the play between December 1997 and March 1998, we were able to use group 2’s pretest to provide data to determine if history, statistical regression, and maturation were alternative explanations for the findings from the pretest–posttest analyses.

First, group 2’s pretest data were compared with group 1’s pretest data to test for equivalence of the groups. If equivalent, then history, statistical regression, and maturation would be unlikely explanations for the findings. Second, group 2’s pretest data were compared with group 1’s posttest data for a test of the effect of the play using group 2 as a “control group” in an intervention-control design. This combination of analysis strategies provided a better test of the effects of the play than the pretest–posttest analyses only, because the intervention-control group analyses can serve to rule out most of the threats to the interpretation of the pretest–posttest analyses and provide additional evidence of which factors were affected by the theatre production.

Differences are considered significant when p<0.05 and approaching significance when p<0.10.

Results

Overall, students very much enjoyed the plays, as indicated by 91.4% of the students in grades 1–3, and 92.7% in grades 4–6. The students also understood the primary messages of the plays, with 97.3% of the students in grades 1–3 identifying that the Big Bad Wolf was a smoker and could not blow hard, and 96% of the students in grades 4–6 reporting that the cigarette company executive visited Tramsos to find new cigarette customers.

The students also participated in classroom or home-based activities which followed the play. Among students in grades 1–3, 49% reported having received a letter from the Big Bad Wolf, 60% learned how their heart feels after exercise, 19% created a poster on the reasons not to start smoking, and 28% reported having done a homework assignment with their parent(s) about smoking. Among students in grades 4–6, 76% reported that they learned about things that people could do instead of smoking, 56% practised ways to say “no” to offers of cigarettes, 38% created “no smoking” advertisements, and 27% completed a homework assignment with their parent(s) about smoking.

GRADES 4–6: PRETEST–POSTTEST ANALYSES

Forty-six questions were examined from the survey. Overall, 16 showed significant change, and two additional questions approached significance. The abbreviated questions, constructs measured, mean values, confidence intervals, and probability values for each question for the pretest–posttest analyses are shown in table1.

After the theatre production, students in grades 4–6 were more likely than before the production to agree that they would never smoke a cigarette, that there are not as many smokers as there used to be, that smoking cigarettes will kill you, that nicotine is more addictive than heroin, that smoking costs a lot of money, that if you smoke you will get in trouble at school, that people don’t smoke cigarettes because they are worried about their health, that smoking cigarettes makes your teeth yellow, that smoking is dumb, and that billboards try to convince you to try smoking cigarettes. They were also significantly more likely to say that there were five (out of eight) very important reasons for them not to smoke.

GRADES 4–6: INTERVENTION VERSUS CONTROL ANALYSES

The equivalence of intervention and control schools was examined by comparing pretest data from intervention and control schools. Of the 46 questions, only two were significantly different from each other in the pretest survey, the number expected by chance alone. On examination, the answers to these two questions suggested a bias in favour of the control schools. The control schools also had a higher percentage of boys (52% v 47%). For the grades 4–6 analyses, the two sets of schools can be considered equivalent.

The same 46 questions were examined for intervention-control differences. Eight questions were found to be significantly different between the two groups, and two approached significance. These are noted by the dagger in table 1. Nine of these 10 questions were those that also were statistically significant or approaching significance in the pretest–posttest analyses.

Students in the intervention schools were more likely than those in the control schools to agree that they will never smoke a cigarette (p<0.10), that there are not as many smokers as there used to be, that nicotine is more addictive than heroin, that smoking costs a lot of money, that smoking cigarettes makes your teeth yellow, and that being a non-smoker is cool.

The intervention students were more likely to disagree with the statement that smoking cigarettes makes you look cool (p<0.07). They were also more likely to say that there were three very important reasons for them not to smoke.

GRADES 1–3: PRETEST POSTTEST ANALYSES

Seventeen questions were examined from the survey. Overall, five showed significant change, and one additionally approached significance. The pretest–posttest analyses are shown in table2.

After the production, the students in grades 1–3 were more likely than before the production to disagree that a lot of adults smoke cigarettes, that a lot of teenagers smoke cigarettes, and that smoking cigarettes makes you look cool. After the production, students were more likely to agree that they will never smoke a cigarette, that smoking cigarettes make your clothes stink, and that people who make cigarettes don’t care if cigarettes make you ill.

GRADES 1–3: INTERVENTION VERSUS CONTROL ANALYSES

The equivalence of intervention and control schools was examined by comparing pretest data from both sets of schools. Of the 17 questions, none was significantly different from the others in the pretest survey. As with the students in grades 4–6, there were more boys in the control schools than in the intervention schools. Again, the intervention and control schools can be considered equivalent.

The same 17 questions were examined for intervention-control differences. Three questions were found to be significantly different between the two groups. These are indicated by the daggers in table 2. All three of these questions had been found to show significant change in the pretest–posttest analyses.

Students in the intervention schools were significantly more likely than those in the control schools to disagree with the statements that a lot of adults smoke cigarettes and a lot of teenagers smoke cigarettes. After the performance, intervention students were also more likely to agree that smoking cigarettes make your clothes stink.

Conclusions and discussion

The theatre production 2 Smart 2 Smokeand accompanying activities had a significant impact on psychosocial risk factors for smoking among students in grades 1–3 and grades 4–6. Among both sets of students, the percentage of students who said that they would “never smoke a cigarette” increased by 10% following the play in the pretest–posttest analyses. This was statistically significant for both groups in the pretest–posttest analyses, and approaching significance (p<0.10) for grades 4–6, and not significant for grades 1–3 in the intervention-control analyses. However, students in grades 1–3 in the control schools were more likely than students in the intervention schools to have said in the pretest survey that they would never smoke (73% v 69%, respectively), so these differences made it more difficult to detect a significant difference when pretest (control) responses (73%) were compared with posttest (intervention) responses (77%). Thus, it is quite likely that the theatre production did affect intentions to not smoke in the future for all of the students. This is particularly notable, as intentions are among the strongest predictors of subsequent smoking onset.1

Students in grades 4–6 also had significantly more negative attitudes and expected outcomes of smoking, and more reasons not to smoke as a result of the production. Students in grades 1–3 changed their normative expectations concerning the number of adults and teenagers who smoke. In addition, the younger students learned that smoking makes your clothes stink. These psychosocial factors form the basis of beliefs and perceptions about smoking that also increase their risk of initiation.1

The changes noted are short term, measured by posttest surveys that were implemented a short time after the play and follow-up activities. Clearly, longer term evaluations are needed to determine the impact of the theatre production on smoking initiation. The short term results are still important, as psychosocial risk factors of smoking were affected, and because changes in behavioural intentions are not easily obtained with this age group. For example, in the Child and Adolescent Trial for Cardiovascular Health, a four-week smoking prevention programme with fifth grade students (aged 10–11 years) and their parents was not able to detect such effects, even with a sample size of 96 schools.14

Students at these ages generally are not supportive of smoking, and many of the items were overwhelmingly endorsed in the pretest survey. This was particularly notable for the current smoking and self-efficacy items. Ceiling effects may therefore have precluded detecting more behavioural changes and other differences from pretest to posttest and between groups for many of the items, especially as the data were analysed with the school as the unit of analysis. More sensitive measures of psychosocial factors in this age group may need to be developed to detect the effects of preventive interventions.

A research design in which both pretest and posttest data could be employed in the intervention-control analyses would have been optimal, but was not logistically possible, as schools participated so as to receive the play. Assigning the date of the play to a given school was a random process, and the adequacy of this randomisation procedure was supported by the equivalent responses between groups in the pretest survey, which also adds strength to the assignment of half of the schools as matching controls. The overlap of significant outcomes from the two sets of analyses makes the results more compelling and strengthens the validity of the findings. The two sets of analyses, therefore, provide converging evidence of which factors were affected by 2 Smart 2 Smoke. However, the research design could not control for familiarity with the survey among intervention group students, a remaining threat to the interpretation of the findings.

The changes observed can be mostly attributed to the play, as the classroom activities were completed by less than half the students, and only about a quarter of the students completed the homework assignment with their parent(s). Thus the play may have been even more potent had the accompanying activities been fully implemented. The results reported are short term changes, but important ones for these age groups, because they suggest an effect on important risk factors for subsequent smoking onset, such as intentions and normative expectations, which may counterbalance the impact of attractive cigarette advertising and promotional activities. Thus, live theatre such as 2 Smart 2 Smoke, which was able to attract the attention of younger children and pre-teens, is a promising strategy warranting further research as part of community-wide efforts to promote non-smoking among children.