Article Text

Development of policy performance indicators to assess the implementation of protection from exposure to secondhand smoke in China
  1. Xia Wan1,
  2. Frances Stillman2,
  3. Huilin Liu3,
  4. Mark Spires2,
  5. Zhen Dai4,
  6. Stephen Tamplin2,
  7. Daiwei Hu1,
  8. Jonathan M Samet5,
  9. Gonghuan Yang1,3
  1. 1Department of Epidemiology and Statistics, Institute of Basic Medical Sciences at Chinese Academy of Medical Sciences; School of Basic Medicine at Peking Union Medical College, Beijing, China
  2. 2Johns Hopkins Bloomberg School of Public Health, Institute for Global Tobacco Control, Baltimore, Maryland, USA
  3. 3Chinese Center for Disease Control and Prevention, Beijing, China
  4. 4Department of Epidemiology, Peking Union Medical College, Cancer Institute of Chinese Academy of Medical Sciences, Beijing, China
  5. 5Department of Preventive Medicine, University of Southern California, Institute for Global Health, Los Angeles, California, USA
  1. Correspondence to Professor Gonghuan Yang, Department of Epidemiology and Statistics, Institute of Basic Medical Sciences at Chinese Academy of Medical Sciences; School of Basic Medicine at Peking Union Medical College, 5# Dongdan San Tiao, Dongcheng District, Beijing 100005, China; yangghuan{at}vip.sina.com

Abstract

Objective To develop an approach for rapid assessment of tobacco control interventions in China. We examined the correlation between components of the Strength of Tobacco Control (SOTC) index and a proposed rapid evaluation indicator, the Policy Performance Indicator (PPI), which is based on protection of non-smokers from secondhand smoke (SHS). The PPI was used to assess the implementation of policies related to SHS at the provincial/municipal level in China.

Methods Stratified random sampling was used to select five types of organisational and household respondents in two municipalities and five provinces in China (Shanghai and Tianjin, Heilongjiang, Henan, Guangdong, Zhejiang and Jiangxi, respectively). Data collection methods included key informant interviews, observation and intercept surveys (organisations), and a modified Global Adult Tobacco Survey (GATS) questionnaire (households). SOTC scores (SHS policy, capacity and efforts), PPI (no smoking in designated smoke-free places) and mid-term to long-term impact (knowledge, attitude and reduced exposure to SHS) were measured, and correlations among them were calculated.

Results The PPI varied across the seven locations. Shanghai led in the component indicators (at 56.5% for indoor workplaces and 49.1% for indoor public places, respectively), followed by Guangdong, Tianjin and Zhejiang (at 30–35% for these two indicators), and finally, Henan and Jiangxi (at 20–25%). Smoke-free policies were more effectively implemented at indoor workplaces than indoor public places. The PPI correlated well with certain components of the SOTC but not with the long-term indicators.

Conclusions The PPI is useful for evaluating implementation of smoke-free policies. As tobacco control programmes are implemented, the PPI offers an indicator to track success and change strategies, without collecting data for a full SOTC index.

  • Secondhand smoke
  • Environment
  • Low/Middle income country

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

The harmful effects of secondhand smoke (SHS) have been documented for decades. In the 1970s, scientific and public health interest in potentially adverse health effects of SHS expanded.1 Since then, evidence for the health consequences of SHS exposure has accumulated from many studies done in different parts of the world. In 2004, worldwide, 40% of children, 33% of male non-smokers and 35% of female non-smokers were exposed to SHS, resulting in an estimated 600 000 deaths attributable annually to SHS.2 One of the most recent summaries of the evidence, the 2006 report of the US Surgeon General's report, found a scientific consensus on SHS as a cause of premature death and disease.3

Because of the adverse health effects of SHS, Article 8 of the WHO Framework Convention on Tobacco Control (FCTC) makes clear that each party of the FCTC shall, within their jurisdiction, actively promote, adopt and implement effective legislative, executive, administrative and/or other measures, to protect against exposure to tobacco smoke in indoor workplaces, public transport, indoor public places and, as appropriate, other public places.4 In addition, WHO has provided the MPOWER package, which includes six policy measures supported by evidence as effective. Of the six, the letter ‘P’ refers to protection of non-smokers from inhaling SHS.5 Smoking bans, if enforced, provide the needed protection for non-smokers. Many countries have now implemented smoking bans. For example, in 2004 Ireland made history as the first country to implement a comprehensive smoking ban covering indoor workplaces, including restaurants and bars,6 ,7 and Uruguay became the first country in the Americas to go 100% smoke-free by enacting a ban on smoking in all public spaces and workplaces, including bars, restaurants and casinos.8 Scotland and other countries have followed.9

For China, as the FCTC entered into effect in 2006, government entities at all levels and various sectors of the society made significant efforts into providing some protection from exposure to SHS by taking part in many comprehensive interventions at the community, organisational and population levels.10 The central government introduced the Regulation on Comprehensive Ban of Smoking in Medical and Health System across China in 201111 and the Regulation on Further Strengthening Tobacco Control in Schools.12 In March 2011, the 4th plenary session of the 11th National People's Congress explicitly endorsed a ‘comprehensive ban of smoking in public places’, in the adopted outline of the 12th 5-year plan of the People's Republic of China on national economic and social development.13 Various provinces and municipalities took part in the creation of smoke-free hospitals, schools and communities, supported and motivated by the Tobacco-Free Olympics, Smoke-free World Expo, Towards a Smoke-free China project, the Promotion of Smoke-free Environment project, and subsidies provided by the central government for tobacco control. Interventions have become more sophisticated, covering a wider range of the population. Conventional education and campaigns, and comprehensive, multidisciplinary approaches are used to combine policy advocacy, social advocacy and population interventions, with varying results.

Evaluation of this comprehensive intervention for tobacco control policy is essential to improve tobacco control efforts, but the selection of effective evaluation strategies and indicators is extremely challenging. For example, the World Bank developed a tobacco control scale for measuring country tobacco control activity,14 and WHO developed an index and scale to assess the policy text for measuring existing policies and compliance,15 but these two methods are based on expert opinion and experience. The approach taken to evaluate the American Stop Smoking Intervention Study (ASSIST), a national intervention programme in the USA, involves a comprehensive evaluation model, including assessment of the construct of Strength of Tobacco Control (SOTC), of pro-tobacco efforts, of policy outcomes and of change in cigarette consumption and smoking prevalence.16–18 Based on this model, with increasing SOTC, better policy outcomes are expected with impact on cigarette consumption and smoking prevalence.

Using the conceptual framework of the ASSIST evaluation, we revised and adapted the SOTC evaluation model for China and applied it to the programme funded by the US National Institutes of Health (NIH) Fogarty International Center in Shanghai, Tianjin, Guangdong, Zhejiang, Henan, Jiangxi and Heilongjiang, located in the north, middle and east of China. We developed two Policy Performance Indicators (PPIs) to assess the implementation of measures for protection from exposure to SHS for China and compared the PPI with the SOTC. This paper describes the PPI and its use to evaluate the implementation of smoke-free policies in China.

Methods

This paper offers indicators for input to a SOTC index (SHS policy, capacity and efforts), a short-term outcome indicator, the PPI, and mid-term to long-term outcome indicators (knowledge, attitude and reduced exposure to SHS). We tested the correlation of the PPI with SOTC and PPI, as well as mid-term to long-term outcomes.

Definitions of indicators

Strength of Tobacco Control (SOTC)

SOTC was first developed by the ASSIST project to assess three major core components of a state-level tobacco control programme: tobacco control resources, capacity and programme efforts focused on policy and environmental change.17 Based on discussions with Chinese tobacco control experts, we adapted the SOTC for China with three components as below:

  • SHS policy where the organisation has adopted legislation that completely bans smoking in public places—that is, smoking is completely prohibited in indoor environments.

  • Capacity, which includes development of a plan, policy advocacy, media advocacy, monitoring and evaluation, provision of training or technical support and establishment of a tobacco control network.

  • Efforts or actions which include comprehensive tobacco control activities and assessment of the quality of those activities.

  • Activities include provision of cessation services, health campaigns and prevention of youth initiation.

  • Quality of the activities was measured by smoke-free environment signage, active dissuasion from smoking, compliance with the smoke-free indoor legislation in selected organisations (ie, by non-detectable odour of tobacco smoke, no cigarette butts found and nobody smoking on the premises).

See online supplementary appendix 1 for detailed definitions.

The SOTC score is computed by percentage assignment19 with the questions organised into four components: Ai, policy measures; Bj, capacity indicators; Ck, tobacco control activities; Dl, quality of the activities. A data reduction methodology was used to summarise the average score (%) of each SOTC component across all organisations within each province. Since each component was based on a different number of questions on SOTC indicators, the arithmetic mean of SOTC in each component was used as the score for that component and expressed as a percentage.

Policy Performance Indicator

PPI included two components (see online supplementary appendix 2): the percentage of indoor workplaces with no smoking in past 30 days and the percentage of indoor public places with no smoking in past 30 days.

Mid-term and long-term outcomes

The mid-term outcome indicator was defined by knowledge (percentage of respondents who agree that SHS causes serious diseases) and attitude (percentage of respondents who agree with complete bans on smoking in indoor public places and in one-person offices). The long-term outcome indicators are the male current smoking rate and the percentage of non-smokers exposed to tobacco smoke emitted from lit cigarettes or exhaled by a smoker on at least 1 day in a typical week—that is, prevalence of exposure to SHS.

See online supplementary appendix 2 for definition of indicators and questions.

Data collection methods

Data were collected in the baseline survey of the Fogarty project in the seven locations, from September to December 2010.

Organisation survey

We used an organisation-based sampling frame to collect SOTC indicators. First, we stratified the province or municipality into five regions: north, east, south, west and central. We listed all cities/districts in each of the five regions, and randomly selected one location. For that location, we listed all hospitals, classifying the hospitals into two categories: primary and secondary as one category and tertiary as the other. We selected three hospitals from each category. In the county or district where the hospital was located, we selected one school, one government department, one health bureau, one Center for Disease Control and Prevention (CDC) office or health education institute (if the CDC and health education institute were separate entities) and two transport stations. The resulting sample included six hospitals, six schools, six government departments, six health bureaus and six CDC offices or six health education institutes, and in addition, 12 transport stations in each city. Key informant interviews, observations and intercept interviews were conducted on the selected organisation premises. We selected 901 organisations and were able to complete interviews or carry out observations in 863 (95.8%) (table 1).

Table 1

Completion of surveys in seven provinces or municipalities

Household survey

In each province or municipality, household surveys were carried out, using a questionnaire based on the instrument used in the Global Adult Tobacco Survey (GATS) in China. The survey data were used to assess the PPI and the mid-term to long-term outcome indicators. The questionnaire was based on the GATS in China,20 with additional questions on attitudes towards banning smoking in indoor public places and in one-person offices. A four-step stratified cluster sampling method at the provincial/municipal level was used. In the locations selected for the organisational surveys, we listed all counties/districts, and randomly selected one county (rural) and one district (urban). In the selected county (district), we divided all rural villages (urban communities) into blocks with about 50 households. We used simple random sampling to select six blocks, and drew a map for use by the field team to survey every household on the block. Interviewers visited the selected households based on a household list, and recorded information on family members as required by the questionnaire. The interviewer then used the Kish table21 to randomly select one respondent who met the selection criteria for interview. Across the seven locations, 16 084 respondents were selected of whom 15 705 completed the survey. All provinces or municipalities completed the survey and the overall response rate was 97.6% (table 1).

The protocol used in this study received approval from the institutional review boards at Peking Union Medical College and the Johns Hopkins Bloomberg School of Public Health. A site survey guide was developed to ensure data quality. Before the survey was conducted, all interviewers were trained for 3 days. For respondent interview, consent was first obtained and the interview was conducted anonymously. For observation of organisation premises, the interviewer entered the premises to observe independently and objectively. There were three levels of supervision of data collection at the national, provincial and city levels; provincial auditors randomly selected 10% of respondents to verify data.

Standardised rate analysis

Provincial-level and municipal-level results from household surveys on PPI, knowledge of, and attitude to, tobacco harm, and on tobacco prevalence were adjusted by age, gender and urban–rural status based on the 2010 census.

Correlation analysis

Correlation analysis was performed on the PPI and SOTC indicators, and on the PPI and mid-term to long-term outcome indicators. The correlation analysis used results from all 35 locations (seven provinces/municipalities×five cities/districts). Either the Pearson correlation coefficient or Spearman's rank correlation was used, depending on the distribution of the data. A value of p<0.05 was considered as statistically significant.

Results

Social and demographic characteristics for the survey sample by province/municipality

The demographic characteristics of the sample differed somewhat from the characteristics of the entire provinces or municipalities, particularly for the urban-rural ratio. The crude prevalence of SHS exposure is >70% in Guangdong, Jiangxi, Heilongjiang and Tianjin, while Shanghai has the lowest exposure at 52.4%. The crude prevalence for current smoking among men is highest in Zhejiang and lowest in Henan (table 2).

Table 2

Social and demographic characteristics for survey sample by province or municipality

PPI in seven provinces/municipalities

Figure 1 shows the findings for the two components of the PPI by location. The surveys show that smoke-free policies were generally more common for indoor workplaces than indoor public places. Shanghai had the highest rates of implementation for these two indicators (at 56.5% for indoor workplaces and 49.1% for indoor public places, respectively), followed by Guangdong, Tianjin and Zhejiang (at 30–35% for these two indicators), and lastly Henan and Jiangxi (at 20–25%).

Figure 1

Policy Performance Indicator (PPI) for seven provinces or municipalities. Access the article online to view this figure in colour.

SOTC scores and correlation between SOTC scores and PPI

Table 3 shows that the overall SHS policy scores were generally low, ranging from 13.8 to 51.1. Of the three types of activities, health campaign scored the highest across all seven provinces/municipalities. For the quality of activities, there were high percentages for no ashtray, no smoking smell, no cigarette butts and no smoking on the premises, but rates for dissuasion from smoking were low. Shanghai scored highest and Jiangxi lowest on most of the SOTC components, a pattern consistent with the PPI.

The two components of the PPI correlated with certain components of SOTC. These components included policy (100% smoke-free indoor policy score) and activities (health education campaign, provision of cessation service, prevention of youth smoking in schools and other premises), as well as smoke-free signage for quality of activities. However, the SOTC capacity component, and one of the activities components (dissuasion from smoking) did not correlate with PPI. Some components of quality of activities (no smell of smoke, no cigarette butts and no smoking seen on the premises) correlated only with no smoking in indoor workplaces, while observing no ashtrays correlated only with no smoking in indoor public places (table 4).

Table 3

Strength of Tobacco Control (SOTC) scores for seven provinces or municipalities

Table 4

Correlation among the Strength of Tobacco Control (SOTC) component scores and the Policy Performance Indicator (PPI)

Standardised rates for mid-term to long-term outcome indicators and correlation between PPI and outcome

The percentages of respondents who agreed that SHS causes serious diseases were relatively high (ranging from 55.3% in Guangdong to 87.3% in Shanghai); this measure correlated significantly only with report of no smoking at indoor workplaces. The percentages of awareness for different specific diseases caused by SHS were lower, with the lowest ranking for heart disease in adults, then lung illness in children and finally, the highest ranking for lung cancer in adults. The degree of awareness correlated with the PPI. Support for a smoke-free policy was high, both for indoor public places and for one-person offices; however, only the latter correlated significantly with PPI. For long-term outcome indicators, SHS exposure prevalence and male current smoking rate were still high, and neither correlated with the PPI.

Among the seven provinces/municipalities, Shanghai had the highest percentages for reported knowledge and favourable attitude, and the lowest SHS exposure prevalence and male smoking rate. Respondents in Tianjin, Heilongjiang and Zhejiang indicated greater knowledge than those in Jiangxi, Guangdong and Henan. For the long-term outcome, Shanghai, Henan and Heilongjiang also had more favourable findings while Jiangxi and Tianjin scored worst on the indicators (tables 5 and 6).

Table 5

Standardised rates for mid-term to long-term outcome indicators

Table 6

Correlation between outcome indicators and Policy Performance Indicator (PPI)

Discussion

This study examined correlations of SOTC (SHS policy, capacity and efforts) with an immediate indicator, the PPI, at the provincial or municipal level. Use of SOTC in China is new and timely, as policy measures are implemented locally and nationally, and China deals more aggressively with its massive epidemic of tobacco smoking. The SOTC construct was adapted from the ASSIST initiative in the USA. We found that the ASSIST indicators were useful to assess smoke-free policy implementation in the seven study provinces and municipalities, particularly the indicators of efforts or actions.

All seven locations had employed many programme activities, but some had not implemented them well according to quality indicators. For example, some measures of quality of activities correlated with only one of the indicators of PPI, but some (dissuasion from smoking) correlated with none. These indicators could be used longitudinally to test the quality of activities and guide interventions. The correlation between capacity and PPI was not significant. Previously, we focused on CDC organisations across China to investigate some components of SOTC. We found that <2% of all staff in the Chinese CDC focused on tobacco control and that most staff (83%) had <5 years’ experience of working in this field. In 2008, tobacco control expenditure in provincial and city-level CDCs and health education institutes accounted for only 0.4% of organisational outlays. The level of tobacco control was further weakened by lack of tobacco control capacity, inadequate training and inability to design effective strategic activities or action plans.22 Consequently, weak tobacco control capacity and lack of budget limited the capacity of the provinces.22 ,23

To develop an easier and more convenient evaluation indicator, the PPI was introduced. We found a strong correlation between most components of the SOTC and the PPI. The SOTC ranking was similar to the PPI ranking in the seven provinces. Thus the PPI appears to be a useful practical indicator, although the SOTC has proved reliable for formal assessment.

PPI correlated with mid-term outcome indicators, but not with long-term outcome indicators. Based on the activities of SOTC components, we found that health education campaigns were the main activity for the seven provinces/municipalities. We cannot yet assess their longer-term impact.

Indicators for monitoring tobacco control policies have been shown to be useful in other countries/regions. A rating system for state indoor air laws24 was used to measure tobacco control activities at the state level in the USA by the American Lung Association.25 WHO developed an evaluation index of existing policies and compliance with MPOWER and applied it to different countries.15 However, both these methods focused on policy, not on implementation. The Tobacco Control Score developed by World Bank quantifies the implementation of tobacco control policies at the country level,14 but the scores were scaled based on expert experience. The International Tobacco Control project also developed an index to evaluate the implementation of smoke-free policy, but the questions referred only to smokers.7 In this paper, the questions for the PPI came from the GATS household survey. Because of the increasing use of GATS, the PPI could have wide applicability.

We note several limitations of the study data. SOTC indicators should include resources, policy, capacity and implementation efforts. In this study, we tried to collect information on resources through snowball sampling. However, tobacco control expenditure was a sensitive topic. A high percentage of respondents declined to answer such questions during interviews. Therefore, we were unable to include this information in this study. SOTC indicators should reflect the whole province's situation; however, it was difficult to obtain such information across large areas and populations. We opted to sample a feasible number of typical establishments. Thus, representativeness may be limited. At this stage, collecting this type of data at the provincial level remains a challenge.

China has no national smoke-free environment policy, although some cities have implemented such measures. Only the regulations of Ha'erbin (the capital city of Heilongjiang) and Tianjin satisfy Article 8 of the FCTC (developed in 2011 and 2012, respectively). While Shanghai has local legislation and the highest scores for SOTC, PPI and mid-term to long-term outcomes, the city does not come close to meeting the FCTC's mandate. Thus, despite some progress, China still faces an enormous challenge in creating smoke-free environments.

What this paper adds

  • The Policy Performance Indicator (PPI) of protection from exposure to secondhand smoke (SHS) is a rapid and simple method to assess provincial progress on control of SHS.

  • The PPI correlates with the Strength of Tobacco Control and reflects some activities that are measured by this indicator.

  • PPI also correlates directly with mid-term outcomes. PPI could be an earlier indicator (short-term) of the progress being made in China on protection from exposure to SHS.

  • Comparison of PPI across provinces (subnational comparison) has been completed.

Acknowledgments

We thank the seven provincial CDC staff and local staff of hospitals, schools, government buildings, health bureaus and public transport facilities at the study sites for collecting the data.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Files in this Data Supplement:

Footnotes

  • Correction notice This article has been corrected since it was published Online First. The * and # symbols have been removed from the table 5 headings as there are no accompanying footnotes for this table.

  • Contributors GY, JMS and FS conceived the study; GY, XW, HL and DH organised and supervised the data collection; XW, HL and ZD analysed the data; XW and GY wrote the first draft of the manuscript. FS and JMS revised and contributed to the writing of the manuscript. ST and MS reviewed and revised the manuscript.

  • Funding This work was supported by NIH project ‘Epidemiology and intervention research for tobacco control in China’ (R01 RFA-TW-06-006).

  • Competing interests None.

  • Patient consent Obtained.

  • Ethics approval This study was conducted with the approval of the Institute of Basic Medical Sciences of Chinese Academy of Medical Sciences and the Johns Hopkins Bloomberg School of Public Health.

  • Provenance and peer review Not commissioned; externally peer reviewed.