HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS REGISTER		Author Keyword(s) Vol Page [Advanced]

This Article Abstract Full Text (PDF) Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this link to a friend Similar articles in this journal Similar articles in PubMed Add article to my folders Download to citation manager Request Permissions Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Campbell, H S. Search for Related Content PubMed PubMed Citation Articles by Campbell, H S.

Minimal dataset for quitlines: a best practice

¹ Centre for Behavioural Research and Program Evaluation, University of Waterloo, Waterloo, Ontario, Canada
² University of Rochester Medical Center, Rochester New York, USA
³ North American Quitline Consortium, Phoenix, Arizona, USA
⁴ ClearWay Minnesota, Minneapolis, USA

Correspondence to:
H Sharon Campbell, Evaluation Studies, Centre for Behavioural Research and Program Evaluation, Lyle Hallman North, University of Waterloo, Waterloo, Ontario, Canada, N2L 3G1; sharoncm{at}healthy.uwaterloo.ca

Received 2 January 2007

Accepted 23 July 2007

	ABSTRACT

TOP ABSTRACT EVALUATION FRAMEWORK DEVELOPMENT PROCESS MINIMAL DATASET DISCUSSION CONCLUSION ACKNOWLEDGMENTS REFERENCES

 
Objectives: This paper discusses the development of a minimal dataset (MDS) for tobacco cessation quitlines across North America. The goal was to create a standardised instrument and protocol that would allow for comparisons and pooling of data across quitlines for evaluation and research purposes. Principles of utilisation focused evaluation were followed to achieve consensus across diverse stakeholder groups in two countries.

Methods: The North American Quitline Consortium (NAQC) assembled a working group with representatives from quitline service providers, funders, evaluators and researchers from Canada and the United States. An extensive, iterative consultation process over two years led to consensus on the evaluation domains, indicators and specific items. Descriptive information on quitline service models, data collection protocols and methodological issues were addressed.

Results: The resulting minimal dataset (MDS) includes 15 items collected from eligible callers at intake and eight items collected from smokers participating in evaluation. Recommendations for selecting evaluation participants, length of follow-up and repeat callers were developed. Full MDS questions and technical documents are available on the NAQC website.

Conclusion: Adoption and implementation of the MDS occurred in the majority of North American quitlines by the end of 2006. Key success factors included a focus on utility and feasibility, a commitment to meeting multiple and varied needs, sensitivity to situational factors and investment in working interactively with stakeholders. The creation and implementation of a MDS across two countries is an important "first" in tobacco control which will help speed the creation of practice based evidence and facilitate practice based research.

Abbreviations: ENQ, European Network of Quitlines; ESCHER, European Smoking Cessation Helplines Evaluation Research; MDS, minimal dataset; NAQC, North American Quitline Consortium; SRNT, Society For Research On Nicotine And Tobacco

Keywords: quitlines; evaluation; research; tobacco; cessation

To achieve meaningful and rapid reductions in tobacco related morbidity and mortality, we must increase both the number of individuals who attempt to quit, and the success rate of those attempts. Among the many available cessation programmes, telephone based cessation services, also known as quitlines, offer the opportunity to achieve both goals because of their broad reach and effectiveness.^1–5 Two Cochrane reviews have supported quitlines, particularly those that offer proactive services, as an effective population based approach to tobacco cessation.^{1 2} Studies have also shown quitlines to be cost effective⁶ and cost beneficial.⁷ In fact, quitlines are increasingly being recognised as a key component of comprehensive tobacco control strategies in Australia,⁸ Canada, the United Kingdom, the United States^{9 10} and the European Union.

The emergence of quitlines as a population based approach to smoking cessation has followed the classic innovation diffusion curve,¹¹ at least in North America. The first North American quitline service at a population level began in California in 1992; by 2002 there were 33 state and provincial quitlines and by 2006 all smokers in North America had access to quitline services.¹² Despite the proliferation of quitlines there were no agreed upon metrics or standards for evaluation, making it difficult for quitlines to learn from each other, and comparative studies impossible.

The first North American Conference of Smoking Cessation Quitlines was held in Arizona in 2002. Participants at that meeting asked for a common approach to evaluation. The North American Quitline Consortium (NAQC), which was established two years later to provide leadership and a unified voice for quitlines, led the initiative to create a common evaluation approach. This paper describes the development of an evaluation framework and standardised set of measures (now known as the minimal dataset or MDS) designed to inform quitline operations, facilitate comparative evaluation studies and enhance research opportunities.

	EVALUATION FRAMEWORK

TOP ABSTRACT EVALUATION FRAMEWORK DEVELOPMENT PROCESS MINIMAL DATASET DISCUSSION CONCLUSION ACKNOWLEDGMENTS REFERENCES

 
All evaluations are concerned with utility, generalisability, scientific rigor and relevance. While there are many approaches to evaluation, utilisation focused evaluation is distinguished by its emphasis on utility, relevance and stakeholder involvement.¹³ What fundamentally distinguishes utilisation focused evaluation is that "the evaluator does not carry the burden for making choices about the nature, purpose, content and methods of evaluation alone. These decisions are shared by an identifiable and organised group of intended users." Guiding principles described by Patton include active involvement of users, a commitment to meeting multiple and varied needs, a concern for utilisation as a driving force, sensitivity to situational factors affecting utilisation and investment in working interactively with stakeholders.

This framework and accompanying principles were especially relevant for the creation of a common dataset for evaluating quitlines. There were a large and diverse number of stakeholders with an interest and enthusiasm for learning what works in what settings and for whom. The rich diversity of quitline service delivery models, clients and target populations in different settings across North America made comparative studies possible via the adoption of a set of common measures.

	DEVELOPMENT PROCESS

TOP ABSTRACT EVALUATION FRAMEWORK DEVELOPMENT PROCESS MINIMAL DATASET DISCUSSION CONCLUSION ACKNOWLEDGMENTS REFERENCES

 
The North American Quitline Consortium to establish a 14-member working group including quitline researchers, service providers, evaluators and funders from across North America. Members of the working group had linkages to leading edge quitlines and key organisational affiliations (National Cancer Institute, US Centers for Disease Control and Prevention, Health Canada, NCIC Centre for Behavioural Research and Program Evaluation). From the outset, the vision was to develop a set of measures that would provide valid, standardised data on a few important indicators.

The process, illustrated in table 1, occurred over two years. The working group operated via teleconference and email, with key face to face meetings attached to other meetings and conferences, all supported by funding from NAQC sponsors, Health Canada and the Canadian Tobacco Control Research Initiative. Members of the working group provided leadership and momentum for the MDS, both internationally and within their own organisations. In addition, researchers with the European Smoking Cessation Helplines Evaluation Research (ESCHER) group of the European Network of Quitlines (ENQ) participated in key meetings.

The Working Group adopted a set of guiding principles to ensure the MDS would facilitate service provision, evaluation and research, make comparisons possible and not impose undue resource burdens on quitlines. These principles were operationalised as follows:

• indicators must inform decisions important to the improvement of quitlines
  
• whenever possible, preservation of questions and wording already in use to allow quitlines to continue historical comparisons
  
• preference given to measures with acceptable reliability and validity, endorsed by scientific bodies (for example, Society For Research On Nicotine And Tobacco, SRNT), or used in national surveys (for example, census demographic questions)
  
• size (total number of items) of the MDS must not create barriers to meeting the needs of smokers calling for help with quitting.
  

After decisions on evaluation domains and question wording were approved, the working group addressed protocols for data collection, issues pertaining to length of follow-up, repeat callers and information needed to understand the uniqueness of each quitline.

The final MDS and protocols were approved by the Research and Evaluation Working Group in February 2005. To facilitate implementation, NAQC shared the MDS with all NAQC members and interested parties by email, on the NAQC website and during the first NAQC annual meeting in May 2005. NAQC also hired a technical expert and offered assistance via conference calls, online resources and individual consultation for a six-month period.

By the end of 2005, all quitlines reported that voluntary implementation of the MDS was planned or under way. A NAQC survey of MDS implementation is planned for 2007 to determine implementation, fidelity, areas for update and new information needs.

	MINIMAL DATASET

TOP ABSTRACT EVALUATION FRAMEWORK DEVELOPMENT PROCESS MINIMAL DATASET DISCUSSION CONCLUSION ACKNOWLEDGMENTS REFERENCES

 
The minimal dataset consists of a set of 15 intake (baseline) questions asked of eligible callers at the time of their first call and eight follow-up questions collected during follow-up interviews seven months after intake. As per the logic model, indicators included those designed to evaluate quitline promotion activities (for example, client awareness and response to promotion strategies, previous calls to the quitline), reach (type of caller, client demographic characteristics, geographic location and tobacco behaviours), service delivery (client satisfaction) and outcomes (changes in tobacco behaviours, including quit attempts, quit rates and switching to other forms of tobacco). As with any evaluation, callers can refuse to answer questions without penalty, although almost all intake questions also informed the counselling intervention.

Three factors that have consistently been shown to predict quit success (nicotine dependence, readiness to quit, use of other quit aids) are also captured. Although many quitlines use stages of change to inform their counselling interventions, the working group did not include it in the MDS given the equivocal evidence of its’ predictive validity for quit success when other factors are taken into account.^14–16

Table 2 outlines the MDS intake and follow-up questions, the corresponding evaluation indicators and examples of the types of decisions informed by each domain. The MDS questions and technical documents are posted on the NAQC website at www.naquitline.org/index.asp?dbid = 2&dbsection = research.

There is also considerable variability in quitlines’ evaluation mandates and resources. With respect to evaluation methodology, the working group concluded: "Each quitline will need to determine how follow-up will be conducted and on which population. ...[quitlines] should strive to survey enough people to draw valid conclusions about their outcomes, but it will be up to the individual quitline to determine whether census surveying, random sampling, cohort sampling, or some other sampling method will be most appropriate."¹⁷ While not ideal from a research perspective, this approach made implementation of the MDS possible with the expectation that in future common evaluation methodologies could also be implemented.

The working group followed expert guidelines for abstinence measures,^{19 20} recommending at minimum, six-month follow-up and 30-day point prevalence. This measure was meaningful to funders and follow-up was feasible for quitlines. Quitlines already measuring prolonged abstinence and 12-month outcomes were encouraged to continue. NAQC also recommended that quitlines report abstinence rates using both an "intent to treat" analysis and analysis of only those who completed the evaluation, with the recognition that the true quit rate lies somewhere between the two measures. It was also determined that a one-month intervention period would permit completion of most or all proactive counselling calls for most quitlines. Thus the working group recommended the follow-up interviews occur seven months after the intake date. This allowed one month for the full intervention (or at least the majority of proactive calls), plus the standard six-month follow-up period.

The final issue tackled by the working group was how to treat repeat callers. A small percentage of callers make extensive use of quitlines and the dilemma was at what point to consider them as making a new quit attempt versus continuing to act on the original counselling intervention. After considerable debate, it was agreed that a 12-month period from the first quitline contact should be considered a new quit attempt. Thus smokers who relapsed but called back for a second quit attempt 13 or more months after their original call to the quitline would be considered as new callers for evaluation purposes.

	DISCUSSION

TOP ABSTRACT EVALUATION FRAMEWORK DEVELOPMENT PROCESS MINIMAL DATASET DISCUSSION CONCLUSION ACKNOWLEDGMENTS REFERENCES

 
The minimal dataset provides the basis for a North American laboratory or "community of practice" for research and evaluation and enables us to capitalise on the diversity across quitlines and the large numbers of callers served. Quitlines themselves represent a success story of the translation of research evidence to public health practice. The MDS represents a best practice in quitline evaluation. It is a living document and will be revised and expanded as evidence, experience and capacity allow.

A desire to keep the MDS small and operationally feasible took paramount importance in discussions. It was recognised that the MDS must be easy to implement and be respectful of quitlines’ service mandate. The multi-stakeholder working group enabled satisfactory resolution between researchers’ desire for comprehensive baseline data and service providers’ concerns about the time required to collect those data. As one participant noted "What made this work was willingness on the part of the researchers to balance scientific rigor with practicality, coupled with a respect for and interest in scientific integrity on the part of program and policy decision makers."

A second key success factor was accommodating differences across quitlines. Individual quitlines each had their own evaluation questions to preserve and reasons for wanting to do so. Acknowledgement that the MDS was an adjustment to existing protocols authenticated individual quitline evaluation processes while at the same time moved the field as a whole toward a core set of items. The resulting MDS replaced some but not all pre-existing questions.

Though time intensive, the multi-stakeholder collaborative approach was another key success factor. As noted by one quitline service provider, "The collaborative approach was unique and progressive. By bringing together such a range of stakeholders, all perspectives were represented from the very beginning, which allowed informed and rapid input, critiquing and feedback to produce a MDS that will be relevant and manageable to implement". Also important to success was the momentum, started by Ossip-Klein in a presentation at the North American Conference of Smoking Cessation Quitlines and continued by the co-chairs and members of the working group. Finally, NAQC’s role in communicating and providing technical assistance to address implementation issues and support quitlines was critical.

There are several early indicators of the success of this endeavour. As of December 2006, the majority of US states and almost all Canadian provinces are implementing the MDS at some level (Bailey, personal communication). Other indicators of success include adoption of the MDS as a template for data collection by the University of Massachusetts Tobacco Treatment Specialist Training Program. The US Centers for Disease Control and Prevention asked quitlines they fund to report on the use of the MDS.⁹ Other institutions have recognised the value of standardised data and are proposing to adapt the MDS or create a similar standardised dataset (Web Assisted Tobacco Intervention Initiative; University of Rochester Medical Center; the Ontario Tobacco Research Unit; and the Association for the Treatment of Tobacco Use and Dependence).

	CONCLUSION

TOP ABSTRACT EVALUATION FRAMEWORK DEVELOPMENT PROCESS MINIMAL DATASET DISCUSSION CONCLUSION ACKNOWLEDGMENTS REFERENCES

 
The North American Quitline Consortium developed, disseminated and oversaw implementation of a minimal dataset for quitlines. This was a joint effort of multiple stakeholders (service providers, funders, evaluators, researchers) in Canada and the United States, with advice from leading research groups in North America (SRNT) and Europe (ESCHER). This achievement should be useful to other areas of tobacco control programming, as the MDS represents a best practice in tobacco control evaluation. The NAQC encourages other components of comprehensive tobacco control strategies to consider building on this experience so we can accelerate reductions in tobacco related mortality and morbidity.

What this paper adds Quitlines have provided a successful translation of intervention research to public health impact, with a range of models and services offered around the world. This rich diversity of quitline models has offered the potential for practice based research and evaluation; however, this opportunity has not been realised because of a lack of standardisation in measures across services and a lack of venues for multi-stakeholder collaborations. The North American Quitline Consortium developed and disseminated a minimal dataset (MDS) for quitlines in North America as a joint effort of multiple stakeholders involved in quitlines in the United States and Canada (service providers, funders, evaluators and researchers) with links to leading edge quitlines and key organisations. This paper documents the development of the MDS, the resulting intake and follow-up items and the decisions they help inform. The resulting MDS opens the door to research and evaluation collaborations across quitlines; the process of developing the MDS provides a model that can be replicated for other tobacco control programmes that involve complex partnerships.

 

	ACKNOWLEDGMENTS

TOP ABSTRACT EVALUATION FRAMEWORK DEVELOPMENT PROCESS MINIMAL DATASET DISCUSSION CONCLUSION ACKNOWLEDGMENTS REFERENCES

 
The authors would like to acknowledge the many different members of the quitline community who provided comments, feedback and support for development of the minimal dataset. HSC also acknowledges funding from the Canadian Cancer Society which supports the Centre for Behavioural Research and Program Evaluation (R&E).

Erik Augustson, Tobacco Control Research Branch, National Cancer Institute; Linda Bailey (Co-Chair MDS Working Group), North American Quitline Consortium; Sharon Campbell (Co-Chair MDS and R&E Working Groups), Centre for Behavioural Research and Program Evaluation, University of Waterloo; Sharon Cummins, California Smokers’ Helpline, University of California San Diego; Donna Czukar, Cancer Information and Support, Canadian Cancer Society; Corinne Husten, Office on Smoking and Health, Centers for Disease Control and Prevention; Ann Malarcher, Office on Smoking and Health, Centers for Disease Control and Prevention; Paul McDonald, Population Health Research Group, University of Waterloo; Deborah Ossip-Klein (Co-Chair R&E Working Group), Smoking Research Program, University of Rochester; Joanne Pike, American Cancer Society; Abby Rosenthal, Office on Smoking and Health, Centers for Disease Control and Prevention; Barbara Schillo, ClearWay Minnesota; Donna Vallone, Research and Evaluation, American Legacy Foundation; Susan Zbikowski, Clinical and Behavioral Sciences, Free & Clear, Inc.

	REFERENCES

TOP ABSTRACT EVALUATION FRAMEWORK DEVELOPMENT PROCESS MINIMAL DATASET DISCUSSION CONCLUSION ACKNOWLEDGMENTS REFERENCES

 

1. Stead L, Lancaster T, Perera R. Telephone counseling for smoking cessation. Cochrane Review. The Cochrane Library, Issue 2 2004.
2. Stead LF, Perera R, Lancaster T. A systematic review of interventions for smokers who contact quitlines. Tob Control 2007; 16(Suppl): i3–8.[Abstract/Free Full Text]
3. Ossip-Klein DJ, McIntosh SM. Quitlines in North America: evidence base and applications. Am J Med Sci 2003; 326: 201–5.[CrossRef][Medline]
4. Zhu SH, Anderson CM, Tedeschi GJ, et al. Evidence of real-world effectiveness of a telephone quitline for smokers. N Engl J Med 2002; 347: 1087–93.[Abstract/Free Full Text]
5. Ossip-Klein DJ, Giovino GA, Megahed N, et al. Effects of a smokers’ hotline: results of a ten county self-help trial. J Consult Clin Psychol 1991; 59: 325–32.[CrossRef][Medline]
6. Tomson T, Helgason AR, Gilljam H. Quitline in smoking cessation: a cost-effectiveness analysis. Int J Tech Assess in Health Care 2004; 20: 469–74.
7. Stephens T, Campbell S, Ghent A. Ontario smokers’ helpline: a cost benefit analysis Presented at the European Society for Research on Nicotine and Tobacco. Prague, Czech Republic, March 2005.
8. MCDS. Australian national tobacco strategy, 2004–2009 . Canberra: Department of Health and Ageing, 2005.
9. Fiore M, Croyle R, Curry S, et al. Preventing 3 million premature deaths and helping 5 million smokers quit: a national action plan for tobacco cessation. Am J Public Health 2004; 94: 205–10.[Abstract/Free Full Text]
10. Centers for Disease Control and Prevention. Telephone quitlines: a resource for development, implementation, and evaluation . Atlanta, GA: US Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health, Final Edition, September, 2004.
11. Rogers EM. Diffusion of innovations . 5th ed. New York: Free Press, 2003.
12. North American Quitline Consortium. Quitlines of North America and Europe 2006 . Phoenix, AZ: NAQC, 2006.
13. Patton MQ. Utilization focused evaluation: the new century text . Thousand Oaks, CA: Sage, 1997.
14. Farkas AJ, Pierce JP, Zhu SH, et al. Addiction versus stages of change models in predicting smoking cessation. Addiction 1996;91:1271–80; discussion 1281–92.
15. Carlson LE, Taenzer P, Koopmans J, et al. Predictive value of aspects of the Transtheoretical Model on smoking cessation in a community-based, large-group cognitive behavioral program. Addict Behav 2003; 28: 725–40.[CrossRef][Medline]
16. Helgason AR, Tomson T, Lund KE, et al. Factors related to abstinence in a telephone helpline for smoking cessation. Eur J Public Health 2004; 14: 306–10.[Abstract/Free Full Text]
17. NAQC. Frequently Asked Questions about the MDS 2005; http://naquitline.org/pdfs/mds_faq.pdf
18. Cummins SE, Bailey L, Campbell S, et al. Tobacco cessation quitlines in North America: a descriptive study. Tob Control 2007; 16(Suppl): i9–15.[Abstract/Free Full Text]
19. Hughes J, Keely JP, Niaura RS, et al. Measures of abstinence from tobacco in clinical trials: issues and recommendations. Nic Tob Res 2003; 5: 13–25.
20. Pierce JP, Gilpin EA. A minimum 6-month prolonged abstinence should be required for evaluating smoking cessation trials. Nic Tob Res 2003; 5: 151–3.[CrossRef]

This article has been cited by other articles:

				E. Lichtenstein Quitlines Tob. Control, December 1, 2007; 16(Suppl_1): i1 - i2. [Full Text] [PDF]

				S. E Cummins, L. Bailey, S. Campbell, C. Koon-Kirby, and S.-H. Zhu Tobacco cessation quitlines in North America: a descriptive study Tob. Control, December 1, 2007; 16(Suppl_1): i9 - i15. [Abstract] [Full Text] [PDF]

				L. M Hayward, H S. Campbell, and C. Sutherland-Brown Aboriginal users of Canadian quitlines: an exploratory analysis Tob. Control, December 1, 2007; 16(Suppl_1): i60 - i64. [Abstract] [Full Text] [PDF]

This Article Abstract Full Text (PDF) Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this link to a friend Similar articles in this journal Similar articles in PubMed Add article to my folders Download to citation manager Request Permissions Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Campbell, H S. Search for Related Content PubMed PubMed Citation Articles by Campbell, H S.