METHODS

Setting and study population

There are 14 community health centres in Rhode Island, operating in 21 separate sites. All centres are privately owned, although they collectively care for a large proportion of the low income population of the state, many of whom receive public assistance for medical care. The percentage of patients below the federal poverty level, in respective health centres, ranged from 26–96% in 1997-98, with higher rates in urban sites.

Training intervention

The training programme was supported by Rhode Island Project ASSIST (American Stop Smoking Intervention Study for Cancer Prevention) and was conducted over a two year period, from October 1997 through September 1999, with eight sites trained in the first year and six trained in the second year. Our guideline dissemination approach included office system strategies found to be effective in improving provider intervention delivery and patient outcomes, such as chart prompts, ancillary staff involvement, and physician audits with feedback, along with skills training, and patient education materials.^4–7 Recent reviews of professional training interventions continue to support systems strategies over training alone,^2,8,9 and multiple strategies are important to address the variety of provider barriers.¹⁰ We also utilised an academic detailing/educational outreach approach, whereby trainers go out to the medical practice and tailor the training to the provider’s needs.¹¹

The training programme consisted of three, one hour sessions, conducted at the health centre sites, by a master’s level clinician as trainer. Continuing medical education (CME) credits were given to physicians and continuing education units (CEUs) were given for nurses and social workers, as an incentive for attendance. Session 1 provided chart review site feedback on their baseline documentation of the 4 A’s, along with a brief review of the AHCPR guideline. All staff with patient contact were invited to this session. A system for identifying and documenting smoking status, as part of the process of taking vital signs, was strongly encouraged.^1,2 We also encouraged a team approach, whereby the medical assistant or nurse might ask about smoking status while checking vital signs, while the clinical staff may provide more formal advice and assistance. Others, such as dieticians, dentists, and receptionists, might help by reinforcing tobacco control messages and/or maintaining patient education supplies. The trainer facilitated goal setting on new systems and facilitated buy-in necessary for implementing such changes. At the second session, about a month later, the trainer facilitated a working group, selected by the larger staff group, to develop a plan to implement the system goals set in session 1. The trainer reviewed samples of tobacco use identification approaches (stickers, stamps, pre-printed forms) and assisted with problem solving around specific barriers.

The third session focused on counselling skills for the health centre clinicians, and involved use of role play with challenging case examples from their own experience with smokers. This session also utilised a physician trainer, to serve as a role model for the doctor–patient interventions, since physicians are less likely to receive prior training in these skills.^4–6 The project provided materials to aid clinicians in counselling, including a laminated chart on cessation medication aids, patient tip sheets in multiple languages which could be photocopied, and samples of AHCPR provider and patient materials.¹²

First year sites received a follow up (booster) visit during the second year, in which post-training chart review feedback was provided and further problem solving was offered on any issues encountered with meeting their goals. A follow up chart review was then conducted with these sites to examine maintenance of changes.

Evaluation design

Based on Healthy People 2000 objectives,¹³ we sought to reach 75% of clinicians across any of the three training sessions. We defined “clinician” broadly as any medical professional with the opportunity to counsel smokers. Other outcomes included chart documentation of each of the four A’s (Ask, Advise, Assist, and Arrange) for the latest (most recent) visit. Charts were eligible for review if the patient was at least age 18 years, and the visit was for primary care services, with physicians or other clinicians in family practice, internal medicine, obstetric or gynaecological (ob/gyn) clinics. Specialty care visits were not eligible nor were appointments for laboratory work or other technical services without clinician contact. Walk-ins were counted and reviewed, as these represented a significant number of visits in a typical day. To protect patient confidentiality, names and medical chart numbers were not recorded on the chart audit recording forms.

Consecutive medical charts were selected at each health centre site by use of the appointment register to identify the last 100 appointments with eligible patients, starting two days before the chart review visit. None of the health centres utilised electronic medical records; therefore all reviews were with paper charts. If these 100 charts did not yield at least 30 smokers, we continued reviewing charts from prior days’ appointments until 30 smokers were identified, yielding different sample sizes per site.

All smokers’ and non-smokers’ charts were included for assessment of the first “A” (Ask), while only smokers’ charts were reviewed for Advise, Assist, and Arrange. (The fifth “A” for Assess was not part of the guideline at the time of this study.) Since chart notation of smoking status was meant to serve as a reminder for intervention, smoking status needed to be clearly visible in one of following places to be counted: chart cover, inside cover, history form, problem list, or in the first or last progress note. “Advise” was considered any notation of advice or discussion about quitting smoking, while “Assist” was distinguished further as notation of materials given to the patient, referral given, including in-house referral, recommendation of nicotine patch, gum, or other medication to aid cessation, or record of counselling or problem solving regarding smoking. Follow up was distinguished as any record of arranging a follow up visit or phone call or letter. The chart review also documented independent variables to be used in analyses: patient date of birth, sex, race/ethnicity, insurance status, and type of visit (for example, acute care, medical follow up visit, ob/gyn visit). The post-training audit was conducted at least six weeks following the third training session. The follow-up chart audit, for first year training sites only, was conducted one year later, six weeks after the follow up visit to the centre.

Since changes in chart documentation may also be influenced by health centre characteristics, we assessed five site level variables. These included size, based on total number of patient visits in fiscal year 1997-98 (large > 4000, medium 2000–4000, small < 2000), location (urban, suburban, rural), percentage of providers attending training, whether the medical director or nurse manager attended training (yes/no), and availability of “champion” on smoking cessation at the centre (yes/no). A “champion” was identified by project staff if there was any individual at the health centre who was a strong advocate on this issue, and who took responsibility to facilitate system changes to support tobacco control interventions.

Statistical analyses

Exploratory logistic regression modelling was used to screen candidate predictors for all four smoking cessation intervention outcomes. Subsequently, a hierarchical mixed effects logistic regression model with random effects for each centre was fitted separately for each outcome. It could be plausibly argued that there is an additional level of nesting introduced by physician; however, no clinician level information was collected from the medical records and only a single site level intraclass correlation coefficient (ICC) was calculated for each outcome. The ICC was allowed to vary across all three time points, in order to accommodate possible changes in the ratio of between- to within-centre variability over the period of observation.

Although follow up chart reviews were performed only in the eight year 1 sites, while pre- and post-reviews were done in all 14 sites, such planned missingness is ignorable in the sense of Little and Rubin¹⁴ when likelihood estimation is employed and all available data were used to estimate a common model across the entire study period. Starting with a model containing fixed effects and second order interactions for all candidate predictors identified during the exploratory phase of the analysis, we employed a backward elimination procedure based on univariate Wald tests to obtain a more parsimonious final model. All results presented were obtained using PROC NLMIXED of SAS 8.1¹⁵; confidence intervals were calculated at the 95% level throughout (95% CI).

RESULTS

Participation

Among the 21 health centre sites in Rhode Island, 15 agreed to participate in the training programme (nine in year 1, six in year 2). The order of participation was not random. The first sites that agreed to participate were scheduled first, with the intent that sites would be divided evenly over the two years of funding. Two initial refusals from the first year agreed to be deferred to the second year and did so. One first year site dropped out mid year because of preparation for Joint Commission on Accreditation of Healthcare Organizations (JCAHO) accreditation, and six other sites’ declined to participate during project year 2, because of JCAHO preparations. (There was no opportunity to defer later.) Therefore pre- and post-data were available from 14 sites, with follow up data available on the eight year 1 sites. Across these 14 sites, 151 individuals with patient contact participated in at least one training session, with eight sites reaching our objective of 75% clinician attendance (range 28–100%). Twenty seven physicians participated, earning a total of 57 hours of CME credits, 53 nurses received 118 CEU hours, and one social worker earned three CEU hours, while 70 other staff attended, including medical assistants, dieticians, WIC counsellors, receptionists, and dentists. Across the three sessions, the first session drew 145 participants (28 physicians, 28 nurses, 88 others); the second session was for a selected work group and a total of 75 attended (16 physicians, 26 nurses, 31 others); and the third skills training session drew a total of 85 (18 physicians, 24 nurses, and 43 others). The number of staff employed at each site varied widely, from as little as four to more than 40. Some sites were not willing to provide us with a total number of staff, although they calculated their attendance percentage for us across the three sessions. These counts were further complicated as some centres had multiple sites and combined staff for training sessions. Also medical staff were frequently shared across some sites. Therefore, it was not possible for us to determine a denominator to calculate average attendance within disciplines or by specific sessions. Among the site level variables, five of the 14 sites had a champion; five sites had a nurse manager or medical director who attended trainings. Size distributions were six large, three medium, and five small sites. Seven sites were urban, four suburban, and three rural.

Patient profile

Table 1 shows the profile of patients whose charts were reviewed in each of the three cross sectional samples, with sites combined. The profile was very similar across these samples. At baseline the average age was 41 (range 18–88), 79% were female, 41% were of white race. For analysis purposes, the smaller categories of Asian, Native American, Portuguese, and Other/unknown were combined as Other. The most frequent type of insurance (29%) was Medicaid (which included the state’s managed Medicaid programme for low income families). For ease of data analysis smaller categories were combined, including Medicare, Indian Health Service, and unknown status (missing in chart). While combined site profiles were similar across time, there was wide variation of demographic characteristics between sites, with a much larger percentage of white patients and an older population in rural sites (not shown).

We were quite successful in locating charts from the appointment lists, with an average of 9% of records unavailable for review, across the 14 sites and three time points. In order to gather the number of charts needed (100 charts or until 30 smokers were identified), the average number of calendar days reviewed was 17.5 (range 4–36) at baseline, 24.15 at post-training (range 7–41), and 15 days at the follow up review (range 4–31). Larger volume sites required fewer clinic days to yield the requisite numbers. One outlier was not included from these averages; this site was on an island with a small year round population and it took as long as 100 clinic days to identify eligible patient visits for the review. While chart review forms were anonymous, we examined overlap of individuals between time points, by matching dates of birth and sex. We found less than 5% match rate between any of the time point comparisons, allowing reasonable confidence in our assumptions of independent samples for analyses.

Outcomes

Figure 1 shows the pattern of chart documentation in the latest visit with percentage recording Ask, Advise, Assist, and Arrange follow up across time. Exploratory mixed effects logistic regression modelling with random site level intercepts found only two covariates pertaining to individual charts (patient sex and visit type), and one pertaining to centres (size of centre) to be associated with smoking cessation intervention outcomes at the individual visit times. These three covariates as well as visit time were then entered as fixed effects in a logistic regression model with random site level intercepts whose variability was allowed to change over time. After simplification, we arrived at the model for Ask shown in table 2. In the models for Ask, predicted probability of asking about smoking status significantly increased at post-training (odds ratio (OR) 3.00, 95% CI 1.19 to 7.55), and at the follow up time point (OR 5.96, 95% CI 1.84 to 19.32), for non-ob/gyn visits, compared to pre-training chart data. There were also significant effects for Ask by visit type (see fig 2 for the empirical rates). Compared to medical follow up visits, patients at a yearly physical were most likely to be asked (OR 35.83, 95% CI 24.90 to 51.57), followed by patients at first or consultation visits (OR 9.82, 95% CI 6.62 to 14.57), ob/gyn (OR 3.88, 95% CI 2.77 to 5.42), and Acute visits (OR 1.31, 95% CI 1.04 to 1.66). There was a significant interaction between sex and acute visit type (OR 1.77, 95% CI 1.15 to 2.73); while males were generally more likely to be asked than females, this difference was especially pronounced for acute visits (see fig 3 for the empirical rates). In addition, there was a significant interaction for Ask between time point and ob/gyn visit type. Contrary to the trend for medical follow up visits, asking at ob/gyn visits fell at post- and follow up time points (see fig 4 for the empirical rates).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Chart documentation of tobacco intervention at last visit. *Follow up chart reviews were done only in the eight year 1 sites, while pre- and post-reviews were done in all 14 sites.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Observed ask rates by visit type. *See fig 1 for explanation.

• Download figure
• Open in new tab
• Download powerpoint

Figure 3

Interaction of sex and visit type on Ask smoking status (all time points).

• Download figure
• Open in new tab
• Download powerpoint

Figure 4

Interaction of visit type and time point on Ask smoking status (both sexes). See fig 1 for explanation.

For documentation of Advise (table 3), there were significantly higher odds for males at pre-training: the odds for a male to be advised were two times greater than for females (OR 2.22, 95% CI 1.28 to 3.84). There was, however, an interaction effect between sex and time: males decreased for Advise from pre- to post-training and to follow up, while females increased at post- and fell off some at follow up (fig 5). There was also a main effect for visit type. Documentation of Advise was significantly more likely with patients at yearly physicals (OR 2.43, 95% CI 1.53 to 3.87), and first or consultation visits (OR 2.17, 95% CI 1.26 to 3.76), and somewhat less likely for patients at ob/gyn visits (OR 0.57, 95% CI 0.32 to 1.00), compared to medical follow up visits (fig 6).

• Download figure
• Open in new tab
• Download powerpoint

Figure 5

Interaction of sex and time point on Advise quitting (all patient types). *See fig 1 for explanation.

• Download figure
• Open in new tab
• Download powerpoint

Figure 6

Observed Advise rates by visit type. *See fig 1 for explanation.

Examining the results by centre revealed that there were increases in 12 of the 14 sites for Ask from pre- to post-training and further increases in 5 of 8 sites from post-training to follow-up. For Advise by centre, 10 of the 14 sites increased rates from pre- to post-training, and 4 of the 8 sites reviewed at follow up showed further increased rates of Advise. Interestingly, although the sites varied widely on Ask and Advise, none of this variation could be attributed to the site level covariates identified. As mentioned at the beginning of this section, only one such covariate, size of centre, appeared significant in exploratory analysis, perhaps because of the low number of sites and subsequent lack of power. Even this fell out of the final models for Ask and Advise. The ICCs for each outcome showed little change over time. Because of their importance in calculating the effective sample size for future studies conducted in similar settings,¹⁶ we note that their baseline values appeared stronger for Ask (ICC 0.29, 95% CI 0.15 to 0.48) than for Advise (ICC 0.09, 95% CI 0.02 to 0.30).

The effect of covariates of interest on the Assist and Arrange follow up could not be assessed because of the low rates of documentation on these outcomes. It is apparent from the crude percentage rates (fig 1) that there was no significant change in these interventions.

DISCUSSION

Our findings for the overall average provider performance rates increased for Ask from 30% pre- to 44% post-training. For Advise, rates increased from 19% to 26%, while Assist and Arrange follow up remained at about 9% and 6%, respectively. Then, in sites where a one year follow up assessment was possible, we observed somewhat decreased rates in Advise documentation. The statistical models told a more complicated story, however, with very different performance across visit types and patient sex. While Ask rates increased for most visit types, it was apparently easier to ask consistently at yearly physicals, or with first or consultation visits, where there may be more time and fewer competing demands and more often a greater emphasis on prevention. It is unclear why documentation of smoking status decreased at ob/gyn visits over time. It is possible that as obstetric visits occur more frequently for individual patients, the standard of asking at every visit is less likely to be applied. In many centres, the systems for managing ob/gyn visits are handled separately from other medical visits, so that the systems for asking smoking status may also be applied differently. It is also possible that ob/gyn physicians were less likely to attend the trainings. In any case, this finding is of concern given the high importance of identifying pregnant women who smoke in order to help them with quitting.

Rates for Advise were also significantly higher at all times for yearly physicals and first visits, compared to medical follow up visits. Chernof and colleagues,¹⁷ from a large sample of patient exit interviews, similarly reported that health habit counselling rates were lower as visit acuity increased. The implications of limiting Ask and Advise to non-acute visits are serious, as this means that smokers who only visit a physician when they are sick will not receive help with their smoking. Since the premise behind documenting smoking status is to prompt provider advice,^1,2 it was disappointing that Advise rates did not significantly increase over time, except for females, where they increased at post-training, but not at follow up. In contrast, Advise rates for males were higher at baseline and were higher for acute visits, but advice to males seemed to fall off at post-training. While the training may have increased advice to females, it is unclear why males received less attention. Young and Ward¹⁸ also found that males were more likely to receive an opportunistic discussion about smoking at an acute visit among physicians who were presented with hypothetical cases.

Documentation at the latest visit is a high standard. As a point of reference, we also recorded documentation in the last year (although statistical analyses were not performed on these measures). While these rates were mostly higher (38% to 43% for Advise, 15% to 18% for Assist, 9% to 9% for Arrange, from pre- to post-training, respectively), the degree of change for these indicators was similar to the latest visit rates. Documentation for Ask was given greater attention in the training than documentation of the other A’s, although documentation was encouraged for all the A’s with the feedback from the centres’ own charts.

The performance rates reported here were lower than others’ rates from surveys of physician self reports and patient reports, but they were similar to others’ reports of chart documentation. Our own survey of a population based sample of primary care providers in Rhode Island in the early 1990s showed that 67% of physicians’ reported they Asked smoking status of “most” (> 80%) of their patients, while 74% Advised, 35% Assisted, and 8% Arranged follow up on most smokers.¹⁹ However, a concurrent survey of Rhode Island smokers indicated that only 51% of patients, who had seen a physician in the past year, reported that anyone in the office had talked to them about smoking, 45% were advised to quit, 15% were offered help, and 3% had a follow up appointment arranged.²⁰ A 1996-97 national survey of patient reports showed similar rates, where 48% of smokers reported physician advice to quit smoking in the past year.²¹ National data from chart documentation (using pre-printed chart form) showed that identification of smoking status was 61% in 1995, while smoking counselling was 21% in the same year.²² These various surveys and methodologies represent different biases from sources of the data, as well as different settings in which the interventions occurred. Chart documentation may more likely under represent provider performance than patient reports or physician self reports. Still, collectively, these rates confirm there is much more to do, especially for Assist and Arrange, to truly help smokers. It is also possible that a community health centre setting may be a more challenging one in which to integrate tobacco interventions amidst competing demands of patient needs. It is unclear what are the “gold standard” rates, since there will always be competing demands in “real world” primary care, when tobacco counselling is not feasible at every visit.

It is not surprising that adoption rates of a practice guideline would vary widely for clinicians, and organisations, just as do smokers’ rates of adoption of advice to quit. In spite of wide availability of clinical practice guidelines for many medical topics, their adoption into routine care has been less than desirable and many barriers have been identified, including lack of training, time, financial compensation, confidence, and the belief that smokers don’t want to quit.^10,23–25 We specifically addressed several of these barriers by attending to office systems to share the tasks of interventions, training in counselling skills, and by redefining “success” as moving the smoker along in readiness to quit, rather than only as success in quitting. However, our intervention may not have been intensive enough to effectively change these systems and beliefs. It is also worth noting that in some sites we found almost complete turnover of clinicians, since our earlier training. Also, while most sites set a goal to implement a vital signs system, we found at the “booster” visits that few succeeded in doing so because of competing demands at the centres. These barriers are all realities that pose challenges to dissemination efforts in the “real” world. A more sustained effort may be needed to change provider behaviour, with repeated offerings over time, and with multiple messages from multiple sources—at local and higher levels. Kottke, in his meta-analysis several years ago,⁴ described the very similar attributes of successful interventions for smokers. However, Solberg, with Kottke and colleagues²⁶ later reported that implementing guidelines is much more complicated than changing behaviour of individual clinicians, as it involves greater organisational support and multiple strategies that take into account the multiple characteristics of the guideline, practice organisation, and external environment.

The AHCPR guideline recommends higher level administrative policies to support tobacco interventions, such as provider reimbursement or other incentives. At the time of this project, there was no statewide tobacco control programme to support smoking cessation. There were few referral options for providers, especially for low income smokers. At this time, the Rhode Island Department of Health (RIDOH) oversees a statewide telephone quit line and website. In addition, health centres that have applied for RIDOH funding have an on-site programme with a certified tobacco cessation specialist and required provider training opportunities. Patients at these sites receive free cessation counselling and nicotine replacement products. Such funding opportunities offer an important incentive. Yet few health centres in Rhode Island are taking advantage of this funding to date. Other policies are needed, from health insurers and other institutions, to apply more leverage on this issue.

Study limitations

There are a number of limitations to this evaluation of a dissemination of the guideline. First, the evaluation design was not a randomised trial, with a control group of health centres. Therefore, any changes, or lack of changes, observed from pre- to post-training, might be attributable to other influences beyond the training. Furthermore, only two thirds of the health centres in Rhode Island chose to participate, limiting generalisability even in our own state. Our ability to document maintenance of changes was limited by only eight sites having the opportunity for a booster session and follow up review. These eight sites may have participated in the first year because they were more interested in the topic, which therefore created a bias in favour of more positive outcomes. However, this bias does not seem to be supported by the results. There are limitations with chart review documentation, as this data source may underreport actual behaviour. The cross sectional sampling of charts may have included some patients more than once, which would threaten the assumption of independent samples. However, our match rate of birthdates and sex was less than 5% and our unit of observation was the visit rather than the patient. One individual on our staff conducted 95% of all the chart reviews, which provided some consistency in judgement applied to ratings. However, we did not conduct inter- or intra-rater reliability checks. Therefore, there may have been bias where judgement was applied to ratings. We may have undercounted smokers and their interventions, if their smoking status was not noted clearly. While efforts were made to operationalise definitions, it may have been difficult to distinguish the subtle differences between Advise and Assist with chart documentation, and extra Assist efforts may have been under reported. While our data specified the type of visit, we did not have data on the type of clinician performing the visit services. Therefore, we cannot comment on variations in performance by type of provider in this context. Nor can we examine outcomes by provider characteristics or their readiness to intervene with smokers. This data would help to improve the focus of further training. Many of these limitations are a function of funding constraints for time and staffing. When state health departments or internal quality improvement teams fund dissemination, such constraints will be common. Practical decisions for project design will be necessary. Still, future efforts should consider steps to insure external and internal validity with site selection and procedures.

In summary, we observed that implementation of Ask and Advise tobacco interventions in community health centres were more likely in non-acute visits, less likely in ob/gyn visits, and uneven across time as a function of patient sex and visit type. Interventions that Assist and Arrange follow up with smokers were not significantly impacted. This report is important because there are few published reports of dissemination efforts with the Smoking cessation clinical practice guideline, with non-research staff applying the recommendations. The guideline represents a very positive message based on extensive research evidence that effective smoking cessation interventions exist for primary care settings. Yet this evidence comes from research studies conducted under more ideal conditions. It is important therefore to recognise the challenges inherent in dissemination of these recommendations in less ideal settings. Unfortunately, the majority of smokers now are in lower income populations,³ and they are more likely to receive medical care in settings that are not ideal. Integration of tobacco interventions across all primary care visits, per the guideline recommendations, is likely to require a more sustained effort with repeated offerings and multiple strategies, including higher level policy reinforcements.