Article Text

PDF
Analysing user-reported data for enhancement of SmokefreeTXT: a national text message smoking cessation intervention
  1. Heather Cole-Lewis1,
  2. Erik Augustson2,
  3. Amy Sanders1,
  4. Mary Schwarz3,
  5. Yisong Geng1,
  6. Kisha Coa1,
  7. Yvonne Hunt2
  1. 1ICF, Rockville, Maryland, USA
  2. 2Division of Cancer Control and Population Sciences, National Cancer Institute, Bethesda, Maryland, USA
  3. 3ICF, Fairfax, Virginia, USA
  1. Correspondence to Dr Erik Augustson, Tobacco Control Research Branch, Behavioral Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute, Room 3E118, 9609 Medical Center Drive, MSC 9761, Bethesda, MD 20892 (for UPS/FedEx, use: Rockville, MD 20850), USA; augustse{at}mail.nih.gov

Abstract

Objective This observational study highlights key insights related to participant engagement and cessation among adults who voluntarily subscribed to the nationwide US-based SmokefreeTXT program, a 42-day mobile phone text message smoking cessation program.

Methods Point prevalence abstinence rates were calculated for subscribers who initiated treatment in the program (n=18 080). The primary outcomes for this study were treatment completion and point prevalence abstinence rate at the end of the 42-day treatment. Secondary outcomes were point prevalence abstinence rates at 7 days postquit, 3 months post-treatment and 6 months post-treatment, as well as response rates to point prevalence abstinence assessments.

Results Over half the sample completed the 42-day treatment (n=9686). The end-of-treatment point prevalence abstinence for subscribers who initiated treatment was 7.2%. Among those who completed the entire 42 days of treatment, the end-of-treatment point prevalence abstinence was 12.9%. For subscribers who completed treatment, point prevalence abstinence results varied: 7 days postquit (23.7%), 3 months post-treatment (7.3%) and 6 months post-treatment (3.7%). Response rates for abstinence assessment messages ranged from 4.36% to 34.48%.

Conclusions Findings from this study illuminate the need to more deeply understand reasons for subscriber non-response and opt out and, in turn, improve program engagement and our ability to increase the likelihood for participants to stop smoking and measure long-term outcomes. Patterns of opt out for the program mirror the relapse curve generally observed for smoking cessation, thus highlighting time points at which to increase efforts to retain participants and provide additional support or incentives.

Statistics from Altmetric.com

Introduction

Mobile technology is ubiquitous and mobile phones are increasingly being used for mobile health (mHealth)—the delivery of healthcare, public health and health information seeking.1 ,2 In 2013, 91% of adults in the USA owned a cell phone and cell phones had become integral to key activities such as searching for health information.3 Mobile phone text message-based smoking cessation interventions have shown promise as an effective method to aid smokers in cessation efforts.4–8 Cost-effective, current solutions to smoking cessation are important, given the prevalent tobacco use in the USA today. Currently, more than 42 million adults in the USA smoke cigarettes, and tobacco use costs the USA more than $289 billion a year, including at least $133 billion in direct medical care for adults and more than $156 billion in lost productivity.9 While almost 70% of smokers want to quit, many of them try but relapse, with the majority of relapses occurring in the first 8 days.10 ,11 Among smokers who make unassisted quit attempts, only about 3–5% maintain abstinence up to 1 year after quitting, and most smokers make several quit attempts before they are able to maintain long-term abstinence.9 ,10 There is a great opportunity to interrupt these smoking trends using text message-based interventions to provide smokers with additional support during quit attempts by intervening at critical moments.

To capitalise on the expansion of mHealth and to further engage US smokers who want to quit smoking, the National Cancer Institute (NCI) launched SmokefreeTXT in 2011, an evidence-informed, fully automated, text message-based smoking cessation program. SmokefreeTXT is a two-way mobile phone text messaging program that simulates one-on-one interaction. Over 35 behaviour change techniques are utilised in the text message library.12 A majority of these messages are used to support behaviour change by addressing motivation and maximising self-regulatory capacity and skills. Several of the messages elicit a response from users to help gather information about their progress throughout the intervention. Importantly, some of these messages are point prevalence abstinence assessments, which provide crucial self-reported data on whether or not the user has been successful in quitting smoking. Users complete the program if they remain subscribed for 6 weeks. SmokefreeTXT aims to provide free, smoking cessation support to the US public—support they may not otherwise have the financial means, time or resources to obtain. As mobile access and usage continues to rise in the USA, particularly across underserved and at-risk populations, there is an increased need to provide the public with effective and engaging mHealth resources.13 Mobile-based smoking cessation interventions can reach large audiences, provide engaging and tailored content to subscribers and decrease the barriers (eg, cost, time investment, scheduling) present in more traditional interventions.4 Mobile interventions can also be more sustainable and cost-effective than many traditional smoking cessation interventions.4 ,14

The demand for convenient, reliable and supportive smoking cessation resources in the form of text messaging appears to be high, as evidenced by the volume of subscribers to the SmokefreeTXT program over the years. Additionally, there is strong evidence for the efficacy of text message-based smoking cessation programs. In recent years, randomised controlled trials (RCTs) of text messaging programs have shown them to be effective for short term and, in a few instances, long-term smoking cessation.4 ,8 Studies also show that text messaging programs have the potential to double a subscriber's chances of quitting compared with usual care.6 ,7 However, the extent to which such programs can be scaled needs more investigation.

There have been few publications in journals that analyse a real-world implementation of a text message-based cessation intervention on a national scale in the US context. Other national-scale text message-based interventions published in journals include Text4Baby in the USA, and Txt2Quit in New Zealand. However, to the best of our knowledge, there have been no reports of program completion and outcomes for real subscribers of the program at scale. Publications on the effectiveness of these programs are based on a subset of participants who sign up to participate in a research trial.15–17 SmokefreeTXT offers a unique opportunity to analyse the real-world implementation of a text message-based smoking cessation intervention on a national scale.

This observational study highlights key insights related to program completion, response to point prevalence abstinence assessments and point prevalence abstinence among adults who voluntarily subscribed to the nationwide implementation of the SmokefreeTXT program, a 42-day mobile phone text message smoking cessation program. The primary objective of the study is to examine self-reported user data from the public-facing program to determine program effectiveness and inform enhancements in an iterative fashion.

Methodology

The SmokefreeTXT intervention is a 6–8 week, fully automated smoking cessation text-message intervention developed by a team of health educators, mobile technology specialists and clinical psychologists specialising in smoking cessation. On subscription, subscribers are encouraged to set a quit date within the next 2 weeks. Based on a subscriber's quit date, s/he may receive up to 2 weeks of preparatory messaging counting down to quit date. A subscriber's quit date triggers the beginning of 6 weeks of intervention messaging that include a mix of tips, motivation and assessment messages. Subscribers receive up to 166 messages that provide general motivational support, tips on preparing to quit, advice on managing cravings, suggestions for smokefree activities, relevant smoking facts and recognition of cessation milestones. Assessment messages capture a user's smoking, mood and craving status, and responses to these assessment messages prompt a reply from SmokefreeTXT. Subscribers receive between one and five messages per day with messages being more frequent in the beginning and levelling off over time. Individuals can receive additional messages if they text one of the following keywords to the program: MOOD, CRAVE or SLIP. Stoyneva et al18 conducted an analysis mapping SmokefreeTXT using the behaviour change techniques taxonomy, and found 41 behaviour change techniques present in the program. Subscribers have the ability to opt out of participation in the program at any time by texting the word STOP to the program. Subscribers who opt out of the program are not recontacted per Cellular Telecommunications Industry Association guidelines.19

Data collection

A STROBE statement summarising essential elements of this observational study is included as an attachment.20 Participation in the SmokefreeTXT program is open to any person in the USA who owns a mobile phone and subscribes to the program by either texting the word QUIT to 47848 or by filling out a web form (smokefree.gov/smokefreetxt). Data were captured during real-world implementation of the SmokefreeTXT program between the dates of 20 September 2011 to 22 May 2014, yielding 34 137 subscribers. Subscribers found out about the SmokefreeTXT program in several different ways. The program is promoted on the Smokefree.gov home page, and is often mentioned as a resource in Smokefree social media posts. We also have partner organisations, including the Centers for Disease Control and Prevention and the Food and Drug Administration, who refer persons to the SmokefreeTXT program. Furthermore, in 2013, there was a large-scale digital promotional campaign to promote people to sign up for SmokefreeTXT.

This analysis considered for inclusion only subscribers who could have completed the 42-day treatment by the date data were pulled for assessment, 25 283 subscribers. Inclusion criteria were (1) legitimate quit date set (ie, the quit date chosen was on or after the date of subscription) and (2) received the first day of treatment (ie, did not opt out on or before their quit date). Thus, subscribers were included in this analysis only if they fully initiated treatment; herein, referred to as ‘treatment initiators’. No personal identifiers were captured on subscription and phone numbers were replaced with subscriber identification numbers in order to protect the privacy of subscribers; therefore, while each subscriber represents a unique phone number, there is no way to determine the number of unique subscribers to the program (ie, a single individual could have signed up multiple times using different phones). Informed consent was not necessary because these data were captured during real-world implementation of the SmokefreeTXT program and the collection of personally identifiable information and browsing metrics are outlined within the Privacy Policy and Terms of Service.

Measures

During program sign up, subscribers were asked to self-report demographic information such as gender, age, state of residence, smoking frequency and whether or not they were using a web-enabled phone. State of residence was categorised into geographic region in accordance with the US Census Bureau designation. Subscriber smoking frequency was classified as daily smoker if the subscriber self-reported a smoking status of ‘every day’; smoking frequency was classified as non-daily smoker if the subscriber self-reported a smoking status of ‘most days’, ‘some days’ or ‘less than that’. A subscriber could discontinue participation in the study (opt out) at any time by texting STOP. The number of subscribers who requested to opt out and the time at which they requested opt out was recorded. Cessation status was assessed via text message on quit date and weekly thereafter, including at 7 days postquit date, 42 days postquit date (ie, end of treatment), 3 months post-treatment (ie, 132 days post quit date) and 6 months post-treatment (ie, 222 days postquit date). To assess cessation status during the intervention, subscribers were asked, ‘Are you still smokefree? Reply: Yes or No’. After the intervention, follow-up messaging to assess cessation status was phrased as, ‘Are you smokefree? Reply: Yes or No’. This type of messaging was chosen for the public-facing program so as not to appear overly clinical in nature. A single attempt was made to contact subscribers at each assessment point, via text message. Subscribers who had opted out of the program prior to a follow-up assessment were not contacted.

Outcomes

Program completion

Program completion refers to participation in the program from the first day until the end of the 42-day treatment. A subscriber who remained in the program until the end of the 42-day treatment was considered to have received complete treatment; herein referred to as ‘treatment completers’. Those who requested to opt out of the program before the end of the 42-day treatment were considered to have received incomplete treatment.

Point prevalence abstinence

The primary cessation outcome measure for this study is point prevalence abstinence at end of treatment, as reported by subscriber responses to text message cessation assessments. Point prevalence abstinence is an assessment of cessation status at the particular point in time when the question is asked.21 Therefore, cessation is independent of subscriber response or non-response on previous or subsequent cessation assessment questions. Secondary outcome measures include point prevalence abstinence at 7 days postquit date, 3 months post-treatment and 6 months post-treatment.

Statistical analysis

Baseline characteristics

Descriptive statistics are used to determine characteristics of the population, including gender, age, baseline smoking frequency and geographic location. Any unreported or non-feasible characteristics are excluded from analysis of the corresponding demographic characteristic (ie, ages under 13 years of age or over 99 years of age).

Program completion outcomes

A sensitivity analysis using logistic regression was conducted to assess any differences in baseline characteristics between (1) those who were included in the study and are referred to as treatment initiators (ie, fully initiated treatment by subscribing with a legitimate quit date and receiving the first full day of treatment) and (2) those who were excluded from the study (ie, did not fully initiate treatment). Additionally, the number of users who requested to opt out of the program before the end of the intervention was plotted on a graph with the percentage of subscribers who remained in the program. Sensitivity analysis was also conducted via logistic regression to assess differences in baseline characteristics of those who opted out of the study before treatment completion and those who completed treatment (ie, treatment completers). Baseline characteristics with >20% missing data were excluded from sensitivity analyses.

Point prevalence abstinence outcomes

For each point prevalence outcome time point, we provide the following quit rate estimates: (1) a quit rate estimate that treats all non-responders as smokers, to account for the possibility that they have relapsed to smoking;22 and (2) a quit rate estimate for the subset of subscribers who completed treatment, treating non-responders as smokers. For the purposes of abstinence rate estimates, non-responders are classified as subscribers who either (1) do not respond to the cessation assessment question or (2) opt out of the program prior to the cessation assessment.

Results

The total population considered for inclusion in the study was 25 283 subscribers, of which 7203 (28.5%) were excluded because they did not fully initiate treatment (ie, set an illegitimate quit date or did not receive the first day of treatment). As a result, the sample for this study includes 18 080 (71.5%) subscribers. Demographic information for those subscribers is provided in table 1. The median self-reported age of the sample was 32 and participants in the sample were distributed across the four regions of the USA. The majority of the sample was women, smoked every day at baseline and had a web-enabled phone.

Table 1

Sample demographic and intervention characteristics (N=18 080)

Program completion outcomes

Sensitivity analysis comparing subscribers who fully initiated treatment and thus were included in the study sample (18 080; 71.5%) to those that did not fully initiate treatment (7203; 28.5%) suggest that those who were non-daily smokers were more likely not to fully initiate treatment than daily smokers (OR: 1.461; 95% CI 1.313 to 1.625). No other baseline characteristics differed significantly by treatment initiation status (data not shown). Baseline characteristic web-enabled phone was not included in sensitivity analyses, as data for this variable were missing for more than 20% of all subscribers considered for inclusion in the study.

Just over half of the sample that fully initiated treatment completed the entire 42-day treatment (53.6%). Figure 1 illustrates that nearly half of treatment opt out (4058 of 8394; 48.3%) occurred during the first week of treatment postquit date; the frequency of opt out decreased as treatment continued.

Figure 1

Percentage of total subscribers remaining in intervention and number of subscribers who opt out on each day of treatment.

Logistic regression comparing baseline characteristics of those who opted out of the study before treatment completion and those who completed treatment suggest that the odds of opting out before the end of treatment are decreased for men compared with women, non-daily smokers compared with daily smokers and as age increases (see table 2). Opt-out status did not vary significantly by region. Thus, subscribers who opted out were more likely to be women, daily smokers at baseline and be younger in age compared with those who completed the entire 42-day treatment.

Table 2

Logistic regression comparing baseline characteristics of those who opted out of the study before treatment completion and those who completed treatment (n=14 032)

Point prevalence abstinence outcomes

End of treatment

Overall, the end-of-treatment point prevalence abstinence for subscribers who initiated treatment was 7.2% (1306 of 18 080 subscribers; see table 3). Among those who completed the entire 42 days of treatment, end-of-treatment point prevalence abstinence was 12.9% (1259 of 9750 treatment completers) when all non-responders considered to be smokers.

Table 3

SmokefreeTXT point prevalence abstinence rates

Seven-day point prevalence abstinence

The 7-day point prevalence abstinence for subscribers who initiated treatment was 20.3% (3672 of 18 080 subscribers) when all non-responders and subscribers who requested to opt out were included in the analysis and assumed to be smoking. Among those who completed the entire 42 days of treatment, the 7-day point prevalence abstinence rate was 23.7% (2304 of 9750 treatment completers) when non-responders were included in the analysis and assumed to be smoking (see table 3).

Three months post-treatment point prevalence abstinence

The 3-month post-treatment point prevalence abstinence for subscribers who initiated treatment was 4.0% (725 of 18 080 subscribers) when all non-responders and subscribers who requested to opt out were treated as smokers. Among treatment completers, 3-month post-treatment point prevalence abstinence was 7.3% (707 of 9750 treatment completers) when all non-responders were considered to be smokers (see table 3).

Six- month post-treatment point prevalence abstinence

The 6-month post-treatment point prevalence abstinence was 2.0% (358 of 18 080 subscribers) for subscribers who initiated treatment when all non-responders and subscribers who requested to opt out were treated as smokers (see table 3). Among treatment completers, 6-month post-treatment point prevalence abstinence was 3.7% (358 of 9750 treatment completers) when all non-responders were included in the analysis and considered to be smokers.

Discussion

To the best of our knowledge, this is the first peer-reviewed publication of program engagement data, including completion and smoking abstinence rates for a population-based smoking cessation intervention delivered via text messaging. The real-world context of this study yielded a large sample size, providing a substantial amount of user-generated data that are not subject to the limitations of traditional data from experimental trials. In the 2.5-year period included in this study, over half of the subscribers completed the entire program for the real-world implementation of SmokefreeTXT and the pattern of opt out was similar to the shape of the relapse curve documented in the literature for smokers trying to quit.10 While these findings demonstrate that text-messaging smoking cessation programs can be implemented on a national scale, assertions about program effectiveness on smoking abstinence cannot be made due to low response rates. The effect size of the program is unknown due to the absence of an appropriate control group and the low response rates to abstinence questions. Trends observed in point prevalence abstinence response rate and opt out highlight the critical need to better understand subscribers in order to increase engagement with the program and retain program participants. These results can serve as a baseline for future iterations of the program to determine if program enhancements improve engagement and abstinence response rates.

The most important findings from this study are that this population-based smoking cessation intervention appears to supports short-term cessation and that over half who subscribe to the program complete the full treatment with no incentive (eg, monetary reward). As evidenced in figure 1, the majority of opt out happens in the first 2 weeks after the quit date, similar to what has been documented in the literature as the average trajectory for smokers trying to quit.10 This presents a particular opportunity for implementers of text message smoking cessation programs—it is worth exploring how characteristics of mobile technology can be best utilised to address this challenge that occurs even when using clinical or pharmaceutical methods of smoking cessation.

Additionally, voluntary response rates to the point prevalence abstinence assessments were low in this observational study of a population-based cessation treatment program, ranging between 4.36% and 31.52%. To the best of our knowledge, there are no other published studies that report response rates for follow-up assessments for a real-world implementation of a text messaging program. An NCI-funded, three-arm RCT evaluating the efficacy of SmokefreeTXT among young adults reported a response rate of 56.7%. Considering this response rate for an incentivised RCT among smokers motivated to quit, we believe this response rate is in line with what can be expected. Non-response to abstinence assessments could have occurred for a variety of reasons; because a subscriber chose not to respond to the message, has changed their phone number, has not changed their smoking behaviour, has relapsed to smoking and does not want to acknowledge it, has achieved abstinence and is thus no longer interested in engaging with the program, or has opted out of the program and thus no longer receives messages. In fact, indepth interviews of subscribers of the program conducted during a process evaluation of SmokefreeTXT revealed that subscribers sometimes opt out of the program because they have stopped smoking and no longer feel the need to receive messages, while others opt out because they slipped and smoked and wanted to reset the cessation program from the beginning.23 Not surprisingly, a number of participants opt out before treatment end and point prevalence abstinence assessment response rates decrease as time from sign-up increases. In the future, in order to determine the true real-world impact of population-based cessation programs, it is imperative to both engage subscribers in a way that will encourage them to complete treatment and to respond to abstinence assessments, as this provides support for determining the true rates for long-term abstinence.

We utilised conservative approaches to estimate point prevalence; therefore, we believe true abstinence rates will be equal to or more than what is reported in this study. The most conservative estimates that considered non-response and opt out subscribers as smokers ranged from 2.0% to 20.3% and less conservative estimates that included only those subscribers who completed treatment and considered those who did not respond as smokers ranged from 3.7% to 23.7% for the time points assessed. These results suggest that if we were able to obtain more information from those who did not respond to the point prevalence abstinence assessments, the true abstinence rate could be higher than the most conservative observed values (2.0–20.3%).

Given the higher abstinence rates at earlier assessment points, if this population-level treatment program were able to increase program completion and/or incentivise subscribers to respond to point prevalence abstinence assessments, the long-term abstinence rate estimates might be higher. Future research should explore potential strategies to increase program completion and response rates, or to creatively measure program engagement with other factors. Moreover, all future methods of engaging subscribers must take into account feasibility for implementation at the population level in regard to cost, time and other resources. Future studies should also seek to understand the optimal length of treatment necessary to achieve cessation and related cost-effectiveness.

Furthermore, our findings show that subscribers who initiate treatment in the program are more likely to be daily smokers than those who do not initiate treatment; however, daily smokers are less likely to complete treatment than non-daily smokers. This suggests that those who initiate treatment are more serious smokers who have much to gain from abstinence, but these are also the subscribers that are more likely to not complete treatment. Most smokers initiate multiple quit attempts before reaching abstinence.9 Therefore, it is important to seek ways to continuously engage subscribers in this treatment program in order to increase their likelihood of reaching abstinence and improve the ability to determine the population-level effectiveness of the program. As part of our program improvement efforts, we continue to collect various types of data and conduct analyses to better understand determinants of early opt out and non-response to abstinence assessments. This information is then used to propose enhancements to the program. The results from this study can serve as a baseline for future iterations of the program and A/B tests and factorial experiments will be used to evaluate the effectiveness of potential modifications to the program.

Beyond the limitations of the nature of this observational study, including low rates of program completion and response to abstinence assessments, we recognise that self-reported outcomes may be accurate but not as definitive as biochemical verifications.24 Though an observational study is not as rigorous as the gold standard RCT, it is more cost-effective, feasible and can easily be replicated. Additionally, this study is less susceptible to selection bias that may result from recruitment and screening in RCTs, where the bar for participation is often high. There is also the ability to quickly leverage lessons learnt to update real-world implementation while this technology is still relevant. Results from large-scale RCTs, while important, take longer to accrue and may not reflect real-life conditions. To that end, this observational study allows the SmokefreeTXT program to remain nimble and continue improving on this free smoking cessation service currently available to the entire US public. Furthermore, findings from this study provide insight into additional nuanced questions that can be explored in future RCTs such as what processes can be utilised to ensure that subscribers remain in treatment and re-engage non-responders so that they respond to abstinence assessments.

Conclusion

This relatively scalable program provides the opportunity to help people who are motivated to stop smoking reach their goal. We cannot make definite conclusions regarding the program effect on abstinence rates, given the low response rates and lack of a control group. Findings from this study illuminate the need to deeply understand reasons for subscriber opt out and non-response and, in turn, improve program delivery and our ability to measure long-term outcomes of the program.

Mobile technologies offer great potential to impact health awareness and behaviour change for large, diverse audiences. The opportunity exists for mHealth interventions to reach great portions the world population for a nominal cost compared with the billions of dollars currently lost to smoking-related illnesses.25 At a population level, SmokefreeTXT is a timely and effective means of smoking cessation. Further scaling of this program may be instrumental in reaching lower socioeconomic status populations, as well as international audiences. Results from this study inform strategies to improve this program and other large-scale text message-based intervention programs.

What this paper adds

  • Mobile phone text message-based smoking cessation interventions have shown promise as a method of smoking cessation. To date, evidence on the effectiveness of text message-based smoking cessation interventions has been obtained primarily from randomised controlled trials. This observational study provides insights on participant program completion, program response rates and rates of cessation for nationwide implementation of a text message-based smoking cessation program. This study shows that a text message-based smoking cessation program can be implemented at national scale with relatively minimal resources. The study highlights additional considerations for improving measurement of cessation outcomes in such an observational study, such as improving participant engagement with point prevalence abstinence assessments and decreasing participant opt out.

Acknowledgments

The authors would like to thank Christopher Griffith, Arun Varghese and Emily Grenen for their contribution to the study.

References

View Abstract

Footnotes

  • Correction notice Heather Cole-Lewis has been moved to first author and Erik Augustson is second author.

  • Contributors EA is responsible for the overall content as guarantor. HC-L, EA, AS, MS and YH conceptualised the study. HC-L and YG conducted the analysis. All authors contributed to the interpretation of the data, revised it critically and provided intellectual content. All authors have approved the final version of the paper.

  • Funding This work was supported by the National Institutes of Health, National Cancer Institute contract number HHSN261200900022C, subcontract number D6-ICF-1.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.