There is a vast amount of research about smoking cessation that’s been published with increasing frequency since the 1970s. Those who have been trained in critical appraisal of evidence in public health develop skills in being able to assess the strengths and weaknesses of research designs, and the ways in which authors of research papers and those who publicise them both select and highlight aspects of studies. In this chapter, I’ll look critically at the types of evidence that are used in debates about particular ways of quitting and discuss some of their implications for the overarching question of how to maximise successful, permanent smoking cessation across whole populations. Many of the limitations of different sorts of evidence need to be kept closely in mind when appraising arguments put forward for various approaches to quitting.

Evidence is not the plural of anecdote

In the early 1990s on a World No Tobacco Day in March, I was the guest of the local health service in Broken Hill, a mining town in the far west of New South Wales. The staff had arranged for me to be interviewed by the local radio station that had a massive footprint across the extensive far west of the state. The host invited former smokers to call in and talk about how they had quit. Consistent with everything we know about the method most ex-smokers use at their final successful attempt to quit, many callers wanted to talk about why they had quit (see Chapter 3). As had occurred many times before during similar interviews, I mostly had to probe them to talk about how they had quit. Nearly all had quit unassisted, going cold turkey.

However, I recall the last caller wanting to tell all listening across the far west of NSW that the various ways of quitting earlier callers had named were all very well. But no one had mentioned the very best method. Could our expert up from Sydney guess it? No, you tell us all, I suggested. Our caller then extolled the importance of letting Jesus Christ into your life. Jesus had stopped him smoking and could stop anyone smoking. Everyone needed to know this, he said. His own experience was all the evidence we needed.

Doubtless there are many people around the world who would make similar claims that their religious faith helped them quit. And those imbued with religious faith often don’t hold back about it. They make these claims sincerely and often passionately, wanting to share their experience with others to inspire them. But how effective is faith in Jesus in helping any random smoker, even one with strong faith, to quit smoking? How many who pray to quit succeed and how many fail?

Lo and behold, we have some information on this! A 2017 paper studied 2,839 people in a US national survey who smoked in the year prior to interview and attempted to quit during that year. It found the odds of reporting no longer smoking at the time of interview “were no greater for those who used prayer, any mind–body therapy, or both, than in those using neither” (Gillum, Santibanez et al. 2009). But such evidence is unlikely to deter those who found prayer successful in stopping smoking from proselytising about their “way”.

If you have quit smoking by taking a particular approach, and know others who have also succeeded that way, it can be hard to understand why there could ever be any debate that your method should not therefore be regarded as a self-evidently successful way to quit. And that anyone wanting to help others to quit too, would not want to shout about their way from the rooftops.

In the 2020 Australian Senate inquiry into vaping, the chair of the Select Committee on Tobacco Harm Reduction conservative Senator Hollie Hughes, declared in an evidence session that she had herself recently quit smoking by taking up vaping. She put this question to two vapers who had been selected to give evidence to the Committee:

So let’s consider why anecdotal evidence about quitting is always placed low in quality and importance when compared with all other forms of evidence (see Figure 2.1).

Self-selection bias

While sometimes compelling, personal anecdotes about how people quit smoking all wear the evidence-constricting crown of self-selection bias. People are far more at ease relating their success stories than failures. Those who have tried and failed to quit using any given method are understandably far less likely to be enthusiastic and evangelical than those who succeeded in stopping. Just as someone who tried to lose weight and failed is highly unlikely to want to take the time to write a political submission about their failure or call up a radio program discussing failed weight-loss methods, so too is it less likely that smokers who tried and failed would bother to spread their stories on every opportune soapbox.

In the opening statement of my evidence to the same 2020 Australian Senate inquiry on tobacco harm reduction I said:

Those who have not succeeded in quitting may feel that part of the problem lay with them, not with the method they used. They may feel they did not persevere as much as they should have, did not use a quit-smoking drug strictly as advised or complete the recommended course of the drug or procedure. They may feel awkward that others might draw that conclusion too, even if they did not mention these things themselves. Therefore we are likely to hear about failures less often.

So when we read the comment sections under online news articles, or hear smokers calling into radio programs with stories of their success, it’s likely that there is considerable self-selection bias at play. Uncritical readers or listeners may get the impression that the drug being talked about is far more useful than it actually is: “Wow, almost everyone who called in was praising this method of quitting.”

Such positive personal testimonies represent self-selection bias (Wikipedia 2020) about success and, while true for the individuals concerned, cannot be given credibility when it comes to making generalisations about the success or otherwise of any cessation method.

However, some find it self-evident that if someone swears by a method of quitting, that’s 95% of all we need to know. If it worked for this person, it will probably work for many others. But pointing out that anecdotal evidence is the lowest level of evidence often raises hackles, as we saw earlier in Senator Hollie Hughes’ attempt to provoke witnesses to the 2020 Senate Committee. I have often seen indignant comments on social media, particularly from vapers, saying, “Apparently I’m not a person with a lived personal experience of quitting smoking through vaping. I’m only an anecdote, understand. You need to speak to real people!”

Across a 45-year career in public health, I’ve heard and read countless testimonies supporting miracle smoking cures. These range from fairground hypnotism, acupuncture, herbal remedies, dipping your cigarettes in unpleasant-tasting potions before you smoke them (Chiang and Chapman 2006), paying someone to point a “laser” at special parts of your body while they charge you hundreds of dollars for the privilege, Alcoholics Anonymous–style smoking temptation story-sharing, thinly disguised religious pitches from church-based health groups talking about “higher powers”, mantras to recite when tempted to smoke, and various offerings from the pharmaceutical industry.

Glowing testimonies can be found from those who’ve tried every imaginable quitting strategy. Take “laser acupuncture”, for example. A very grateful “Crystal” (Crystal 2020) has this to say:

Acuquit, an Australian chain of 11 laser clinics, charges its customers $495 for an “overall treatment time [of] 30–45 minutes”. If required, a follow-up session is available for the knock-down price of $195. Its website states, “Laser Acupuncture had an 84% success rate in recent research (Lim 2018) with many participants reporting they no longer had the urge to smoke.” That claim is based on self-reports of those after they had completed seven sessions of laser therapy (“there was no long-term follow-up”). This is what is known as “end-of-treatment” results: the smoking status of participants as they finish a course of treatment and complete a questionnaire before leaving.

Because of the endemic problem of relapse into smoking (see later in this chapter), end-of-treatment quit outcomes are a highly inflated way of describing success rates in quitting. And when they are self-reports which are not verified by any biochemical testing, their status sinks even lower. This is particularly so when the record of the self-report is given to someone connected with the delivery of treatment. Smokers “get” that those trying to help them quit are very hopeful that they succeed, and so a “pleasing the therapist” phenomenon can occur where smokers say they have quit, even when they may have not.

The Instant Laser Clinic in Melbourne provides three 15-minute laser sessions for people wishing to quit smoking. Its website says:

Yes, these claims are indeed quite amazing, because few visitors to this website would be aware that the evidence for acupuncture and related methods (which include laser procedures) is “not shown to be more effective than a waiting list control for long-term abstinence”. In other words, the long-term impact of acupuncture or laser treatment on smoking cessation is no different to the effect of placing your name on a waiting list for such treatments but not ever actually having them (White, Rampes et al. 2014). Few would also be aware that detectable nicotine remains in the bloodstream for only one to three days, with its metabolite cotinine able to be detected for up to 10 days. So having a body free of nicotine if you fully stopped smoking using any method would result in exactly the same lack of nicotine as the claim made by Instant Laser Clinic.

Randomised controlled trials

The 2020–21 COVID-19 pandemic has seen two words, efficacy and effectiveness, given perhaps their most intense ever public workout as debate rages about vaccines. Often the two words are used interchangeably by journalists and the public. But in public health and epidemiology their meaning is quite different (Streiner 2002).

Efficacy refers to the performance of an intervention (such as a drug) under the near-to-ideal conditions that can be organised when conducting carefully controlled and monitored trials. Effectiveness refers to performance in uncontrolled real-world use: how well a drug works in the circumstances of its actual use, away from regular monitoring or oversight from researchers. As I’ll now explain, these differences are critically important for any understanding of what research tells us about how useful any quit-smoking method is or is likely to be throughout a population.

Randomised controlled trials (RCTs) have often been used to assess the efficacy of smoking cessation interventions. RCTs are revered in experimental and clinical science as being “gold standard” evidence about whether an intervention (often a drug) makes a difference to outcomes of interest, such as smoking cessation.

In the world of evidence-based medicine and public health, RCTs sit just below the apex of what is known as the quality of evidence pyramid (see Figure 2.1 earlier). The only evidence sitting above them in importance are studies that pool all quality RCTs on the same issues, weight results from the higher standard RCTs and draw conclusions about outcomes across all studies which reach high standards of evidence. RCTs are venerated because randomisation should ensure equal probability to any conceivable confounding variable that might bias the probability of any outcome being equally distributed between those in a trial allocated to the active and control arms of a trial.

Ideally, RCTs should also be double blinded: when the effects of a drug and a placebo are being compared, it is ideal if both those taking the drugs and placebos and those conducting the analysis of the results do not know who is in the control (often placebo) group and who is in the active drug group. Only after analysis is complete should the status of those in the placebo and active drug allocations be unmasked to those in the trial and to those who conducted the analysis.

If trial participants know they are taking the active drug or placebo, their expectations of effect will be different. If trial staff know who is in what group, they may inadvertently let slip body language or hints to the participants about the group they are in, compromising the integrity of blindness.

But as we will see below, there is a problem in preserving the blindness integrity of nicotine replacement drugs (including e-cigarettes which deliver nicotine) when all trial subjects are people who often have many years experience of negative biofeedback when they are being deprived of nicotine. If they are allocated to the placebo (no nicotine) arm of a trial, they tend to work that out very quickly.

And when RCTs exclude many subjects who in the real world would be considered high-priority candidates for an effective treatment, we have to ask questions about whether the results from such RCTs can be seamlessly generalised to real-world use. As we will see, this is another huge problem for RCTs in smoking cessation.

The Cochrane Collaboration is a global project founded in 1993 to assess evidence of safety and efficacy of therapeutics, interventions and diagnostics. Its tobacco addiction site (Cochrane Collaboration 2020) lists 78 reviews, including RCTs with bupropion, varenicline, various forms of nicotine replacement therapy, antidepressants, anxiolytics, e-cigarettes, and many other drugs and interventions to stop and reduce smoking. It never includes anecdotes in its assessments.

RCTs can compare a drug with another drug used for a similar purpose, with a placebo, or with “usual care”. Usual care in smoking cessation RCTs can be the sort of advice that a doctor or other health professional might ordinarily offer to a smoker when they were not participating in a study. As such advice is often provided as part of responsible clinical practice, especially when a medication is involved, it is important to assess whether the medication has any additional cessation effect on top of the advice or routines to which smokers would normally be exposed in their interactions with a healthcare provider or service.

But when smokers access drugs in real-world circumstances, they are most likely to receive no support or advice (for example, when buying NRT from a supermarket) or only brief, sometimes perfunctory advice when a healthcare provider or pharmacist is too busy to spend much time with a customer. A NSW survey of 700 pharmacies (Paul, Tzelepis et al. 2007) reported that pharmacists claimed to spend an average of five minutes discussing stop smoking medications with smokers, which means that many would spend less time than that.

Smoking cessation medications do not act rapidly like an analgesic, a sleeping tablet, a topical mild local anaesthetic for an insect bite or a decongestant. Their pharmacological effect is far more subtle; often it is “slow release” and imperceptible. If smokers are not given support or detailed instructions about what to expect, it’s understandable that many might quickly discontinue use, thinking that the drug is not working.

Those conducting RCTs can recruit their participants in a variety of ways, some of which introduce important biases into the population being studied. In the smoking cessation field, we often see subjects recruited from sources like quit-smoking clinics, telephone quitline callers, general practitioner and other primary healthcare patients, smoking cessation or vaping website and chat room visitors. Vaping studies sometimes recruit from vaping online chat rooms populated by deeply committed vapers embracing a “vaping lifestyle”. With each of these, we need to ask whether smokers or vapers recruited in such ways are different in important ways to randomly selected smokers or vapers in the population at large. Self-selection bias is very relevant here. We are likely to be dealing with those who are more help-seeking. This may mean they are more motivated to quit than smokers in the general population, and it may also mean they are people with lower self-efficacy (lower confidence in their ability to quit unaided).

Often researchers attempt to address this concern by demonstrating that those who have been recruited into trials are comparable to smokers in the whole population on a range of variables like demographics, smoking history, level of nicotine dependency, intention to quit and so on. But, beyond all these characteristics in a very important respect they are different: they have often taken steps at help-seeking in their hopes to stop smoking. As we will see in Chapter 3, the great majority of smokers who quit don’t seek help to do so when they finally succeed. So those who volunteer to take part in trials recruited in these ways are help-seeking volunteers.

Trial exclusion criteria

For a large variety of reasons, trialists are often unrepresentative of the general population (Schulz and Grimes 2002, Rothwell 2005). This can reflect characteristics of those who are willing to volunteer or consent compared to those who are not. Those running trials will often exclude people from trial participation for a variety of reasons. Those who have language problems are often excluded as interpreters are expensive to add to constrained research budgets. Those with drug or alcohol dependency, serious mental health problems like depression, psychosis or bipolar disorder can also be excluded, as can those with no fixed address, or who move addresses often, are in prison or who have a serious illness which might reduce their life expectancy (and so participation in the study down the track). Those with low motivation to quit can also be excluded.

One study (Le Strat, Rehm et al. 2011) reviewed 54 smoking cessation RCTs for criteria for exclusion and found 25 separate criteria being used across these trials. They then applied 12 of the most commonly used criteria to 4,962 adults with nicotine dependence in the past 12 months from a US national survey on alcohol use (National Epidemiologic Survey on Alcohol and Related Conditions – NESARC) and to a subgroup of participants motivated to quit (See Table 2.1).

They found two-thirds of participants with nicotine dependence would have been excluded from clinical trials by at least one criterion, with 59% of the subgroup of motivated-to-quit smokers also being excluded. Those in such trials are thus very unrepresentative of all smokers wanting to quit. This may result in important participation biases which reduce the applicability of the results to smokers at large, or even smokers at large who want to quit.

Of note in Table 2.1 above is the exclusion from trials of those who have mental health problems (depression, psychosis, bipolar disorder, eating disorders). A 2000 paper in the Journal of the American Medical Association reporting on smoking rates in the 1991–92 US National Comorbidity Survey found that current smoking rates for those with mental illness were 41% (past-month mental illness), 34.8% (any lifetime mental illness) and 22.5% for those with no mental illness. Those with any mental disorder in the past month consumed approximately 44.3% of all cigarettes smoked by this nationally representative sample (Lasser, Boyd et al. 2000). By excluding those with mental illness from smoking cessation trials, RCTs shut out a hugely significant proportion of smokers in the USA.

In an analysis of data from the 2007 Australian National Survey of Mental Health and Wellbeing, having a mental illness in the past 12 months was the most prevalent factor strongly associated with smoking, and associated with both increased current smoking and reduced likelihood of smoking cessation (Lawrence, Hafekost et al. 2013). The situation is likely to be similar in many other countries.

Hawthorne, attention and social desirability effects in RCTs

The Hawthorne effect refers to behavioural change attributable to awareness of being observed or monitored. While the originally described Hawthorne effect has been challenged and debated (Wickstrom and Bendix 2000, Kompier 2006, Berthelot, Le Goff et al. 2011), there is little dispute that participation in a study or trial in itself can cause changes in outcomes that would not occur in people’s lives had they not been involved in a study where awareness of the observations and judgements of others may influence changes in behaviour or response. One of these participation effects is the social desirability effect where some study participants answer in particular ways that they are aware would be considered more socially desirable than others (Persoskie and Nelson 2013). By offering these responses, interviewees might anticipate being thought of more positively by researchers who they might not have ever met before or are unlikely to ever again.

In smoking cessation trials, subjects understand that quitting smoking is the key outcome of interest to the researchers and are likely to assume that the personnel associated with the conduct of the trial have their hopes up that many trial participants who have been allocated to the active drug arm of the trial will quit smoking. In my long experience in the tobacco control field, this is highly likely to be the case. While trial staff will have been instructed to say or do nothing when interacting with participants that would indicate any predictions or hopes for outcomes, it is highly likely that this neutrality is often breached in conversational asides and other often unintentional ways.

When you are involved in a smoking cessation study, lots of attention is often paid to you. You get screened to ensure you are eligible to be in the study. You consent to be contacted, sometimes quite often, by the research team. For example, in the Jorenby et al. varenicline trial, subjects were contacted by study staff 28 times (eight by telephone, 20 in person), of which 18 involved some counselling (Jorenby, Hays et al. 2006). The Niaura et al. trial (Niaura, Hays et al. 2008) involved 24 contacts including counselling on 13 occasions. Walsh’s 2008 review of 12 studies of over-the-counter (OTC), non-prescribed NRT use for their generalisability to real-world conditions of use found study subjects had an average of 7.6 interactions with research staff (range 4–11) (Walsh 2008).

Trialists sometimes attend a pre-trial information session with other study participants, where a sense of teamwork to help science can be fostered. This, and the frequent contact with the research staff who are doing their best to ensure low rates of trial dropout, can combine to create an influential backdrop to using a quit-smoking medication or approach which is very different to the way people will use the same drugs or approach in “real-world” conditions outside a trial.

Trial participant retention strategies

Those running trials routinely put a lot of effort into maximising trial cohort retention rates. If lots of people drop out of the groups being studied, this can greatly compromise the integrity of trials, as important questions can be asked about whether those who pulled out or were lost to follow-up differed in important ways to those who remained in a study across its entire course.

Importantly, real-world studies have found high levels of premature discontinuation of medication use. A four-nation study of 1,219 smokers and recent quitters who had used medication in the last year found most (69.1%) discontinued medication use prematurely (71.4% of NRT users and 59.6% of bupropion and varenicline). NRT users who obtained their patches or gum OTC without prescription were particularly likely to discontinue (76.3%) (Balmford, Borland et al. 2011). A small national cross-sectional 2021 Australian study found 28% of those using cessation medications adhered to the recommended regimens (Mersha, Kennedy et al. 2021).

Evidence from Australia’s Pharmaceutical Benefits Scheme (PBS) which subsidises the cost of drugs to patients, including smoking cessation drugs, shows that many who are prescribed smoking cessation drugs do not take them as directed. Data on the real-world experiences of bupropion and varenicline use indicate stark differences from experiences under research conditions. A 2002 New South Wales study of 151 smokers recruited from 11 general practices who were prescribed bupropion found 84% were taking the drug for a range of 1–12 weeks, with only 19% taking it for or beyond the minimum recommended duration of seven weeks (Zwar, Nasser et al. 2002). Forty-four to fifty percent of patients who received subsidised prescriptions for varenicline failed to commence the last eight weeks of treatment (no data were available to indicate what proportion of the remainder completed the last eight weeks of treatment), in contrast to 12-week completion rates of 68–76% in clinical trials (Walsh 2011). An unknown proportion complete even the first eight weeks after collecting their drugs from a pharmacy. Compliance is much higher in trials: for example, 69% (Niaura, Hays et al. 2008) and 76% of trialists (Jorenby, Hays et al. 2006) completed 12 weeks of treatment.

Yet between January 2008 and October 2009, the Australian government spent $93 million on varenicline prescriptions. This compares with $59 million allocated over four years to social marketing campaigns designed to promote quit attempts in Australia. Given this relatively high spending on pharmacotherapy, it is essential that we are realistic about its potential impact on population smoking prevalence, and whether attention would be better focused on boosting the campaigns known to stimulate mass cessation (see Chapter 8).

Much wisdom has accumulated in professional trial communities about cohort retention. Strategies include reducing any barriers to participation, efforts to build a sense of community and belonging among trialists, follow-up and reminder strategies, and tracing techniques (Teague, Youssef et al. 2018). Community-building strategies can be particularly important, as well as trial staff who have good “people” skills. This often fosters positive attitudes and a sense of responsibility among participants about helping the trial maintain low levels of dropout. They can be made to feel important that they are contributing to the advance of science and the health of communities.

Trial staff often include young investigators whose PhD or research work is focused on a trial. These people have particularly strong motivation to develop good personal relationships with trialists as the work they do will be assessed by their thesis and publication reviewers, and major problems like high dropout rates can be fatal to publication. Someone mildly irritated with the ongoing demands of a study to complete questionnaires, provide biological samples and keep personal data records may feel a sense of “that nice young researcher who contacts me every few months would be very unhappy if I pulled out”. Strategies like sending thank-you, birthday and holiday cards, trial newsletters, supplying trial logo material like caps and T-shirts are also often used.

Trialists are often paid and drugs are free

The drugs used in trials are given free of charge to trialists. Even where governments subsidise the cost of approved prescribed medications, the drugs are only seldom handed out free, such as during special quit-smoking promotions (Miller, Frieden et al. 2005), and to those on very low incomes. Even subsidised drugs can still constitute a significant outlay to those on low incomes. This may inhibit them being used into the medium or longer term by those who feel they need to continue using them.

It is also increasingly common for trialists to be paid for their participation in trials (National Health and Medical Research Council 2019). This is intended to act as both fair compensation for their time and cover any out-of-pocket expenses like travel to the research centre, but may also act as an incentive to continue participation, particularly for those on low incomes or who are unemployed. In real-world, unmonitored or unsupervised quit attempts, smokers are never paid to use quitting aids. These differences may give an extra boost to high compliance across the recommended course of smoking cessation aid use, something that is often far from the case in real-world use.

Blindness integrity problems

In most RCTs, as mentioned above, participants are not told whether they have been randomised to receive the active or placebo (control) drug. This is called subject “blinding”: they are blind to whether they are getting the active drug or the dummy, inert, control drug. Research team members are also often blinded to which treatment or control arm each study participant has been allocated. This is called double blinding and is undertaken to remove the possibility of researchers actively or inadvertently communicating expectations of effects to study participants. A researcher who might have hopes that a particular treatment is efficacious and who knows that certain study participants have been allocated to the active drug may make comments to these patients that suggest to them it is likely that they are on the active drug. Researchers with expectations that successful outcomes of a trial (i.e. where the active drug is shown to be far better than a placebo) might lead to valuable, career-enhancing opportunities may sometimes be tempted to compromise the integrity of the blinding of a trial.

NRT is a strong candidate for a failure in blindness integrity. Nearly all smokers have often experienced interoceptive cues when they are craving nicotine. Here, we need only think of the speed with which many smokers light up a cigarette soon after waking each morning, the once-common sight of smokers rushing to light up after alighting from non-smoking public transport, and standing outside office blocks and restaurants. These sights tell us that smokers are very familiar with sensations that remind them of their need to re-dose with nicotine and the relief and pleasant sensations they experience shortly after doing this. Let’s stay with this pleasure issue for a moment.

The pleasures of smoking?

We sometimes hear smokers talking about the “pleasures” they get from smoking. The picture being painted here is that if you smoke, your days will be filled with particular sensual delights inaccessible to non-smokers. With cigarettes, not only do smokers have the accoutrements for the full public smoking performance (the elegant cigarette, a tasteful lighter, the full hand gesturing and exhaling repertoire catalogued in Richard Klein’s Cigarettes are sublime (Klein 1993), but they are constantly pleasuring themselves around the clock in a way denied to non-smokers who have not woken up to the joys of nicotine.

But what is it that nicotine-dependent people “like” about pulling smoke and nicotine deep into their lungs 87,660 times a year (12 puffs per cigarette x 20 cigarettes a day x 365.25 days)?

In 1994, the New York Times published the ratings of two of the USA’s most renowned addiction specialists, Neil Benowitz and Jack Henningfield, on the relative addictiveness of nicotine, caffeine, heroin, cocaine, alcohol and marijuana (cannabis) (Hilts 1994). They rated each of these on a scale of 1 (most serious) to 6 (least serious) – see Table 2.2.

Both rated nicotine higher on the dependence criterion than all the other drugs. By “dependence” they meant “how difficult it is for the user to quit, the relapse rate, the percentage of people who eventually become dependent”. Nicotine withdrawal also rated high (third behind the often-depicted agonies of alcohol delirium tremens and heroin withdrawal). Both experts rated nicotine fourth behind cocaine, heroin and alcohol when it came to reinforcement (essentially the pleasure given by the drug). But both rated nicotine last on intoxication, behind even caffeine.

Taking all this together, a picture emerges of nicotine-dependent people living in full knowledge of their high dependency, experiencing often unpleasant and insistent withdrawal symptoms when they have not been able to smoke for a while, and being quickly relieved of this unpleasantness when lighting up another cigarette.

Nicotine withdrawal symptoms can include headache, nausea, constipation or diarrhoea, fatigue, drowsiness and insomnia, irritability, difficulty concentrating, anxiety, depressed mood, increased hunger and caloric intake, and, of course, constant tobacco cravings.

Smokers know from the earliest days of their addiction that these feelings can disappear within seconds as nicotine is rapidly transported from their lungs to their brains where dopamine is released and experienced as pleasurable.

Smokers and vapers insist that the pleasure from this release can somehow be experienced independently of the pleasures of the nicotine withdrawal symptoms rapidly dissipating: they keep smoking to pleasure themselves, not to relieve withdrawal symptoms.

So what is the “pleasure” being experienced here? When you have a bad toothache and this is relieved by a strong analgesic, your mood can elevate by the minute as the codeine begins to kick in. We’ve all sat through an execrable movie in a cinema and decided we will endure it rather than disturbing all those between where we are sitting trapped and the end of the row. The agony of watching piles higher and higher, and the eventual escape outside is experienced as pure bliss. But few of us would disagree that while escape from a bad movie or concert is pleasurable, we don’t seek out awful movies or music to experience the pleasure of escaping from them.

The argument that smoking and inhaling nicotine is “pleasurable” is a bit like saying that being beaten up several times every day when you haven’t been able to smoke is something you want to continue with, because it feels so good when the beating stops for a while.

When a smoker is randomly allocated to receive a placebo in an NRT–placebo blinded trial, it seems highly likely that those allocated to placebo will quickly feel very confident that they are not in the active NRT arm. Their body will be telling this to them as it has every day when they knew that they badly needed more nicotine. All this suggests that those in NRT trials who are allocated to placebos (gum, patch, inhaler or lozenge not containing nicotine) will be able to guess this very quickly.

Can smokers guess if they have been allocated to the placebo arm?

In a very important way, RCTs of smoking cessation drugs differ from those involving many other drugs. If you are involved in a trial assessing the efficacy of a drug for a condition which has few if any obvious symptoms, and where the outcome must be assessed by some test, this is very different to a situation where the trialists experience symptoms or sensations which make it very clear to them that they have been allocated to the active drug in the trial, and not to the placebo arm. Trials of blood pressure or cholesterol reducing drugs are good examples of where trialists may not be able to accurately guess their allocation, with the changes only being detectable by a sphygmomanometer (blood pressure) or blood test (cholesterol).

Smokers who are nicotine dependent are very aware of when they feel compelled to light up their next cigarette. They have become thoroughly attuned across sometimes decades of smoking to recognising when they feel nicotine-deprived and to the relief and pleasure they experience when they inhale the first puff of nicotine from a new cigarette. For this reason, there is an obvious cause for concern that smokers participating in RCTs where they are not told whether they have been allocated to the active (NRT) arm of the trial or to the control (placebo) arm can accurately guess to which arm they have in fact (Schnoll, Epstein et al. 2008) been allocated.

In 2004, Mooney, White and Hatsukami published a review of blinding integrity in 73 NRT trials (Mooney, White et al. 2004). Remarkably, they found that only 17 (23%) reported any assessment of blindness integrity, and of these 12 (71%) found that subjects accurately judged treatment assignment at a rate significantly above chance. Of those allocated to placebo, 63.6% accurately guessed they were in the placebo arm, and 57% of those using NRT correctly guessed they were getting nicotine. Only three of the 17 trials which assessed blindness integrity adjusted for it in reporting their results.

A similar concern may apply to other smoking cessation medications. In another study of smokers randomised to take bupropion or placebo (Schnoll, Epstein et al. 2008), participants were asked to guess which they were on (or were not sure). Overall, 55% of subjects guessed their allocation correctly. Compared to guessing “not sure”, those who guessed they were taking bupropion were more than twice as likely to have been randomised to bupropion. Equally, those who guessed placebo were twice as likely to have been randomised to placebo. Importantly, when the authors included treatment arm guess with actual treatment arm allocation in their modelling, the odds ratio of bupropion being more successful than placebo significantly reduced when measured at end of treatment, at both 6 months and 12 months.

It is not difficult to place yourselves in the shoes of a trialist who believes they had been allocated to the placebo arm of a smoking cessation trial. Such a belief would evaporate expectations that what you were taking was likely to have any benefit. This would likely reduce the probability of quitting. Equally, if you believed you had been allocated to the active arm of a trial, this would likely give you faith that what you were using might help you stop smoking.

These considerations add another layer of important difference to what happens when you try to quit smoking with a drug in an RCT with what happens when you use the drug in real-world conditions.

Competing interest bias

There is another important source of bias in smoking cessation studies: the presence of research or researcher funding by industries which have a financial interest in the outcome of the research. The adage that “those who pay the piper, call the tune” is familiar to all. We understand from it that when you are being paid by someone – particularly on a continuing basis – there are expectations that what you produce will be pleasing to those paying you. If you are in the habit of producing information that creates serious problems for your benefactor’s business, it is likely that such funders will stop funding your work.

Cochrane has very strong rules about researchers with competing interests authoring reviews for Cochrane (Cochrane Community 2020). Taking effect from 2020, it now requires that:

• Authors without conflicts of interest must make up at least two-thirds of the author team.
• Both last-listed as well as first-listed authors must be entirely free of conflicts of interest (the first and last authors on papers are commonly those who have most influence on the planning, conduct and reporting of studies).
• Authors of clinical studies that are funded by industry and are relevant to the topic of a review may not be the first or last author of a Cochrane Review. 

In biomedical research, it has long been known that studies with authors who have competing interests are more likely to report outcomes which are helpful to the interests funding the research. This has been documented across a wide range of research areas (Dunn, Coiera et al. 2016) that includes pharmaceuticals, asbestos, gambling, food additives, sugary drinks, alcohol, tobacco and e-cigarettes (Pisinger, Godtfredsen et al. 2019). Biases associated with receipt of funding include selective reporting of outcomes, poorer study quality and reliability, and an increased likelihood that funded authors will interpret evidence as supporting an intervention and megaphone this in publicity surrounding the publication of their research, all to the delight of their funders.

Positive outcome bias

Further, “positive outcome bias” – the tendency for studies reporting positive outcomes to be published and accepted for presentation at scientific conferences – is a recognised phenomenon, with a recent systematic review concluding:

Trial registration (De Angelis, Drazen et al. 2004) and journals making it mandatory that all studies submitted for publication must supply details of that registration are a major step in the direction of allowing greater transparency to others about the protocols and methods used in trials. But it may be that reviewers and especially editors working to publishers’ space limit budgets may be less inclined to publish trials with negative or unclear findings about efficacy (Hopewell, Loudon et al. 2009, Bala, Akl et al. 2013). Rejected papers are often “moved down the food chain” to journals less preferred by authors where they are still published (Nguyen and Chapman 2005).

Many studies on smoking cessation drugs are funded by pharmaceutical companies which plainly have strong interests and hopes that their new drugs, formulations or delivery systems will be found to be useful to smokers. Armed with such evidence, many opportunities open up for marketing and promoting these drugs. Huge increases in sales and profits can follow. The same is obviously true with tobacco and vaping industry-funded research about putative harm-reduction products. Researchers who seek and accept pharmaceutical, tobacco or vaping industry funding also have competing interests in that successful outcomes with treatments manufactured by these sources are more likely to see further funding sent their way than would be the case if the treatment outcome was unsatisfactory. Pharmaceutical companies invest heavily in promoting the findings of favourable trials to both smokers and doctors.

In 2021, a study that received extensive global media publicity, was retracted by the European Respiratory Journal, which had accepted it for publication and published it online before the editors realised that authors on the paper with tobacco industry financial support had not declared these interests at the time of the paper’s submission, as they were required to do. The paper had concluded that smokers had a lower incidence of COVID-19 infection and severity than non-smokers in Mexico (Editors of Eur Respir J 2021), a finding that would have been very pleasing to those in the tobacco industry.

Etter and colleagues (Etter, Burri et al. 2007) assessed whether source of funding affected the results of trials of NRT for smoking cessation. They reviewed 90 trials of gum which were included in a Cochrane Review (52 with nicotine gum and 38 with nicotine patch. Forty-nine studies had received industry support). They found 51% of industry-supported trials reported statistically significant results, compared with nine (22%) trials not supported by NRT manufacturers. They concluded that “compared with independent trials, industry-supported trials were more likely to produce statistically significant results and larger odds ratios.” They also speculated that,

Given all the preceding discussion about the important and varied differences between RCTs and real-world use, it is remarkable that the authors of this paper felt moved to describe a 5% success rate (i.e. a 95% failure rate) as a “considerable public health benefit” when the true real-world impact away from RCTs would have probably been far less.

Another 2010 examination of 107 smoking cessation trials (70 industry funded and 37 non-industry funded) sought to test anecdotal evidence that researchers might attempt to increase the likelihood of obtaining a statistically significant result in trials “by reducing the rate of placebo responding” (i.e. reducing quitting in placebo-allocated participants). They found that reduced placebo responses were responsible for greater than 70% of the variation in placebo arm quit rates and concluded: “These results suggest that there may be important differences in the design and conduct of industry-sponsored trials compared with non-industry trials, which impact specifically upon placebo response rates to increase the likelihood of observing a statistically significant treatment effect” (Greene, Taylor et al. 2010).

A 2019 review of 826 tobacco harm reduction (THR) publications published between 1992 and 2016 found only 23.9% disclosed industry associations. The authors reported that support from the e-cigarette, tobacco or pharmaceutical industries was significantly associated with supportive stance on THR in analyses, and emphasised that public health practitioners and researchers need to account for industry funding when interpreting evidence in THR debates (Hendlin, Vora et al. 2019).

Authors who are funded by industries with a strong commercial interest in particular outcomes are often deeply offended by any suggestion that their work ought to be viewed with heightened circumspection. They often demand of anyone suggesting this that they provide precise evidence of any scientific misconduct, problematic analysis or unwarranted interpretation. Such challenges can be difficult to meet because of the multitude of ways that researchers hoping to “produce” a conclusion can avoid full transparency of all the decisions taken during the course of their research.

“Intention to treat” analysis

When RCTs commence, it is almost inevitable that some recruits who consent to participate drop out of the trials before they conclude. People sometimes move from where they live, change their email addresses and phone numbers, and are lost to follow-up. Some die or become too ill to participate. Some withdraw from the study. This can be for a variety of reasons, some of which have little to do with the key outcomes. But we know that those who withdraw from RCTs on smoking cessation are more likely to be smokers who have relapsed back to smoking. Some of these people may feel embarrassed about their lack of resolve or failure to quit and anticipate awkwardness when interacting with the researchers, even when these researchers may be contracted survey company staff with no personal interest or investment in a cessation intervention succeeding.

If we see RCTs as being a reasonable guide to how a smoking cessation drug performs, and believe they have high relevance for real-world policy and practice, we need to express final outcome measures in terms of the number of participants who started the trial intending to quit smoking. This is known as “intention to treat” analysis. It is highly misleading to conveniently remove all those who dropped out or were lost to follow-up from study participant denominators, because this would artificially flatter success rates by setting aside a subgroup of participants that is likely to include many non-successes.

Most journals insist on data being reported in intention to treat analysis. But some do not. It is wise to carefully look to see whether authors have avoided this and presented flattering data.

Citation bias

Large, attention-grabbing numbers are almost by definition more memorable and repeatable than those not waving look-at-me flags and blowing loud public relations sirens. So when journalists report research findings, it’s perhaps predictable that a big bold number in a report is likely to draw both their and the readers or audiences’ attention more than numbers less startling. This can have important implications for smoking cessation statistics being thrown about in policy debates.

In 2008, I had noticed in the introductory and discussion sections of research papers and in press reports and websites that smoking prevalence among people with schizophrenia was often reported as being much higher than that in the general population. It was frequently described as being “around”, “up to” or “about” 90%, when smoking in the general population in nations like the US, the UK and Australia at the time were in the early 20% region.

At the time, the most recent review of studies reporting on smoking and schizophrenia showed that the pooled prevalence of smoking in people with schizophrenia in published studies across 20 nations was 62%, with a range of 14–88% (de Leon and Diaz 2005). Smoking prevalence of over 80% was found in just 6 out of 42 (14.3%) of the studies, with the numbers of smoking patients in these six studies totalling 484 out of 4686 (10.3%) of all smokers across all of the studies in the review. A widely publicised report on smoking by people with mental illness in Australia recycled the same statement (“People with schizophrenia in particular have extremely high rates of smoking, with most studies finding a prevalence rate of about 90%”) (Chapman 2008b).

This looked to us like a classic case of citation bias playing out. Citation bias is the selective citation of published results to support the findings, arguments or interests of authors and those funding their work (Egger and Smith 1998). Those wanting to draw the attention of journalists, the public and policymakers to the far greater rates of smoking in people with psychosis probably thought, “Let’s find a study with a very high smoking prevalence number in it.”

So where did this “90%” come from? It is likely that it originated from a small but highly influential early paper by Hughes and others, which showed the prevalence of smoking in a sample of people with schizophrenia to be 88% (Hughes, Hatsukami et al. 1986). This finding derived from a sample of just 24 people with schizophrenia living in one US city and attending a hospital outpatient service in 1981–82. As of March 2022, researchers have since cited the paper an amazing 1,433 times, despite its age and very small sample size. One example of how it was cited was in 2008 (27 years after the data collection) in Physiological Reviews where the authors wrote, “it has been shown that people with schizophrenia smoke cigarettes at a very high rate, ~80–90% compared with the 45–70% of patients with other psychiatric disorders and 30% of the general population” (Lendvai and Vizi 2008).

Numbers like this, while not being wrong, are nonetheless very misleading when seen in the context of all other relevant studies. Erroneous assumptions about the near inevitability of smoking in people with schizophrenia may reinforce institutional and clinical neglect of this stigmatised group of people and stultify innovation in targeted support to help this group. While it is possible that the decades-long repetition of the “around 90%” factoid may be motivated by a well-meaning concern to magnify the severity of the issue to attract support or funding, uncritical recitation of statements about misleadingly high smoking rates in schizophrenic patients is often inaccurate and should be challenged.

When critically assessing claims about smoking cessation, it is therefore important to ask whether commonly quoted numbers might reflect citation bias. Big and bold numbers repeatedly used to demonstrate the success of a way of quitting should be traced to the source and compared with estimates in meta-analyses and reviews.

I’ll now turn to two other very common ways of studying cessation: real-world observational studies. We’ll first look at cross-sectional surveys (including time-series studies) and then at longitudinal cohort studies.

Real-world observational studies 1: Cross-sectional surveys

Cross-sectional studies reporting on smoking cessation are those where researchers select participants from the general or particular subpopulations (for example, armed forces, Indigenous populations, school students), and ask them questions about their smoking. These studies can be single “snapshot” surveys or part of a time-series where the same questions are put to different samples selected in the same way annually or every third year, for example. With time-series reports, differences between data collected in different years can be compared, trend lines constructed and statistical tests of significance for differences calculated to provide descriptive accounts of “changes”.

Such repeated, time-series cross-sectional series can be very useful in illuminating some questions about cessation, but have inherent limitations when addressing others. Snapshot and time-series surveys allow us to measure changes in the prevalence of smoking overall or in the prevalence of measurements like the proportion of a population who smoke over time, including changes happening in different groups, but they do not allow any causal inferences to be drawn on what factors are responsible for the changes noted. Other weaknesses include:

• Inability to determine whether an outcome has followed exposure in time or whether exposure resulted from the outcome (reverse causality).
• Inability to measure incidence (new cases of a disease or behaviour like smoking across a specified period).
• Susceptibility to bias due to low response and misclassification due to recall bias when subjects are asked to provide information from the past.

Low response rates in cross-sectional surveys

In recent decades, survey research has been deeply impacted by declines in response rates to surveys, particularly when conducted by telephone. Technological advances (increased use of unlisted mobile phones, use of answering machines and voicemail to screen unwanted calls and caller identification, screening and blocking) have caused increases in under‐reporting. According to the California Tobacco Surveys, response rates fell from 70% in 1992–93 to 51.1% in 1998–99 (Biener, Garrett et al. 2004). However, one study comparing estimates obtained from the US Current Population Survey, which used expensive door‐to‐door interviewing and obtained significantly higher response rates than phone surveys, showed that “under or over‐representation of population sub‐groups has not changed as response rates have declined”. In 2003 the Canadian Marketing Research and Intelligence Association reported that refusal rates to one‐off telephone surveys increased from 66% in 1995 to 78% in 2003 (Allen, Ambrose et al. 2003). By 2018, the US Pew Research Center reported that telephone survey response rates had fallen to a truly dismal 6% (Kennedy and Hartig 2019).

Online surveys can perform better. The popular platform Survey Monkey reports response rates “as high as 20 to 30 percent”, meaning that 70 to 80 percent of those contacted decline (Porter 2021). Incentives can boost responses. A University of Michigan series of experiments in increasing web-mailed survey completion rates between 2011 and 2012 found “fresh” first-time requests to do an online survey saw 17.4% completed compared with 34.5% when offered a US$5 incentive. Recontacting people saw 50.3% respond with no incentive, rising to 70.1% with an incentive (Suzer-Gurtekin, McBee et al. 2016).

Self-selecting, motivated samples vs. whole population randomly selected samples

It is always important to look at studies of smoking cessation for details of the study population involved and how they have been recruited. Those promoting particular methods of quitting will often gloss over such details in the rush to highlight headline results. But results from surveys of clearly biased populations of smokers are of almost no value in extrapolating findings to all smokers. An online survey of over 19,000 people who completed a questionnaire on an electronic cigarette advocacy website reported that 81% of smokers had completely abandoned smoking (Farsalinos, Romagna et al. 2014). This finding was lauded by vaping advocates on social media who seemed to believe that a survey completed by like-minded dedicated pro-vaping advocates had any relevance for what was happening with vaping in the wider population. It was rather like a survey of members of a whisky appreciation club on their drinking habits being seen by some as a reasonable guide to whisky drinking in the whole population.

Many studies of smoking cessation are conducted with study populations of smokers who have come forward to participate in a clinical service providing smoking cessation interventions. Others involve outreach efforts to attract smokers by advertising the availability of self-help materials or minimal, low-intensity support like supportive phone calls, booklets, online quit-buddy support networks, or quit apps. In all these cases, those being researched are smokers who have an intention to quit which has motivated them to seek help or enquire about self-help materials that are unlikely to require them to attend a sometimes time-consuming quit-smoking service. An early review of 10 prospective trials of smokers intending to quit who had responded to advertisements offering self-help materials or seeking intending quitters who wanted to try to quit without assistance, found a 13.9% quit rate at 12 months (Cohen, Lichtenstein et al. 1989).

By contrast, Australian researchers Baillie, Mattick and Hall in 1995 published a meta-analysis of the available literature on the rate of smoking cessation in randomised controlled trials which involved control groups who were deliberately given no smoking cessation treatment or given only “usual care” by staff at health facilities where they were recruited into the studies. Fourteen such studies were found and results pooled in the meta-analysis. Across the studies they found a 7.3% quit rate over a 10-month period, nearly half that found in the analysis of smokers who were intending to quit, described above (Baillie, Mattick et al. 1995). The Australian authors’ finding was based on the longest follow-up period reported in each of the 14 papers they reviewed, which varied. However, this 7.3% figure was not a finding of continuous abstinence across that period. Some smokers may have been continually abstinent, while others may have relapsed during the follow-up periods but may have quit again and were not smoking when the final, longest follow-up was conducted.

Comparisons of quit rates between those given assistance to quit and matched smoking controls not given any special assistance allow us to consider the size of any additional quit rate above and beyond that which might have been expected to occur among a group of smokers not offered assistance. So, taking the two studies just described, 13.9% less 7.3% gives an increased absolute quit rate of 6.6%, some 90% above the background unassisted quit rate. For decades, those promoting assisted cessation have spun this difference via the highly memorable claim that research shows that assisted cessation “doubles your chances of quitting”.

But for the many reasons discussed so far, there are strong caveats that should be applied to this glib comparison.

A 2016 paper in Addiction reporting on a cross-sectional survey of 27,460 Europeans aged 15 and over in all 28 EU nations concluded that an estimated 6.1 million citizens had quit smoking with the help of e-cigarettes (Farsalinos, Poulas et al. 2016). The take-home message highlighted in publicity about this study was that there were over 6 million people in Europe who used to smoke and reported that they now no longer did on the day they answered the questionnaire, thanks to taking up vaping. But this paper was savaged in a response which made the following criticisms (Maziak and Ben Taleb 2017):

1. It is impossible to know how many of those who claim that they have stopped with the aid of e-cigarettes would have stopped anyway, and how many of those who used an e-cigarette but failed to stop would have stopped had they used another method.
2. With smoking status being highly unstable (smokers quitting, then relapsing, then quitting again) the cross-sectional design could never account for known high levels of relapse in those who make quit attempts.
3. The study’s key question (“Did the use of electronic cigarettes or any similar device help you to stop or reduce your tobacco consumption?”) can result in the misclassification of short cessation periods as full cessation.

Real-world observational studies 2. Longitudinal cohorts

Longitudinal cohort studies involve a group of people being studied across successive time intervals to measure changes or transitions in outcomes of interest. In smoking cohort studies, we find studies looking at uptake of smoking and vaping, relapse back to smoking, attempts at quitting and how long these last, further attempts at quitting after failed attempts and dual use (smoking as well as vaping by individuals). Like snapshot cross-sectional studies, cohorts can recruit randomly drawn samples from the general population, or focus only on special populations.

The important difference between cross-sectional “snapshot” studies and longitudinal cohort studies is that with the latter, the same individuals are followed, repeatedly questioned at several points in time and often biochemically tested for signs of smoking (typically, exhaled carbon monoxide levels or salivary or urine cotinine, a metabolite of nicotine). Enduring cohorts can thus have many data points across the duration of the study. While snapshot prevalence studies often include questions about past quit attempts and their duration, longitudinal data for the same individuals allow direct analysis of stability or transitions in self-reported smoking and quitting attempts. As will be discussed, serious problems with accurate recall of quit attempts and number of cigarettes smoked are far more common in snapshot cross-sectional studies that ask about past smoking than in cohort studies where current status is typically recorded for recent periods.

Moreover, Hughes et al. note that, unlike RCTs, most cohort samples of smokers “have few inclusion criteria and most are of smokers not enrolled in any formal treatment program” (Hughes, Peters et al. 2011). They are therefore important data sets for considering real-world quitting transitions.

Because it is critical to the very core of cohort studies’ value that they have high respondent retention rates, those managing these studies generally invest in ensuring that loss to follow-up is as low as possible. Cohort studies invariably experience attrition or loss to follow-up problems where those in the study either cannot be located at later phases of the study, or decline to continue to be involved. There are statistical methods that can be used to adjust for attrition but these are often not undertaken, and attrition rates of greater than 20% pose serious threats to validity (Bankhead, Aronson et al. 2017).

Relapse

When we learn that someone has quit smoking, this can mean many different things. At one end of the spectrum, it can mean that a person has made an effort to stop either alone or with assistance and a very short time later (for example, at the end of the last session of a multi-session stop-smoking course) declares that right now, they are not smoking. It might mean that the day after their planned quit day, they have not had a cigarette for a day. This is often referred to as an “end-of-treatment” result and can often be found on commercial quit-smoking-quick websites. Today such information would struggle to find publication in anything but a pay-to-publish junk journal, because of decades of knowledge about relapse or remission back to smoking.

While there are some people who try to quit smoking and succeed permanently on their very first attempt, this is unusual. By far the most common pathway to permanent quitting is for smokers to have several and sometimes many attempts at quitting, only to relapse back to smoking for weeks, months or years and then repeat that cycle until their final, successful quit attempt. This has enormous implications for any critical appraisal of data in research reports on quit rates.

Relapse has been much studied across several decades. A 1994 Californian study found 71.1% of quit attempts lasted just two days before smokers lit up again; 58.5% last at least three days; 39.2% for a week or more; 19.6% for one month; and 14.1%, for three months (Gilpin and Pierce 1994). One of the most cited papers looking at this phenomenon is by US researcher John Hughes and colleagues from 2004, “The shape of the relapse curve and long-term abstinence in unaided quitters” (Hughes, Keely et al. 2004). This paper reviewed the paucity of prospective studies of people trying to quit without assistance (just two studies), and studies which had no-treatment control groups (five studies) available at that time. Summarising earlier work, the authors stated that “3–5% of self-quitters achieve prolonged abstinence for 6–12 months after a given quit attempt” (See Figure 2.2).

The data used in Figure 2.2 date from studies published in the late 1980s and the 1990s, and so could not reflect the more recent experiences of people trying to quit smoking in eras when modern tobacco control policies like smoke-free laws, significant tobacco tax increases, total advertising bans, graphic health warnings, extensive public awareness campaigns, plain packaging and the growing denormalisation of smoking combined to create an environment that is very different to that typical of earlier decades.

So what do more recent data show? A 2012 paper analysing seven years of data from 21,613 smokers recruited into the International Tobacco Control (ITC) four-country study (Australia, Canada, UK, USA) found that 40.1% of smokers reported quit attempts in the past year, with an average of 2.1 attempts. When the authors adjusted for recall bias (see below) and only included quit attempts made in the last month, the average fell to one per year (Borland, Partos et al. 2012a). This seems a peculiarly narrow, stringent window. Intuitively, it’s hard to imagine that anything deserving to be seriously called a quit attempt would decay from memory after just one month. Another paper from the same study estimated that by the time the average smoker reaches 40 years, they will have made 40 attempts to quit (Borland, Partos et al. 2012b).

A British study of 1,578 former smokers who had quit for at least a year between 1991 and 2006 participating in the annual British Household Panel Survey, and followed up for a mean of 5.2 years after their initial one-year smoking abstinence had the authors estimate that 37% would relapse within 10 years. Increased length of abstinence, increased age, being married, being educated to degree level, and having a high frequency of general practitioner visits were significantly associated with a lower risk of relapse, while higher relapse rates were significantly associated with mental health problems and having a partner who started smoking (Hawkins, Hollingworth et al. 2010).

Most estimates of the average number of quit attempts made before final, long-term success derive from cross-sectional studies where smokers are asked to state how many lifetime attempts they have made, or how many in a more recent period, such as the past 12 months. As I will discuss below, recall of quit attempts and indeed agreement about what a “quit attempt” actually means are problematic. To reduce this problem, in 2016 Chaiton and colleagues looked at how many quit attempts smokers make by questioning the same 1,277 Ontario smokers who had reported a subjectively “serious” quit attempt during the last year. They re-interviewed these smokers every six months for up to three years. This enabled the researchers to validate more recent answers with those supplied by respondents in former years.

In a complex paper befitting an apparently simple but in fact very challenging question, they used four different approaches in their estimations, ranging from those which assumed that quit attempts reported in recent years would have also applied in more distant years to those which adjusted for various expected reporting biases and concluded, “The estimated average number of quit attempts expected before quitting successfully ranged from 6.1 under the assumptions consistent with prior research, 19.6 using a constant rate approach, 29.6 using the method with the expected lowest bias, to 142 using an approach including previous recall history” (Chaiton, Diemert et al. 2016).

Their open-access paper explains in great detail the strengths and limitations of each method, and their reasons for settling for the “Life Table, Observed Quit Rates” method which suggests that a smoker tries to quit on average 30 times or more before successfully quitting for one year or longer.

Finally, no discussion of relapse in a book looking at unassisted cessation could avoid considering a letter published in Addiction in 2012, authored by five giants of smoking cessation research (Hughes, Cummings et al. 2012). They were writing to criticise a press release about a case-control study in Tobacco Control where the authors concluded that “NRT is no more effective in helping people stop smoking in the long term than trying to quit on one’s own” (Alpert, Connolly et al. 2012).

The Alpert et al. paper was important because it provided data on a question that is probably front of mind in most smokers wanting to quit and considering using NRT or a medication to do so: “How successful will this treatment be in getting me to stop smoking permanently?” I doubt that there would be many smokers who, in considering any given smoking cessation treatment, would ask, “How successful will this treatment be in getting me to stop smoking for a few days, a few weeks, a few months or even for a year?” Most would be thinking that, as they were making the effort to quit, those recommending a treatment would understand that most smokers would be interested in what the evidence showed about permanent quitting, not just temporary cessation.

The five authors wrote, “the [Alpert et al.] study tests whether the use of NRT in the distant past (up to 2 years prior to the survey) prevents relapse during the subsequent period [of] years after use of NRT. Studies have found that the therapeutic effect of NRT is concentrated during the weeks it is being used, and after this the rate of relapse is similar between NRT and control conditions. Thus, NRT does increase long-term abstinence, primarily by increasing the initial number of quitters” [my emphasis] (Hughes, Cummings et al. 2012). Here, they referenced a 2006 meta-analysis of 12 RCTs (Etter and Stapleton 2006) in support of their claim about the superiority of NRT to no-treatment in both short- and long-term success.

For all the reasons discussed in Chapter 2, we need to be very cautious in extrapolating NRT results to real-world results. And the Hughes et al. letter omitted to mention that the Etter and Stapleton meta-analysis stated “initial relapse after one year has the effect of diminishing the number of ex-smokers that can be ultimately attributed to NRT”. They wrote that the frequent use of 6–12 month cessation data in reviews and treatment guidelines “will overestimate the lifetime benefit and cost-efficacy of NRT by about 30% … the long-term benefit of NRT is modest”.

So from this exchange, the best complexion we can put on the question of how good NRT is in keeping smokers abstinent into the longer term (here two years), is to say that NRT fares better than unassisted quitting while it is being used, but that both strongly fade as the months and years go by, to the point that there is no difference at two years. Smokers’ curiosity about whether they will fare better in the long-term with a course of NRT than with unassisted cessation therefore looks like a “no”.

Recall bias

A 2012 International Tobacco Control (ITC) Four Country study paper discussed earlier by Borland and others about systematic biases in cross-sectional studies of smoking cessation argued that these types of study are likely to underestimate the effectiveness of smoking cessation aids because of recall bias (Borland, Partos et al. 2012a).

The paper reported that those using stop smoking medications (SSMs) remembered quit attempts from further back than those attempting to quit unassisted. This finding was surely highly predictable. When you take SSMs (depending on the drug), you sometimes have to go and see a doctor to get a prescription. You then have to go and buy the drug, sometimes carry them around (as with nicotine gum or inhalers), and are meant to take them each day over a sustained period. There are therefore many cues to remembering that you took them compared to trying to quit without using any of these products. Unassisted “attempts” are often little more than an empty ritual in the style of “This weekend, I’m really going to stop”. So when relapse occurs hours or days later, all rationalisations about “Well, I wasn’t really serious” probably dissipate quickly, never to be recalled.

I certainly don’t recall how many times in the last two years that I decided to lose five kilograms and so I just avoided alcohol during the week, tried smaller meal portions, walked to the shops a little quicker and walked 30 minutes to and from my regular tennis instead of driving, and then went back to my usual lower level of daily activity. But I certainly remember my “assisted” attempts: going on the 5:2 diet for three months, signing up for a gym, increasing my tennis to several times a week, and buying an indoor exercise bike and a rowing machine. If a researcher had called me, I would have recalled that the bike sat in a corner of the TV room unused after about 10 uses – I would have recalled the failed “assisted” attempt but probably not all the half-hearted unassisted ones.

But if your primary interest is in what approaches across whole populations produce the most permanent ex-smokers, failed attempts are not what is most important. The key question is what method was used at the final permanently successful attempt when ex-smokers are questioned: it’s not about quitting attempts, but about the method used when a smoker actually finally quits. I cannot imagine any smoker who quit assisted or unassisted failing to recall how they quit, particularly when the final attempt was recent.

This 2012 Borland, Parthos et al. paper framed its research question against a background of questioning from some (a paper by me was referenced) about usefulness of SSMs in quitting when the real-world cohort data have often shown disappointing SSM effectiveness. The data used in their argument were on recalled quit attempts, not quit attempts that actually succeeded into the long-term. So I think the authors used a sleight of hand here: they drew attention to something that is not being disputed (that smokers make many attempts to quit). They thereby hoped to provide evidence about the question of the comparative head-to-head quit rates obtained from assisted vs. unassisted cessation attempts when these attempts were successful.

Given the variable reliability of recall of quit attempts, and indeed questions about what ought to be even described as a quit attempt, it seems sensible to approach the issue from a different direction. Instead of asking, “Are smokers more likely to quit by using assistance than by trying to quit unassisted?” we ask, “What if we take 1,000 former smokers who have not smoked for 12 months, and ask them what method they used in their final, successful attempt?”; we move from an individual to a population perspective in answering the question about what approach to quitting yields most long-term ex-smokers. And the answer has always been unassisted cessation.

Indication bias

When we see data that show real-world use of quit-smoking aids not performing as well as they did in many often highly publicised clinical trials, we tend to see authors and commentators arguing that this is unsurprising because of what is termed “indication bias”. Saul Shiffman described it this way:

So if this assumption is correct, it would follow that in a large sample of smokers trying to quit who are recruited into a study from the community, there will be a higher concentration of heavily dependent smokers in those using NRT, a prescribed drug or by vaping than there will be in those who are trying to quit without any aids or professional assistance.

Being a more dependent or addicted smoker has long been known to predict relapse compared with less addicted smokers who try to quit. So when the proportions of successful quitters in both groups (aided vs. unaided) are compared, no one should be surprised if, at first blush, it looks like unassisted quitters did as well as or even better than those who were using assistance.

Appreciating the impact of this confounder, some studies take care to control for this form of bias. Here, researchers can include questions designed to measure the degree of addiction or heaviness of smoking (Chaiton, Cohen et al. 2007) in their battery of questions. If it emerges that, in fact, there are indeed significant differences in these variables between the assisted and unassisted groups (Borland, Partos et al. 2012a), weighting can be introduced into the statistical analysis of the outcomes for the two groups to produce an adjusted “apples with apples” comparison.

What we see happening here is a quiet refinement of expectations for assisted cessation methods. Starting from an argument that all smokers would benefit from pharmaceutical and professional assistance, and explicit warnings to not try to quit cold turkey (see Chapter 5), we then see statistical techniques applied in cessation studies to mask the fact that more heavily nicotine-dependent smokers will likely have poorer quit outcomes than less dependent smokers; often worse than control-group comparators who are commonly those who attempt to quit unassisted.

Despite these heavily qualified outcomes, bald public statements about the performance of various forms of assisted smoking cessation being superior to unassisted quitting remain very common. Part of the explanation for this is the sheer brevity of most health communication. Public messaging about smoking cessation is often highly constrained by cost factors in paid advertising (15 or 30 seconds being the most common duration) and in sound bites used by news bulletins. My work on the parameters of Sydney television news items found the average duration of comment by a person appearing in a news item is just 7.2 seconds (Chapman, Holding et al. 2009).

Much discussion of quitting has long been infected with commercial agendas. Clinical and research consultants to the pharmaceutical industry are often given media training with product-friendly talking points. So when a smoker sees an advertisement for a quit smoking drug or hears an expert being interviewed on television, what survives are headline messages, slogans and sound bites like “twice as effective” or “double your chances of quitting”. Smokers won’t be told that hidden behind such breezy claims are the sort of caveats I discuss throughout this book. “Doubles your chance of quitting” is a sales pitch, not a relative risk statement.

Ways of quitting smoking

I refer in this book to both assisted and unassisted cessation. It’s important that this distinction is clarified. In a 2015 review I conducted with Andrea Smith and others of unassisted smoking cessation in Australian research literature, we defined assisted smoking cessation as:

This is the approach I have again adopted in this book. It means that by “assisted cessation”, I’m including the following ways of attempting to quit: pharmacotherapy (NRT, bupropion, varenicline or other prescribed drugs); vapourised nicotine products; behavioural individual or group counselling whether delivered through a dedicated smoking cessation service, by a HCP in formal sessions, the use of a telephone quitline, an online website or phone app over several sessions; the use of a guided book such as the Allan Carr program; and complementary and alternative therapies (e.g. hypnosis and acupuncture).

What, in my view, cannot reasonably be considered formal assistance is the brief mention of the desirability of quitting by a healthcare practitioner during the course of consultation for another purpose; a one-off curiosity call to a quitline; reading an article every now and then about quitting smoking online, in a magazine or newspaper; talking with an ex-smoking friend or acquaintance about how they quit; seeing smoking cessation advertisements on television, noticing graphic health warnings on cigarette packs or many other encounters with routinely “smoking denormalised” signage, customs or policies (such as working in a non-smoking building).

This is because all these activities are extremely commonplace, unavoidable, often only crudely quantifiable and rarely researched in any detail. Collectively, they are all part of what we might call the “background environment” of factors that together, acting synergistically, bring smokers to points in their smoking careers when they decide to try to quit (see Chapter 8). To sweep them all into a very broad definition of “assisted” cessation would mean that all quit attempts would have to be regarded as assisted. There is no smoker in any country with graphic cigarette pack warnings, who has never seen one of these. There are possibly smokers who have never been in a conversation with a friend, family member or workmate who has quit and wanted to talk about it, but they would be very rare.

A very recent paper from the International Tobacco Control (ITC) Four Country Smoking and Vaping Survey looked at the different quit methods used at the most recent quit attempt in samples from the USA, UK, Canada and Australia in the last 24 months (Gravely, Cummings et al. 2021). The most common method used was “no aid” (i.e. unassisted cessation) named by 38.6%, followed by NRT (28.8%) and NVP (nicotine vaping products) at 28%. Many used different methods in combination at their latest quit attempt. In total, the authors stated that 61.4% made an aided quit attempt.

However, a footnote in the paper unpacks a residual category named “other support” which a substantial 16.5% of respondents reported using. This category included “mobile apps, cessation website, pamphlets/brochures, books, acupuncture, laser therapy, hypnosis, support groups, social media, cognitive behavioural therapy, meditation/mindfulness.” Some of these (cessation websites, pamphlets/brochures, books and social media) may well have involved quite superficial and fleeting engagement. A person who responded “cessation website” or “social media” for example, could have just briefly browsed such sites or could have fully engaged with them over a sustained period. The paper provides no way of knowing. So it seems reasonable to be sceptical that all 16.5% of the sample were indeed meaningfully being seriously “aided” in their most recent quit attempt, rather than casually browsing material related to quitting smoking.

Success rates versus intervention and policy reach

When we look at research evaluating smoking cessation policies and interventions through a population-wide focused lens, the two most elementary questions we need to ask concern the long-term success rate of an intervention or policy and their “reach” into the population at large, here meaning all smokers. In this chapter I’ve looked at most of the caveats and qualifiers that need to be borne in mind when critically appraising claims about success. But so often claims about the great promise shown by a quit-smoking method or a policy initiative which might stimulate lots of quitting activity fail to look openly at the considerations of how many smokers are ever likely to actually use a method or be exposed to a policy.

A method of quitting shown experimentally to be spectacularly successful but which would be hugely expensive to manufacture or require a large professional workforce to deliver it (where such a workforce would often not even exist) would invariably have inconsequential uptake in the real world, regardless of how efficacious it might be. For example, if it could be shown experimentally that paying smokers several thousands of dollars to quit resulted in pleasingly high rates of permanent quitting, the very first questions needing to be asked are, “How many employers would ever be willing to implement such a policy?” and “Would any government ever adopt such a policy?”. Also, many non-smokers might reasonably ask, “So, do I also get a few thousand dollars for never taking up smoking? This seems very unfair”. I look at paid incentives further in Chapter 4, along with perhaps the world’s largest experiment in government funding of labour and pharmacological intensive smoking cessation, the decade long English embrace of quit-smoking centres.

Flipping this the other way, we can also see that an intervention with very low rates of success which becomes part of the day-to-day world of every smoker might nonetheless have an important public health impact across the whole population of smokers. Policies like graphic health warnings on cigarette packs, tobacco tax rises which impact on retail prices, and the introduction of policies which prevent smoking in workplaces and public places where smokers might otherwise be smoking lots of cigarettes each day reach every smoker.

Consider a hypothetical stop-smoking intervention which has been shown in trials or pilot studies to cause 20% of smokers to permanently quit. Imagine it was taken up by a national company employing 10,000 smokers, of which 20% participated in the intervention offered at work. This means that 400 smokers would quit.

But then consider the rollout of a rotated set of see-and-never-forget graphic pack warnings. Imagine a post-rollout evaluation of randomly selected smokers in a national population of 3 million smokers being interviewed about possible impacts of the warnings on their intentions to quit. Imagine that only 5% of smokers interviewed reported that the warnings had stimulated them to make a serious quit attempt, and that a derisory 2% of those had succeeded and were comfortable in attributing their decision to quit as being a response to the warnings – a straw that had broken the camel’s back.

So with all smokers seeing the graphic warnings every time they notice or reach for their pack, here we have 3 million x 5% being stimulated to make a quit attempt (150,000) x 2% who quit (3,000 ex-smokers), far more than our hypothetical stop-smoking intervention described above.

Moreover, in Chapter 4, I’ll look at how common it is for researched interventions like trials of workplace quit-smoking programs to be “upscaled” to a situation where they proliferate in wholesale ways throughout communities. Spoiler: they very rarely do.

All this also assumes huge faith in reductionism – the belief that, when it comes to explaining population-wide changes in complex health behaviours like diet, physical activity and smoking, it is possible to forensically and neatly reduce explanations of causal factors to individual influences. In Chapter 8, I’ll take this up again.