Science Lesson: Immortal Person-Time Error And Stock Vs. Flow Errors
“Immortal person-time” is a favorite topic in epidemiology courses, even though most students might never encounter this error in practice. By contrast, the critical difference between stock and flow is almost never taught, though it has similar implications that arise more frequently. Both are technical points about research methods that will probably make some readers’ eyes glaze over. But it happens that both are critical to understanding some recent errors in anti-vaping junk science.
A study has an immortal person-time error when someone could not possibly have had an important outcome during some period, but that period is still counted in the statistics. Consider a study looking at the life-expectancy effect of kidney donation. The researchers contact a group of kidney donors to recruit subjects. For each donor they then recruit a control subject from people at the same hospital with a minor ailment or as a visitor, matching the donor’s age, sex and general health. They then monitor everyone and compare their survival.
The problem is that from the time of the donation and to the start of data collection (sometime after the donor volunteers to join the research), it is impossible for the donor to have died. No one was really immortal, of course, but anyone who died during that period could not have been in the study. The study gives the kidney donors credit for living for a period that they could not have died, but the controls are mortal from the day they enter the study. This biases the measured longevity in favor of the donors.
The bias is definite, goes an obvious direction and is fairly small, making this a favorite topic to teach: Epidemiology teachers like to pretend that the typical error in the field is a minor quantitative bias that can be recognized by a careful reader, rather than a complete train wreck of bad methodology or a hidden bias like model shopping. But occasionally it is possible to botch a study design so badly that this bias is indeed a train wreck.
One example is the previously critiqued paper by Stanton Glantz and colleagues that compared the vaping history of a 2014 cross-section of former smokers and current smokers. Among its several fatal flaws was a version of immortal person-time: People who quit smoking before e-cigarettes became popular were “immortal” with respect to vaping — they could not have vaped before they quit smoking. (They could, of course, have tried vaping long after quitting smoking but, as noted in the previous critique, this actually represents a further flaw in the study.) Counting all these “immortal” former smokers as if they were evidence of vaping not being useful makes the analysis in the paper complete junk. This error created the illusion that vaping causes someone to be more likely to keep smoking (because those who vaped were more likely to still be smokers). In reality, it merely meant that most former smokers in 2014 never had a chance to try quitting by vaping.
The more fundamental problem here is confusing stock and flow. Water coming out of a faucet is the flow (a rate, measured in quantity-per-time), whereas the quantity of water in the sink is stock (just a quantity). Most epidemiology is about flow, thus this phrase “person-time.” Therefore, most outcomes are best understood in terms of rate, such as “chance of someone getting bladder cancer each year” rather than “chance of getting bladder cancer ever.” A lot of study methods look at stock and try to draw conclusions about flow, and the correct techniques make this possible when the past is similar to the present. But Glantz did not use the correct techniques, and obviously the past is nothing like the present in terms of vaping.
As noted in the previous critique, that study would still have been fatally flawed even apart from this error. One of the other errors is a stock-flow error of a different sort. It is the same error that pollutes the literature with other “vapers are less likely to quit” papers, the ones that allowed the recent National Academies report to pretend this was possible.
To illustrate this problem, consider the population of nursing homes. Most people admitted to nursing homes (the flow) are entering for what is expected to a short stay, recovering from an injury or hospitalization. But more than half the people in nursing homes at a given time (the stock) are there for long-term care, typically for the rest of their lives. The reason for this difference should be obvious: those who are there for long-term care stay a long time, and thus are a disproportionate part of the stock compared to the flow.
Putting this in terms of smoking cessation, most smokers who start trying to quit (flow) quickly succeed or decide to stop trying for a while. They might try one cessation aid, or maybe none at all. Meanwhile, the population who are actively trying to quit smoking (stock) includes a lot of people who have tried and failed, often many times, but are still pursuing it. Like the long-term care residents of nursing homes, they are most of the “currently trying to quit smoking” stock even though they are a minority among the flow who ever try to quit. Those individuals — who have demonstrated that they are less likely than average to quit smoking after deciding to try — try many cessation aids and techniques, which now is likely to include vaping.
A study that samples from the stock of those currently trying to quit smoking, perhaps by recruiting people who call a quitline or sign up for a cessation program, will include a disproportionate number with a history of failed cessation attempts. If their history of vaping or any other smoking cessation aid, along with their near-future quit rate, is compared to that of everyone who quits, it will create the illusion that all the aids are detrimental. Most of those who actually quit did not try them, but the sample is biased toward people who have ever tried a particular aid and failed (and thus will probably fail again). This is an invalid comparison of stock and flow. But more subtly, even if the comparison is within the sample, it still has the same problem. Some of the smokers in the sample, to whom the accumulated stock of failed quitters are being compared, are effectively drawn from the flow (less likely to use an aid, more likely to succeed).
For the case of nursing home residents, this would be equivalent to looking at whether being discharged, versus dying in the nursing home, is associated with ever experiencing some event there. All such experiences — whether harmful, like a fall, or something beneficial or benign, like participating in enrichment activities — will be associated with dying there. The long-term residents are more likely to ever experience it because they have more time to do so. The error would be worst if all ever-residents were compared. But it would still exist if a sample of residents at a given time was collected, since such a large portion of them are long-term. This is exactly the error made (perhaps intentionally) by the authors of the many studies that seem to suggest that having tried vaping makes someone less likely to be successful in an attempt to quit smoking.
There is no reason why vaping (or any other cessation aid) might actually impede smoking cessation, which makes it obvious that study result suggesting that is wrong. Those who claim that studies demonstrate this offer no plausible story for how it would even be possible. But it is still useful to understand exactly what errors in the studies created this illusion.
Follow Dr. Phillips on Twitter