Statistics: Chance and Blinded Controlled Trials

“Why should I have a flu shot when I had one last year and still caught the flu later,” said my friend in frustration. “Even worse, I understand you can get the flu from the shot!” I’ve been in the public health/doctoring business for nearly 50 years, and that lament hasn’t changed much, even though the influenza vaccine’s efficacy has improved a lot. Much in life is subject to chance; most medical and public health interventions, like flu shots, have been shown to improve those chances without negative side effects.

The answer to my friend’s question of why get a shot if “it didn’t work last year”, is that, like most risk reductions, it doesn’t work 100% of the time and/or doesn’t always completely prevent its illness target. Thus people who have had flu shots still have a 60% chance of getting influenza if exposed, but the glass is nearly half full because they do have a 40% chance of not getting it. And even if they do contract the flu, they have a much reduced chance of missing work (healthy adults) or of dying (especially the very young and very old) from it.  Most people don’t appreciate these gradations of benefits; many, unconsciously or not, expect full prevention or cure from a treatment, and are surprised or frustrated when the result is less than perfect.

And the efficacy of the flu shot varies from year to year. Last year it was only, on average, 25% effective rather than the usual 60% on average. That was because the US Center for Disease Control (CDC) has to guess in May or June which variants are most likely to be around the next winter. It takes 4 months for manufacturers to produce the vaccine after being given that direction. Even though the vaccine contains 3 (trivalent) or 4 (quadrivalent) influenza virus types, sometimes the CDC guess/estimate is wrong or imperfect. They too have to play the odds in making those estimates. That’s what happened last year; the CDC guessed wrong on which strains of influenza would be the greatest problem that year so the shot was less effective, though it still helped.

So we shouldn’t let the perfect result be the enemy of a good result. Not dying from the flu is still better than the alternative, even if you get sick with it. How do we know when the intervention benefits outweigh letting nature take its course? We know when we have statistics to guide us. And they have to be good, well done statistics, not ones like the Roman physician Galen cited, I hope jokingly, when he said: “All those who drink of this remedy recover in a short time, except those whom it does not help, who die. Therefore, it is obvious that it fails only in incurable cases.”

The so-called blinded controlled scientific trial, invented in the past 100 years, has allowed us to measure the efficacy, or the lack thereof, of many previously used treatments by comparing outcomes in two groups of people with a condition (or lack of it in the case of flu shots), and administering them two different treatments, one of which is usually nothing (the “controlled” part). Hundreds and sometimes thousands of patients are studied; neither patients nor doctors know which treatment is given, thereby avoiding biased interpretation of symptoms by either (that’s the “blinding”). As a result of such trials, we now don’t: use leaches or bleeding; aspirin in little children with fever; bed rest for childbirth, back pain, or heart attacks; enemas for nonspecific symptoms; and many more examples

But many people, like Mark Twain, who complained about “lies, damned lies and statistics,” are more confused than helped by the statistical analysis of medical tests or treatments. They either should work or not work, in their minds. They get frustrated, like my friend, when the intervention works in only a percentage of the time or only partly. But we and our health are part of the natural world; we are not surprised when weather predictions are wrong some days and imperfect on others. Nor should we be surprised when flu shots prevent the flu in only some of us some of the time, and ameliorate it for some but not all; at least they are better than the alternative of no flu shots. President Lincoln once cautioned, “You can please some of the people all the time, all the people some of the time, but never all the people all the time”. So too in medicine and public health.

Oh, and my friend’s challenge that flu shots “give you the flu” is not true, but rather a misinterpretation of side effects. A flu shot is a killed vaccine, and hence cannot infect you. But the way it works is that it contains broken down pieces of the influenza virus to which your body reacts by engulfing them into your white blood cells, and then delivering them to other white cells, which make antibodies to those bits of the virus. Those antibodies kill live viruses with the same bits on their surfaces when you are exposed in the future. That process makes you feel a little “icky” (local soreness, aching, and even low grade fever) because enzymes are released from your white cells as that process happens; those enzymes cause some transient inflammation in your body. Such symptoms are side effects, which some people, but not all, can feel. Yet another layer of subtle ambiguity; sorry!

Dan Onion, MD, MPH

Mt. Vernon/Vienna Health Officer

293-2076; dkonion@gmail.com

January, 2016