SURVIVORSHIP BIAS

My inclination is to stop here, but my bias propels me to continue, as with most of humanity, bias holds sway.

I write about the pervasiveness of pseudoscience bridging the gap between the known knowns and the unknown unknowns. The bridge is made of ropes and missing wooden planks from peak to peak across a deep chasm of mist and a rocky river below. And yet false science continues to replace the missing planks as they fall off trying to keep the path opened between what is and what is not.

We live in a world filled with survivorship bias. We accumulate data and call it evidence. We add to the burgeoning asymmetry of information and call it fact. The nuance of such charlatanism is lost on the populace as the deluge of information overcomes doubt in most minds. But there it is. It is still, bias.

Remember Amgen a pharmaceutical company failed to verify 46 of the landmark 53 studies and the iconoclastic John Ionnadis searched the literature and discovered that 53% of all the “important” studies could not be validated?

Those two findings are the hallmark of the crumbling bridge of bias. In medicine we have fought our way from the basic science to the science of probability. And even though that gives us reasons to pursue a hypothesis, it does not become a tested fact. Survivorship bias is alive and well in the field of medicine as in science. Even met analytical studies include a larger subset of "positive" studies to prove the Author's bias. More and more articles are written only to prove through statistical fiat that a product, a drug or an action has merit. No negative studies are printed. It is akin to the Morningstar assessment of the Bond funds. If one only continues to monitor the surviving bond funds and does not add the bankrupt ones then the optimistic advantage is easily leant to those managers that are still in the game. Such biases can bring in double digit percentage of benefit to those who survive and make them look like heroes; just by surviving the time duration being tested. Just like nominal returns differ greatly from real returns and a perfectly timed market yields very high returns based on retrospective analysis, survivorship bias falls prey to the travails of such fallacies.

I am not going to belabor this monolog more than it needs to be. It is meant to tweak the critical thinking lens of those who dare to aspire a better understanding in the halls of current scientific inquiry and literacy. The misinformed, misguided few can stoke the emotions of the well-meaning many and visit an ocean of storms of untold misery upon the rest.

As Abraham Wald did during WWII, maybe we should look at the fighters that never came back and ask the question, Why! Why armor plating the engines is a better idea than the fuselage where the bullet holes show damage? Maybe we will learn as he did, that fighters that did not return, didn’t because a single bullet to the engine sealed the fate for the pilot and his aircraft.

So for all those piloting the future of health, plots of the forests that you see in well-meaning high impact journals, it would behoove us to take the time to see where the bias resides before making decisions on other’s lives and livelihoods.

Truth matters!