Statistics Matter Thursday, January 26, 2023
Bias is the sin of science.
- Confirmation Bias
The primary reason for the scientific method is to attempt to minimize bias. One of the more common and obvious types of bias is confirmation bias. Confirmation bias can be unintentional (“emotional decision making” and “therapeutic illusion’). The recent winner of the Nobel Prize (Thinking Fast and Slow by Daniel Kahneman) makes the compelling argument that most of our seemingly rational decisions are emotionally based because our emotions are faster, more primitive and more powerful. Our confirmation bias can also be intentional (“conflict of interest” or “fraud”). Confirmation bias, conflict of interest, and fraud are so ubiquitous and covert that some well-respected experts claim it is currently impossible to distinguish true science from adulterated science. (1,2)
II. Design Bias
The way a study is designed sets it up for its own bias. For instance, a study that includes all kinds of people is more likely to represent you. However, this kind of study will have more confounders and more selection bias, so that accurate conclusions are less likely. A study that is highly controlled and looks only at a select study population will have more accurate conclusions, but they may not be as applicable to you or I. Most studies which attempt to introduce a new drug or device are highly controlled and look to prove only that a drug is “effective” without addressing whether it works in the general population (“effectiveness”)— or what the harms are, or what the cost is. Most pharmaceutical studies are designed for theoretical benefit without addressing the pragmatic (effectiveness, harm, affordability, and value).
Design bias can be a form of deception if it intentionally uses a surrogate marker (disease-oriented) without bringing attention to the fact that if it is not ultimately patient-oriented—what is the point?
III. Analysis Bias
This may be one of the greatest areas of deception since most physicians, are poorly educated in finding “statistical tricks” used by pharmaceutical companies and biased authors who make positive conclusions from negative data. There are many tricks which include “conflating correlation with causation”, “combined endpoints”, “dichotomizing results”, “subgroup data dredging”, “changing the primary data points midway through the study” or “ending the study when the data pushes into “statistical significance”’. This is a moving target when pharmaceutical companies or paid researchers use increasingly sophisticated measures (eg. non-inferiority, propensity scores, etc) for which the average physician knows literally nothing.
IV. Publication Bias
Without question, publication bias is the most common and overlooked bias because no one in medicine reads the studies that are refused for publication (most of which have negative conclusions). Tami-flu is the poster child for this which had a meta-analysis used by our government and the CDC to embrace Tami-flu as effective and stock-piled billions of dollars of the drug (this decision was influenced by then US department of Defense, Dick Cheney who was chairman of the board of the company who sold Tamiflu to Roche and who kept his large stockholdings when coming to Washington). Only later, was it revealed in a Cochrane review that the eight unpublished studies on Tami-flu were negative and when combined with the two positive published studies, the overall conclusion is that Tami-flu was not effective.(3) This bias repeats itself daily with every meta-analysis and systematic review of published randomized controlled trials. Publication bias and the way decisions are made to publish expose a great deal of fraud in medicine.
references
- Angell, m. (2009) Drug companies and doctors: a story of corruption . The New York review of books.
- Stegenga, j (2018) Medical Nihilism
- Doshi, p. (2009) “Neuraminidase inhibitors: the story behind the Cochrane review.” BMJ 344, d7898.
Written by Dr. Mark Mosley