In a recent blog , we looked at the failure of Vitamin A to prevent lung cancer in human trials despite massive hype and other positive research. This study demonstrated the rule that we don’t know something is safe and effective in people until it has been adequately tested in people. In this and upcoming blogs, we are going to look at why this is the case starting with the limitations of basic science and animal research. If you care about avoiding falling for medical bullshit, this blog is important; many news headlines, viral stories, and product claims are based solely on basic science or animal research when you go to the source of their claims. This blog is also important to understand key differences between how medical science advances and how medical bullshit advances. "it is no secret in the scientific community that animal models do not reliably predict how treatments will work in people." It is no secret in the scientific community that animal models do not reliably predict how treatments will work in people.1 Many things that are safe and work in animals aren’t safe and don’t work in people, and some things that work in people don’t work in animals.2 There are several reasons why animal models fail to predict how treatments will work in people including: Differences between species: Put another way, people are not simply large hairless rats (although there are some people who I wonder about). People differ in many important ways from other animals, and these differences can impact how and whether treatments will work or be safe. Differences between the model and the disease: Many human diseases don’t naturally occur in animals. When scientists try to create models of the human illness, there may be important ways that the model fails to replicate the disease in people. For example, some Parkinson’s disease animal models involve giving massive doses of a neurotoxin, a scenario that is not similar to how most people develop Parkinson’s. Biases in animal research: Just as with human studies, animal research can suffer from biases ranging from a lack of appropriate blinding of investigators to publication bias (people are more likely to publish positive findings than research showing something doesn’t work). So why do we use animal studies at all? Because animal studies have led to advances in medical science and new treatments that would have been difficult, if not impossible, to do without animal studies.3 Animal studies are an important step for developing and testing certain therapies but they are no guarantee that a therapy will work in people. So what can we learn from the successes and failures of animal experimentation: Promising results from studies in animals should lead to trials in people, not treatment in people. Looking at the Vitamin A and cancer example: when early animal studies looked promising, serious scientists called for large trials in people4 (which were conducted, and proved Vitamin A didn’t work). Meanwhile, news media, health books, and supplement manufacturers were ready to move straight to sales to the public. The problem here is not animal research, but how it is publicized. Until media and supplements act more responsibly, it will be up to you to draw the appropriate conclusions There is room to improve the quality, reliability, and reproducibility of animal research. The scientific community is taking the failure of many animal models to lead to useful treatments quite seriously.5 This includes progress in understanding differences between species, improving disease models, and calls for increasing the rigor and reproducibility of animal studies.6 Improving the quality and focus of animal studies may also improve their ethical acceptance, along with progress in seeking alternatives to animal research and raising standards for the humane treatment of animal subjects.7 We can all play a role in reducing medical bullshit related to animal research. This includes being more savvy readers of research, being more responsible about what we share, and always seeking to find the source of claims in news and on products. If you are working in news media, consider using more accurate headlines, and if you are a media consumer, call out your media sources when they are misleading. For scientists and medical professionals, we also need to be responsible for how we communicate results of animal studies and, if we perform such studies, ensure they are ethically justified and of the highest scientific rigor. References: 1. Perel P, Roberts I, Sena E, et al. Comparison of treatment effects between animal experiments and clinical trials: systematic review. BMJ 2007;334:197. 2. Bracken MB. Why animal studies are often poor predictors of human reactions to exposure. J R Soc Med 2009;102:120-122. 3. Carbone L. The utility of basic animal research. Hastings Cent Rep 2012;Suppl:S12-15 4. Peto R, Doll R, Buckley JD, Sporn MB. Can dietary beta-carotene materially reduce human cancer rates? Nature 1981;290:201-208. 5. Akhtar A. The flaws and human harms of animal experimentation. Camb Q Healthc Ethics 2015;24:407-419. 6. Frommlet F. Improving reproducibility in animal research. Sci Rep 2020;10:19239. 7. Gilbert S. Progress in the animal research war. Hastings Cent Rep 2012;Suppl:S2-3.

In a recent blog, I looked at the failure of Vitamin A to prevent lung cancer in human trials–despite massive hype and other positive research–to demonstrate the rule that we don’t know something is safe and effective in people until it has been adequately tested in people. In my last blog , I looked at some of the limitations of animal research in predicting human safety and efficacy. In this blog, we will look at how easy it is for correlations to be misleading, even if based on a large numbers of observations. In contrast to much of medicine that studies disease and health in individuals, epidemiology studies health and disease at a population level. As with animal research, there are certain advantages to this approach, such as being able to uncover the impact of certain environmental exposures on health, or determine the impact of public health policy on pandemic spread. There are also limitations, particularly when looking at correlational studies. In a correlation study, researchers collect data on one or more health outcomes of interest (e.g. lung cancer, longevity, happiness) and several potential predictors of this outcome (e.g. smoking, diet, TV watching, zip code) in a sample of people. Researchers then look for correlations between the predictors and health outcomes. This seems like a pretty straight forward way to determine whether a certain predictor causes a certain health outcome or disease, but there are many ways this can go wrong: There could be bias in the sample. If I’m interested in determining whether farm work is associated with certain diseases, but only sample English-speaking people, I could underestimate some significant risks that may impact more vulnerable non-English speakers. There could be bias in who responds. If I send out a survey on “Cannabis and Happiness,” it’s likely that people who respond to the survey may be more likely to have strong feelings on the topic than people who don’t respond. The results could simply represent a statistical fluke. Ironically, the more predictors researchers look at, the more likely it is that they will come up with an erroneous conclusion. In fact, if you look at enough predictors, you can almost guarantee that you will make an error, as happened to a Swedish research group that sought to determine whether living close to power lines caused any of a list of over 800 diseases . Even if the correlation is real, it does not prove causation. Sometimes a correlation may arise because of a shared, but unmeasured, causal factor. For example, yellow teeth may be associated with lung cancer, but that is because both are associated with smoking; teeth whitening will not prevent cancer. Sometimes the conclusions drawn may actually reflect reverse causation. For example, one may see a correlation between smoking and schizophrenia, and conclude that smoking causes schizophrenia; however, it appears that at least some of this correlation may reflect persons with schizophrenia finding some symptom relief from smoking. Sometimes a correlation may simply reflect larger trends in society or other confounding factors. This website goes into this and other causation errors in depth, including a striking graph on the correlation (NOT CAUSATION) of U.S. spending on science and deaths by hanging. The key takeaway here is that one must be skeptical of drawing strong conclusions, particularly about causation, from observational and correlational studies. This happens all the time; many news headlines and medical bullshit books are based on very weak and spurious correlations when you track down the source of the claim.