Healthy volunteer bias often leads us astray

Beware of results from observational studies

We are bombarded with messages from boards of health, drug agencies, drug companies, other institutions, the media, magazines, influencers, quacks, vegetarians, vegans, and do-gooders of all kinds that this or that makes us live longer or healthier.

As an example, the Danish National Board of Health launched the “Six a day” campaign in 2001,¹ which ran for a very long time in the media. It was about eating six pieces of fruit or vegetables every day, e.g. three apples, one large tomato, one large carrot and a banana.

I tried to follow the advice, just for the fun of it, but it was impossible for me.

Your first question when you hear of dietary advice should always be: Was it based on randomised trials? Surely not. It would not be feasible to randomise people to eat six apples or whatever, or none of that, for several years.

So, the evidence must be from observational studies where people decide themselves what to eat. People who eat a lot of fruit and vegetables differ from others in all sorts of ways that could increase their longevity. We call it the healthy volunteer bias.

As the Board of Health said nothing about the basis for its recommendation, I searched the medical literature. Many apples later, a large meta-analysis of 95 studies was published.² It reported that, for fruits and vegetables combined, the risk ratio for all-cause mortality was 0.90 (95% confidence interval 0.87 to 0.93) if the intake was 200 g per day (corresponding to two apples). If you ate eight apples a day, the reduction in mortality was 31%.

The authors concluded that “An estimated 5.6 and 7.8 million premature deaths worldwide in 2013 may be attributable to a fruit and vegetable intake below 500 and 800 g/day, respectively, if the observed associations are causal.”

That’s exactly the point. We have no idea if the observed associations are causal or just bias. The uncertainties in observational studies are far too great to allow us to draw any conclusions about eating apples when the differences in mortality are relatively minor.

Researchers should not scare people with such messages. I wondered how many apples a day it takes to keep the busybodies away. Some of our best epidemiologists have stated that, considering how easy it is to be fooled, a threefold risk increase is only persuasive if the lower limit of the 95% confidence level falls above a threefold increased risk (which means that we are 95% certain that the true risk is increased at least threefold).³

I have always been pretty immune to dietary advice. I eat a varied diet of things I like and avoid becoming overweight. I also drink a lot of coffee. Coffee must be very healthy since it has been subjected to numerous observational studies and has survived these attacks despite the unavoidable biases in such research. For example, if you are stressed, you might drink more coffee and be at greater risk for cardiovascular disease; and more smokers are also found among coffee drinkers. I have never understood why so many Americans ask for decaffeinated coffee when it seems that coffee cannot harm you. In a study of 58,397 deaths, the dose-response relationship was similar for caffeinated and decaffeinated coffee, and already at 2-3 cups a day, the mortality was 18% lower than for non-drinkers.⁴

An another example, people who drink alcohol are different to people who don’t drink alcohol, and comparisons of their risk of dying will therefore inevitably be flawed. It was believed for many years, and confirmed in numerous large‐scale epidemiological studies, systematic reviews, and meta‐analyses, that the relation between alcohol consumption and mortality was a U- or J-shaped curve, so that a little alcohol every day was beneficial.⁵ But the studies were biased. The teetotallers included former drinkers who had quit due to poor health.⁶

The unbearable lightness of observational studies

First, a few indisputable facts.

Randomisation is the only experimental method we have which guarantees that any difference at baseline between two groups is random. This is a prerequisite for the validity of statistical testing. If we randomly assign one group to a cholesterol-lowering drug and the other to a placebo, we may test the hypothesis: What is the chance (the P-value) that, if the drug has no effect on mortality, we see this difference in deaths between the two groups or an even bigger difference?

If P < 0.05, we traditionally say the difference is statistically significant and conclude that the drug changed mortality.

Few doctors and medical researchers know that statistical testing of results derived from observational studies are invalid because we cannot assume that the two groups were comparable at baseline. There are therefore too many statistically significant P-values in observational studies.⁷

Since people who are being compared in observational studies differ in many respects other than the outcome of interest, it is common to adjust for baseline differences with statistical methods. However, it is not possible to adjust reliably for baseline differences. When a statistician used raw data from two large, randomised trials as the basis for observational studies that could have been carried out, he found that the more baseline variables he included in a logistic regression, the further he got from the truth.⁸ He also found that comparisons may sometimes be more biased when the groups appear comparable than when they do not. He warned that no empirical studies have ever shown that adjustment, on average, reduces bias.

An example of the healthy volunteer bias is related to cholesterol lowering. In a trial of a lipid-lowering agent, clofibrate, there was no difference in mortality between drug and placebo.⁹ But among those who took more than 80% of the drug, only 15% died, compared to 25% among the rest (P = 0.0001). This doesn’t prove that the drug works of course, and the same difference was seen in the group that received placebo, 15% versus 28% (P = 5 · 10^–16). Patients who do what they are told are generally healthier than others and therefore have better survival even when the drug is placebo.

Unfortunately, the world is flooded with unreliable observational studies that lead us astray because they are much easier and much cheaper to do than randomised trials. Since observational studies have a poor reputation, those who do them have invented the euphemism ”real-world evidence,”¹⁰ as if such evidence were superior to that derived from randomised trials.

When I did research for my first vaccine book,¹¹ I was shocked to find out that two key US institutions, the CDC (Centers for Disease Control and Prevention) and the FDA (Food and Drug Administration) often seriously misinform the public about vaccines, with ill-founded claims about miraculous effects while ignoring important harms.

For vaccines, the CDC excels in case-control studies, which are notoriously unreliable for studying benefits of interventions.¹² After having read what the CDC claimed about flu shots, I distrusted the agency totally. The CDC not only claimed an effect on mortality in healthy people, which has never been shown in randomised trials, they even claimed a much bigger effect than what is mathematically possible, considering the prevalence of influenza and the number of people who die during the winter season.¹¹

The misinformation from the FDA was similar to the misinformation from the CDC: “A lot of the illness and death caused by the influenza virus can be prevented by a yearly influenza vaccine.” This is simply not true.^11,12

What I find most appalling is that the CDC and the FDA does not even warn the readers about the biases in observational studies. The CDC announces colossal effects of vaccination without the slightest hint that these estimates come from highly unreliable research such as case-control studies.^11,12 And when the FDA informed about its use of “real-world evidence” (last updated 2 June 2026), there was nothing about bias.¹³

In a paper that appeared earlier this month in JAMA Intern Med,¹⁴ the supplemental online content, which virtually no one reads, contained data that invalidate the whole study.¹⁵ The authors reported that, at 8 months, the COVID-19 vaccine was associated with a 38% lower risk of COVID-19-associated major adverse cardiovascular events. However, this was the healthy vaccinee bias. All cause mortality curves separated already by day 10, which is impossibly fast.¹⁵

When I was young, the study tutor at the University of Copenhagen advised in a newspaper that it was a good idea to take a year off after school and travel around and see the world because a study had shown that more of those who did this finished their university studies than those who didn’t take a year off.

I think you can see what the problem is with this advice. The healthy student bias. Those who have the resources and confidence to take a year off also have the resources and confidence to finish their studies.

References

1 Gøtzsche PC. Survival in an overmedicated world: look up the evidence yourself. Copenhagen: People’s Press; 2019.

2 Aune D, Giovannucci E, Boffetta P, et al. Fruit and vegetable intake and the risk of cardiovascular disease, total cancer and all-cause mortality - a systematic review and dose-response meta-analysis of prospective studies. Int J Epidemiol 2017;46:1029-56.

3 Taubes G. Epidemiology faces its limits. Science 1995;269:164-9.

4 Park SY, Freedman ND, Haiman CA, et al. Association of coffee consumption with total and cause-specific mortality among nonwhite populations. Ann Intern Med 2017;167:228-35.

5 Miller AP. Still rethinking the J-shaped curve: A commentary on Kember et al., 2024. Alcohol Clin Exp Res 2025;49:503-6.

6 Shaper AG, Wannamethee G, Walker M. Alcohol and mortality in British men: explaining the U-shaped curve. Lancet 1988;2:1267-73.

7 Gøtzsche PC. Believability of relative risks and odds ratios in abstracts: cross-sectional study. BMJ 2006;333:231-4.

8 Deeks JJ, Dinnes J, D’Amico R, et al. Evaluating non-randomised intervention studies. Health Technol Assess 2003;7:1-173.

9 The Coronary Drug Project Research Group. Influence of adherence to treatment and response of cholesterol on mortality in the coronary drug project. N Engl J Med 1980;303:1038-41.

10 Chodankar D. Introduction to real-world evidence studies. Perspect Clin Res 2021;12:171-4.

11 Gøtzsche PC. Vaccines: truth, lies, and controversy. New York: Skyhorse; 2021.

12 Gøtzsche PC. CDC, the extended arm of the vaccine industry. Has seriously misinformed the American public. Substack 2026;June 24.

13 CDRH and Real-World Evidence. FDA 2026;June 2.

14 Cai M, Xie Y, Al-Aly Z. 2024-2025 COVID-19 vaccine and major adverse cardiovascular events among US veterans. JAMA Intern Med 2026;Jun 15:e261929.

15 Prasad V. A new JAMA IM paper on covid shots contains a preposterous result that invalidates the entire paper. Substack 2026;June 16.