Evaluating any nutrition claim

TL;DR. A claim arrives — a TikTok clip, a podcast quote, a headline, a supplement label, an aside from your doctor. Most readers either accept it because it sounded confident or reject it because it sounded wrong. Both reactions are guesses. Seven questions, in order, settle most of them in under a minute. What is the actual claim? What is the source — paper, anecdote, vibe? Who funded it? What study design produced it? What does the absolute risk look like once you peel back the relative-risk percentage? Has it been replicated? Who is the human delivering it, and what do they make money on? This module is a capstone: it pulls the study-design hierarchy from D1 and the funding-machine anatomy from D3 into a checklist you can actually use. The point is not to win arguments. It is to stop reorganizing your kitchen every time a confident stranger speaks.

What you'll learn

• The seven-question checklist, in the order that filters fastest.
• How to convert relative risk into the absolute numbers that should anchor any decision.
• How to find the actual paper behind any "studies show" headline in two clicks.
• Why most supplements fail randomized trials but win observational ones.
• The animal / cell-culture / human gradient and why dose ratios usually tell you the answer.
• The vitamin-E-in-pills-vs-vitamin-E-in-almonds problem, and why it generalizes.
• What Mendelian randomization tells you and what it doesn't.
• The recurring tells of wellness-influencer and journalist content.
• When the honest answer is "we don't know" and what to do then.

1. The seven questions

This is the checklist. The order matters: each question filters out a class of bad claims so the later questions have less to do.

Q1. What is the actual claim? Most viral statements collapse on contact. "Seed oils cause inflammation" — in whom, at what dose, against what baseline? "Coffee causes cancer" — which preparation, which cancer site, what intake range? If the claim does not specify the population, dose, endpoint, and comparator, it is a slogan, not a finding.

Q2. What is the source — a study, a headline, an anecdote, or a vibe? Trace it backward. A WhatsApp probably cites a podcast; the podcast cites an article; the article should cite a paper; the paper has a DOI. Each step adds error. If the chain dead-ends without a paper, that is the answer to whether the claim is evidence-based.

Q3. Who funded it? Lesser et al.'s 2007 PLoS Medicine analysis found industry-funded beverage studies four to eight times more likely to favor the sponsor; Spector cites the subset where industry-funded drink studies favor the sponsor twenty times more often. Food companies fund roughly eleven times more nutrition research than the NIH does. This does not make industry findings wrong; it shifts the prior. Read the funding declaration, the author disclosures, and the funding of any meta-analysis aggregating the underlying papers.

Q4. What is the study type? The hierarchy from D1, compressed: RCT with hard endpoints > prospective cohort with repeated dietary assessment, replicated across populations > Mendelian randomization on a well-instrumented exposure > controlled feeding trial with biomarker endpoints > single cohort > cross-sectional/ecological (prone to the ecological fallacy) > case-control (recall bias) > animal or cell-culture (mechanism only) > anecdote (not evidence). A 30-person crossover feeding trial and a 500,000-person cohort with twenty-year follow-up should not produce equal confidence even if they reach the same number. Rank the design before you read the headline number.

Q5. What is the absolute (not relative) effect size? The single most weaponized statistic in nutrition reporting. A 30 percent relative risk increase sounds catastrophic and is usually a tiny absolute change. We work this out in detail in the next section.

Q6. Was this peer-reviewed and replicated? A single paper is a single roll of a noisy die. Findings that have crossed replication boundaries — different populations, different research groups, different designs — earn trust the original paper alone cannot. Findings in only one design (only mice, only case-control, only one cohort) are weaker.

Q7. Who is the source telling me, and what are their incentives? Every claim arrives via a human with a product, a brand, a clinic, a supplement line, or a book to sell. A registered dietitian salaried at a hospital and an influencer pushing a $79 "metabolic reset" course are not interchangeable narrators, even if they say identical sentences.

Most claims fail at Q1 or Q2 — they cannot survive being asked precisely what they mean or where they came from.

2. The relative-risk trap with worked examples

Almost every viral nutrition headline reports relative risk. Almost none reports absolute risk. The gap is the largest source of public misunderstanding in the field.

Example 1: processed meat and colorectal cancer. The IARC's 2015 Group 1 classification drove headlines reading "processed meat raises colorectal cancer risk by 18 percent" — a relative risk per 50g daily serving. The absolute numbers underneath: lifetime colorectal cancer risk in the Western population is around 5 percent. An 18 percent relative increase on a 5 percent baseline lifts lifetime risk to roughly 5.9 percent — a 0.9-percentage-point change. Spector's analog: the average Italian meat eater's processed-meat cancer risk equals smoking three cigarettes per year. Same data; different framing.

Example 2: the 2018 Lancet alcohol study. "No safe level of alcohol" used the same maneuver in the opposite direction. The absolute numbers: one drink per day raises the risk of an alcohol-related event by roughly one per 1.25 million bottles of wine consumed. Technically defensible — a non-zero risk exists. Trivially small at moderate intake.

Three rules to apply when a percentage lands on the page:

1. What is the baseline rate? Without it, no relative risk has meaning. A 50 percent increase on a 0.0002 percent risk is a different sentence from a 50 percent increase on a 5 percent risk.
2. What is the absolute change? Translate "X percent higher" into "X.Y percentage points," or "one extra case per Z thousand people-years," or into number needed to harm.
3. What is the sample size and follow-up length? A 2 percent relative effect detected across 500,000 people over 20 years is plausible; the same effect in 200 people over six weeks is noise.

The discipline of converting relative to absolute is the single highest-leverage habit a reader can build. Most headlines collapse on it.

3. The "studies show" tell — finding the actual paper

Headlines almost never link the paper; press releases almost never link the paper; podcasts almost never link the paper. The two-click recovery: search Google Scholar or PubMed for the lead researcher's last name plus the journal plus the year. The paper will surface as a top result. Find the DOI in the citation; resolve it via doi.org or look for an open-access version on the author's institutional page, PubMed Central, medRxiv, or bioRxiv.

Then read three things before anything else: the abstract conclusion, the funding declaration in the small-print footer, and the limitations section. A surprising fraction of "groundbreaking" headline papers acknowledge, in their own limitations section, exactly the confounders that should have stopped the headline.

If the chain dead-ends — no paper exists — the claim is anecdote, mechanism speculation, or marketing.

4. The supplement evidence question

Half of Americans take supplements; the global market is heading toward roughly $200 billion. Almost every major supplement tested in a proper randomized trial has failed or shown harm. The structural reason is healthy-user bias: people who take supplements differ from people who do not in dozens of measurable ways (income, education, smoking, exercise, regular medical care) and dozens of unmeasured ways. Observational studies see those differences as a supplement effect. Randomized trials, which assign supplements regardless of who the person is, do not.

The pattern repeats. Vitamin D for fractures: glowing observational data, then the MR analysis of over 500,000 people and 188,000 fractures finding no causal effect. Fish oil for heart disease: dropped by the AHA after a 25,000-person trial and a 79-trial meta-analysis covering 112,000 people. Calcium for bones: WHI in 35,000+ women showed slight density gains but no fewer hip fractures, plus more kidney stones. Vitamin E for cancer prevention: SELECT increased prostate cancer.

Reading any new supplement claim: ask whether the supporting evidence is observational (probably healthy-user-biased), whether any RCT with hard endpoints exists, and what it showed. A supplement with a strong observational signal and either no RCT or null RCT data usually does nothing useful.

5. The animal / in vitro / human gradient

Most viral nutrition mechanism claims begin in cell culture or rodent studies. Mice are not 70 kg humans. Three filters:

• Species. Rodent metabolism differs from human metabolism in many specific ways. The acrylamide cancer scare came from rodents fed doses no human approaches in a lifetime. Saccharin caused bladder cancer in rats via a pH mechanism that does not exist in humans.
• Dose. Cell-culture studies routinely use concentrations ten- to ten-thousand-fold higher than physiological levels. A compound that kills cancer cells at 100 micromolar may never reach 1 micromolar in any real human cell. Always read the dose against realistic intake.
• Endpoint. Mechanism studies measure surrogate markers — gene expression, cytokine levels, oxidative stress — which may or may not correspond to anything clinical. A diet that "reduces inflammation markers" may or may not reduce any inflammatory disease.

Spector's rule: mechanism work is the beginning of a question, not the end. A finding restricted to mice or cells is hypothesis-generating. If it survives translation to humans at realistic doses with real endpoints, it earns the next tier of trust.

6. The single-nutrient fallacy — vitamin E in pills versus almonds

The clearest case study in nutritionism's failure is vitamin E. Observational data through the 1990s showed people with higher dietary vitamin E intake had less heart disease. The mechanism was plausible (antioxidant protection of LDL from oxidation). The pharmaceutical move was obvious: put vitamin E in a pill and test it.

The HOPE-TOO trial (Lonn et al., JAMA 2005), GISSI Prevention, and the Women's Health Study together randomized tens of thousands of people to vitamin E or placebo. None showed cardiovascular benefit. HOPE-TOO showed a small significant increase in heart-failure hospitalizations in the vitamin E arm. SELECT showed more prostate cancer in the vitamin E group.

The observational data were not wrong about almonds. Vitamin E does not work in isolation. Almonds contain vitamin E plus dozens of other compounds — monounsaturated fats, magnesium, fiber, polyphenols, plant sterols — that act together. Pull out alpha-tocopherol, put it in a gelcap, and the system that did the work is gone. Pollan's line: a leaf of thyme contains dozens of antioxidants; isolate one and you have a different compound in a different context.

This generalizes. Beta-carotene supplements increased lung cancer in smokers (CARET). Calcium supplements appear to raise cardiovascular risk while calcium from food does not. Whenever you see "compound X has benefits — and you can take it as a pill," the prior should be that the pill does not replicate the food.

7. Mendelian randomization — what it does and doesn't tell you

Mendelian randomization (MR) is one of the strongest tools modern nutrition epidemiology has, and one of the most easily misread. It uses genetic variants associated with an exposure (e.g., a variant that raises lifetime LDL, or alters lifetime vitamin D) as an instrumental variable. Because alleles are randomly assigned at conception, the variant approximates a lifelong randomized exposure unconfounded by lifestyle. If gene-variant carriers have more or less of an outcome, that is causal evidence, not just correlation.

The canonical positive case is LDL and cardiovascular disease. Multiple MR analyses across hundreds of thousands of people show that lifelong genetically higher LDL causes CVD. The canonical negative case is vitamin D and fractures: the MR analysis of over 500,000 people and 188,000 fractures found no causal effect of vitamin D status on fracture risk. Observational data had repeatedly suggested benefit; the genetic design said no.

MR is a sharp tool for a narrow class of questions. It cannot speak to exposures without strong, specific genetic instruments (most foods, most dietary patterns), non-linear effects, or short-term effects. When a claim invokes MR, check the instrument and the population size. When a claim could be tested by MR but hasn't been, that is informative too.

8. The wellness-influencer pattern

A recurring set of moves shows up in influencer-driven nutrition content. Recognizing them is most of the work.

Credential laundering. "Doctor" can mean MD, DO, PhD, ND (naturopath), DC (chiropractor), or honorary doctorate. "Nutritionist" is unregulated in most US states; "registered dietitian" (RD) is the credentialed equivalent. "Biohacker" has no certification. "Functional medicine practitioner" is a private credential, not a state license. Read the actual letters.

The mechanism aside. A confident-sounding mechanism story ("seed oils oxidize and damage your mitochondria") presented as if it were outcome evidence. Mechanism stories are cheap; outcome data are expensive. A persuasive mechanism with no outcome data is hypothesis, not finding.

The sponsorship disclosure. Most influencer videos contain a "use code X for 20% off" segment. The product being sold is the answer to Q7.

The single-study cite. One study, often small, often in mice. The full literature — systematic reviews, contradicting cohorts, negative trials — is unmentioned. Find the paper, then search Google Scholar's "cited by" for the response literature.

The "they don't want you to know" frame. Authority-distrust narrative borrows credibility from the real history of industry capture (D3) and applies it to claims that have nothing to do with industry. Distrust authorities — including the influencer.

The reversal-of-everything tell. "Everything you've been told about X is wrong" is rarely true. A creator whose every claim is a reversal is selling the reversal as a product.

9. The journalist tells

Headlines are written by editors, not researchers. Phrases that should slow your reading:

• "Scientists baffled." Almost no scientists are baffled. The headline is doing emotional work.
• "This one trick." No serious finding fits in a trick.
• "The food they don't want you to eat." No "they" exists; the food is in every supermarket.
• "New study finds." "New" usually means "single," the weakest position in evidence-land.
• "Linked to." Almost always observational, almost always a headline-inflated relative risk against a small absolute effect.
• "May reduce risk of." "May" is doing all the work. Often a mechanism study or single weak association.
• "Up to X percent." "Up to" is doing all the work. The actual effect could be near zero.
• "Doctors hate this." Marketing copy.

None of these guarantees the underlying paper is bad. They guarantee the headline is not a reliable summary of it.

10. What "I don't know" means

The honest answer in nutrition is often uncertainty. The field is young, the tools are limited, the funding is captured, and the replication crisis is real. A reader who can say "I don't know" is in a better epistemic position than a reader who confidently believes the latest TikTok.

The practical implication is not paralysis. It is calibration. Build your eating around the durable findings — patterns that have survived across designs, populations, and decades. Mediterranean. Mostly plants. Less ultra-processed food. Less sugar-sweetened beverages. Whole grains over refined. These are not maybes; they are the strongest signals the field produces. Hold the rest with light hands.

When a new claim arrives, run the seven questions. Most fail at Q1, Q2, or Q3 and require no further attention. The ones that survive deserve actual reading. The very few that survive that reading deserve a small change in behavior — usually small, often reversible, never the kitchen-overhaul the headline implied.

FAQ

Q: Should I trust [X] podcast?

Trust the claims, not the host. A podcaster with strong credentials can make a poorly evidenced claim; one with no credentials can repeat a well-evidenced finding. Run the checklist on the specific claim, not on the personality.

Q: Why do studies contradict each other?

Populations respond differently (PREDICT showed tenfold variation in glucose response to identical meals). Different designs catch different signals. About 5 percent of well-run studies produce a "significant" finding by chance. And industry funding skews the literature predictably. Convergence across designs, populations, and laboratories is what real signal looks like.

Q: Are RCTs always best?

No. Willett devotes Chapter 3 of Eat, Drink, and Be Healthy to defending cohort designs against the reflexive "RCTs are gold" rule. Long-term food RCTs are largely infeasible — you cannot blind people to broccoli, adherence collapses, hard endpoints develop over decades. The $415-million WHI low-fat trial was undermined when the "low-fat" arm did not actually eat much less fat than controls. A well-run prospective cohort with repeated dietary assessment can be more informative than a small short RCT.

Q: What is a meta-analysis, and should I trust them?

A meta-analysis statistically pools multiple studies on the same question. The quality depends almost entirely on the studies pooled inside it. A meta-analysis of fifteen weak observational studies produces a confident-looking number no stronger than its inputs. Read the funding, the inclusion criteria, and whether the pooled studies converge. The 2019 Canadian meta-analysis declaring red meat safe was funded through ILSI and excluded most harm-direction data.

Q: Who's a credible nutrition source?

A starter list: registered dietitians without product lines; Harvard T.H. Chan School of Public Health Nutrition Source; the NIH Office of Dietary Supplements; the Cochrane Collaboration; researchers with long publication records who declare their conflicts. None are infallible — the seven questions still apply.

Q: Is anything in PubMed reliable?

PubMed indexes papers; it does not vouch for them. A paper on PubMed has crossed peer review at some journal, not necessarily a strong one. Reliability depends on design, sample size, funding, and replication — not the fact of indexing.

Q: Should I follow my doctor or my dietitian?

For medical and pharmacological questions: doctor. For day-to-day nutrition planning and disease-specific eating patterns: dietitian. Most US physicians receive fewer than three practical hours of nutrition training in a six-year degree; registered dietitians are the credentialed specialists. For complex cases (diabetes, kidney disease, eating disorders, IBD): both, coordinated.

Q: What if I run the seven questions and the answer is still ambiguous?

That is often the correct answer. Most live nutrition questions sit in an evidentiary gray zone. The right response is calibration, not certainty. Make small changes you can sustain; avoid big changes based on small evidence. The goal is not to know everything. It is to stop being knocked around by every confident stranger.

Sources

• Pollan, M. In Defense of Food: An Eater's Manifesto — Chapter 9, "Bad Science," for the methodological dissection and the FFQ underreporting figures. Penguin, 2008.
• Nestle, M. Food Politics — Chapters 5 and 6 for the anatomy of industry funding of nutrition professionals and journals. University of California Press, 2002 (10th-anniversary ed. 2013).
• Spector, T. Spoon-Fed — Introduction, Chapter 5 (supplements), Chapter 9 (meat and the IARC), and Chapter 20 (alcohol) for the relative-vs-absolute-risk worked examples. Vintage, 2020.
• Willett, W. Eat, Drink, and Be Healthy — Chapter 3 for the study-design hierarchy from the Harvard cohorts' perspective. Free Press, revised 2017.
• Schatzker, M. The Dorito Effect — Chapters 2 and 3 for the dilution effect and the legal/process distinction behind "natural flavor." Simon & Schuster, 2015.
• Kearns, C., Schmidt, L., Glantz, S. "Sugar industry and coronary heart disease research: a historical analysis of internal industry documents." JAMA Internal Medicine 176(11):1680-1685, 2016. DOI: 10.1001/jamainternmed.2016.5394.
• Lesser, L. et al. "Relationship between funding source and conclusion among nutrition-related scientific articles." PLoS Medicine 4(1):e5, 2007. DOI: 10.1371/journal.pmed.0040005.
• Lonn, E. et al. "Effects of long-term vitamin E supplementation on cardiovascular events and cancer: the HOPE and HOPE-TOO trial." JAMA 293(11):1338-1347, 2005. DOI: 10.1001/jama.293.11.1338.
• Hall, K. et al. "Ultra-processed diets cause excess calorie intake and weight gain." Cell Metabolism 30(1):67-77.e3, 2019. DOI: 10.1016/j.cmet.2019.05.008.
• IARC Monograph 114 (2018), "Red Meat and Processed Meat," for the source of the 18-percent-per-50g processed-meat colorectal cancer relative risk used as the worked example.

Related modules

• How nutrition science actually works — the long-form treatment of the study-design hierarchy and the funding-machine anatomy this checklist compresses.
• Big food vs. public health — the political-economy backstory of why Q3 matters.
• Reading labels — the same evaluative posture applied to the small-print claims on packaging.

Related glossary

• Epidemiology — the study of how diseases distribute across populations.
• Randomized controlled trial — the design, its limits in food research.
• Cohort study — the prospective design that powers most nutritional epidemiology.
• Case-control study — the retrospective design that produces most early cancer-and-food findings, with its recall-bias caveat.
• P-hacking — selective analysis and reporting that inflates the false-positive rate.
• Ecological fallacy — drawing individual-level conclusions from population-level data.
• Relative risk — the ratio between exposed and unexposed groups' incidence; the most weaponized statistic in nutrition reporting.
• Absolute risk — the actual percentage-point change in incidence; the number that should anchor any reporting.
• Number needed to treat / harm — how many people need the exposure for one to be affected; the most intuitive translation of effect size.
• Conflict of interest — disclosed financial ties between researchers and product manufacturers.
• Mendelian randomization — using genetic variants as instrumental variables for lifelong exposures.
• Systematic review — a pre-specified search-and-inclusion protocol summarizing a literature.
• Meta-analysis — a statistical pooling of multiple studies' results.