Probiotics are one of the few supplements where the research and the marketing are both loud, and where the gap between them is easiest to demonstrate — not by arguing, but by reading what was measured. Two meta-analyses, from different areas of medicine, four years apart, tell the same structural story: the biomarkers move, and the patient-facing outcomes either were not measured or did not move.
The 2019 meta-analysis: five significant results, all in a blood tube
The first document is a 2019 systematic review and meta-analysis in Pharmacological Research. Its framing is unusually candid for this literature: the role of gut microbiota in diabetes management is established and several trials are investigating probiotics and prebiotics to modulate the intestinal microbiota, but their findings are controversial.
The search covered PubMed, Web of Science and The Cochrane Library from inception to 31 October 2018. Sixteen eligible randomized controlled trials, 1060 participants. Studies were assessed for risk of bias and synthesised with models accounting for clinical heterogeneity.
The pooled results:
| outcome | direction | standardized mean difference | 95 % CI |
|---|---|---|---|
| hs-CRP (inflammation) | lower | −0.38 | −0.51 to −0.24 |
| Malondialdehyde (oxidative stress) | lower | −0.61 | −0.89 to −0.32 |
| Total antioxidant capacity | higher | +0.31 | 0.09 to 0.52 |
| Nitric oxide | higher | +0.62 | 0.25 to 0.99 |
| Glutathione | higher | +0.41 | 0.26 to 0.55 |
Five significant results, five confidence intervals that do not touch zero. This is a well-executed meta-analysis and the numbers are real.
Now read the outcome column again. C-reactive protein. Malondialdehyde. Antioxidant capacity. Nitric oxide. Glutathione. Every single one is a concentration measured in blood. Not one of them is a symptom, a quality-of-life score, an infection rate, or a hospital admission.
And the review says so itself, in its own conclusion: these supplements may help improve biomarkers of inflammation and oxidative stress in diabetic patients, and further studies are needed to develop clinical practice guidelines. A document that ends by saying clinical guidelines cannot yet be written is telling you the clinical layer is missing.
The qualification buried in the methods section
One sentence in that review deserves more attention than any of the five effect sizes, and it appears in the results almost as an aside:
“The methodological quality varied across these trials.”
That is the paper telling you, in its own words, that the sixteen studies did not all earn their place in a pooled estimate equally well. It is not a fatal objection — meta-analyses pool uneven evidence all the time, and the authors assessed risk of bias rather than excluding on it. But it does mean that the precision of those confidence intervals is not the same thing as the reliability of the finding. A narrow interval around an estimate from mixed-quality trials is still a narrow interval around an estimate from mixed-quality trials.
The 2022 meta-analysis: three trials, 179 people, no effect
The second document tests the same logic in a completely different clinical area, and it is the more useful of the two for a general reader.
A 2022 systematic review and meta-analysis in Nutritional Neuroscience asked whether probiotic supplementation affects migraine attacks. The rationale is plausible: migraine prevalence is higher in patients with gastrointestinal disorders, and possible mechanisms include increased intestinal permeability and systemic inflammation.
The search ran across ISI Web of Science, PubMed/Medline, Scopus, the Cochrane Library, EMBASE, Google Scholar and two regional indexes, for studies published up to October 2019. Pooled effect estimated with a random-effects model.
Then the number that matters:
- three randomized controlled trials;
- 179 patients — 94 probiotic, 85 placebo.
The results:
- frequency: weighted mean difference −2.54 attacks per month, 95 % CI −5.31 to 0.22, p = 0.071;
- severity: weighted mean difference −1.23 on a visual analogue scale, 95 % CI −3.37 to 0.92, p = 0.262;
- heterogeneity: I² = 98 %, p < 0.001.
Three findings, and they have to be read together. The point estimates point in the same direction as the biomarker literature — fewer attacks, less severe. But both confidence intervals cross zero, so neither is statistically significant. And the heterogeneity is 98 %, which on that scale means the trials are close to describing entirely different things.
An I² of 98 % is not a nuisance statistic. It means that if you picked a participant at random from one trial and one from another, you have almost no information about which would benefit. It is the formal statement of “these trials disagree with each other.”
Putting the two together
The two documents were written four years apart, for different conditions, by different teams, and they are not in conflict. What they share is a shape:
- A plausible mechanism. Gut permeability, systemic inflammation, microbial modulation. In the migraine paper the mechanism is stated explicitly as the rationale for the review.
- A real or plausible signal on an intermediate measure. Five biomarkers in 2019; point estimates that point the right way in 2022.
- Either no outcome data, or outcome data that misses significance. The first review measured nothing but biomarkers. The second measured outcomes and got p = 0.071 and p = 0.262.
This is the normal architecture of a promising field. It is also exactly the architecture that produces headlines, because steps 1 and 2 are the interesting ones and step 3 is the boring one that nobody reads.
What these two meta-analyses do not show
They do not show that probiotics do nothing. A p value of 0.071 on three trials with 179 participants is weak evidence of absence, not evidence of absence. The authors’ conclusion is that the available evidence is insufficient — which is a statement about the evidence, not about the organism.
They do not show that the biomarkers mean anything to the person taking it. A lower C-reactive protein is a real laboratory finding. Whether it translates into fewer infections, better glycaemic control, less fatigue, or anything a person would notice is a separate question, and it is precisely the question these two documents did not answer.
The 2019 review does not show that supplementation helps people with diabetes avoid complications. The population is diabetic patients and the outcomes are blood markers. The word “diabetes” in the title describes who was studied, not what improved.
The heterogeneity of 98 % does not get fixed by adding more small trials. It gets fixed by trials that share a protocol.
Neither review establishes safety at scale. Trials of a few weeks in controlled settings answer a different question than a product consumed indefinitely by a general population.
The five questions, and why the last one is the important one
- Was it randomized? Both reviews restricted to randomized controlled trials. That is the entry fee.
- What was the endpoint? A biomarker or a symptom. This is the fork in the road, and it is the question the marketing never answers.
- How many trials and how many people? Three trials and 179 people is not a base to argue from.
- How heterogeneous are they? I² of 98 % is not a footnote; it means the studies do not agree.
- Does the paper say a clinical guideline is needed? The 2019 review does, in its conclusion. When a meta-analysis ends by saying guidelines cannot yet be written, that sentence is the finding.
The fifth question generalises past probiotics, past migraine, past diabetes. It is the sentence to look for in every abstract you read. The same five questions work on a document that arrives with a batch number on it instead of a journal name, and what a supplier’s assay sheet has to show walks through them one line at a time.
