Baloney Inspection Report: Claim #1053 · Inspected Sep 29, 2026, 6:37 PM PDT · Scoring model v2 · Earlier report, model v1
74%“If most scientists agree, the science is settled forever.”
74%Radioactive Baloney — Dangerously Toxic
Dangerously misleading.
Overwhelmingly falseContradicts known factsPromotes false beliefsPotential for harm
Serve It Fresh
Ready to Serve10 servings
Serve it the way you want it
The Verdict
“If most scientists agree, the science is settled forever.”74% Radioactive Baloney — Dangerously ToxicVerified blind by 5 frontier AIs · Baloney Inspection Report: baloney.ai/baloney/if-most-scientists-agree-the-science-is-settled-foreverThe Baloney Sampler
The claim fails because the scientific method makes no conclusions permanently final. While widespread expert agreement is our best guide, science is inherently provisional and relies on continuous testing. Major paradigm shifts are common when new evidence emerges. For example, the U.S. Geological Survey and the Nobel Prize organization document massive consensus reversals, such as the initial rejection of plate tectonics and the discovery that bacteria—not stress—cause peptic ulcers.
Furthermore, "most scientists" is a weak threshold that could represent a bare majority or include experts outside the relevant field. The National Academies of Sciences, Engineering, and Medicine stresses that scientific uncertainty never reaches zero. While institutions like NASA rely on near-unanimous consensus to inform policy on issues like human-caused climate change, they treat these conclusions as practically actionable and highly reliable for current decisions, rather than literally settled forever.
Resting on National Academies of Sciences, Engineering, and Medicine · NASA · U.S. Geological Survey · Nobel Prize
Original Claim Source https://nap.nationalacademies.org/catalog/27894/understanding-and-addressing-misinformation-about-science
Other ways to slice it — phrasings that might have changed the rating
- “When an overwhelming majority of experts agree based on multiple lines of evidence, the science is settled enough for practical policy decisions.” The reports note that organizations like NASA and the National Academies use strong, evidence-based consensus to justify immediate societal action, even if knowledge remains epistemologically provisional.
- “Strong scientific consensus is the most reliable indicator of the best available evidence at a given time.” The evaluators explicitly state that while consensus isn't permanent proof, it serves as a highly reliable proxy for synthesizing the current state of knowledge.
- “Some core scientific theories have been so rigorously tested that their foundational principles are practically unlikely to be overturned.” The lab reports acknowledge that for deeply replicated findings like germ theory or heliocentrism, abrupt reversal is highly improbable, making 'settled' a defensible colloquial shorthand.
Written from the panel’s reports by Google · gemini-3.1-pro-preview. Nothing here changed the score.
The Order Ticket
01:37:30CLAIM RECEIVED — QUEUED FOR INSPECTION✓
01:38:07claude-opus-5 REPORTED — SCORE 87, CONFIDENCE 95✓
01:37:58gpt-5.5 REPORTED — SCORE 72, CONFIDENCE 93✓
01:38:02gemini-3.1-pro-preview REPORTED — SCORE 85, CONFIDENCE 98✓
01:37:37sonar-pro REPORTED — SCORE 68, CONFIDENCE 97✓
01:39:07grok-4.6 REPORTED — SCORE 74, CONFIDENCE 86✓
01:40:03PRIMARY MASTER GEMINI — SCORE 85✓
01:39:11BLIND VALIDATION PERPLEXITY — SCORE 94✓
01:40:21AGREEMENT MODERATE DISAGREEMENT — DELTA 20✓
01:40:21SCORING MODEL V2 — MEDIAN OF 5 COUNTED LAB SCORES: 87, 72, 85, 68, 74 → 74.✓
01:40:21SOURCE RULE — LABS CITED 25 SOURCES: 0 DISCARDED FOR HAVING NO URL, 0 DISCARDED AS DEAD LINKS (404/410), 2 COULD NOT BE OPENED BY US AND ARE MARKED UNVERIFIED.✓
01:40:21SOURCE RULE — PRIMARY MASTER RULED ON 20 SOURCES: 0 DISCARDED FOR HAVING NO URL, 0 DISCARDED AS DEAD LINKS, 2 MARKED UNVERIFIED, 18 VERIFIED.✓
How This One Was Sliced
Station 1 · Blind Inspection — Five Graders, Zero Peeking
Each lab scored the claim independently, without seeing the others. Open a breakdown for its evidence, its sources, and how sure it was.
Anthropic · EQUAL WEIGHT · 37.0s
claude-opus-5
87%BALONEY
ROUND-1 BREAKDOWN
Evidence offered for the claim- Consensus among experts is a genuinely strong epistemic signal: on questions like human-caused climate change, surveys of the peer-reviewed literature find ~97% agreement, and that consensus has strengthened rather than reversed over decades of additional evidence (Cook et al. 2013; NASA).
- Some well-replicated findings — germ theory, heliocentrism, the atomic nature of matter, common descent — have never been overturned and are, in practical terms, not going to be. For such core results, 'settled' is a defensible shorthand, so the claim is not absurd in every instance.
- Scientific method is self-correcting and cumulative: theories that survive repeated attempts at falsification (e.g., plate tectonics after 1960s seafloor-spreading data) tend to be refined rather than discarded, so majority agreement usually does predict durability.
Evidence against the claim- 'Forever' is the fatal word: scientific conclusions are provisional by construction. All major scientific bodies treat findings as revisable in light of new evidence; no result is exempt from re-testing.
- Documented reversals of majority opinion: continental drift was rejected by most geologists for ~50 years before becoming consensus plate tectonics (USGS). Peptic ulcers were near-universally attributed to stress/acid until Marshall and Warren showed H. pylori causation — a view initially dismissed, later awarded the 2005 Nobel Prize and now standard treatment guidance (NIDDK).
- Dan Shechtman's quasicrystal observations contradicted established crystallography and were publicly ridiculed by eminent chemists; he received the 2011 Nobel Prize in Chemistry when the consensus was overturned.
- Newtonian mechanics was the overwhelming consensus for ~200 years and was superseded (as a limiting case) by relativity and quantum mechanics — showing that even extremely well-supported consensus can be revised in scope.
- Replication research shows that agreement in the published literature can itself be unreliable: the Open Science Collaboration (2015) successfully replicated only ~36–47% of 100 psychology studies, and Ioannidis (2005) argued a large fraction of published findings are false. The National Academies' 2019 report treats replication failure as a normal, expected feature of science.
- 'Most scientists agree' is also a weak premise as stated — it can describe a bare 51% majority, agreement outside a scientist's own field, or agreement measured by opinion poll rather than by evidence. Consensus is a proxy for evidence, not a substitute for it.
- The claim, as written, is an appeal-to-authority fallacy dressed as epistemology; scientific institutions explicitly reject it (the Royal Society's founding motto 'nullius in verba' — take nobody's word for it).
How sure this lab was95/100 — the lab’s own confidence in the score above, not a second reading of the claim.
SourcesEnvironmental Research Letters (IOP) — Cook et al. 2013journal · 88 · NASAgov · 95 · U.S. Geological Survey, 'This Dynamic Earth'gov · 95 · Nobel Prize Organisation (Marshall & Warren, H. pylori)other · 92 · Nobel Prize Organisation (Shechtman, quasicrystals)other · 92 · NIH / NIDDKgov · 95 · Science — Open Science Collaboration, 'Estimating the reproducibility of psychological science' (2015)journal · 93 · PLOS Medicine — Ioannidis, 'Why Most Published Research Findings Are False' (2005)journal · 85 · National Academies of Sciences, Engineering, and Medicine — 'Reproducibility and Replicability in Science' (2019), via NCBI Bookshelfgov · 94
OpenAI · EQUAL WEIGHT · 28.0s
gpt-5.5
72%BALONEY
ROUND-1 BREAKDOWN
Evidence offered for the claim- Broad expert agreement is often strong evidence that a conclusion is well supported, especially when it reflects many independent lines of evidence rather than a simple vote. For example, NASA summarizes peer-reviewed studies finding that 97% or more of actively publishing climate scientists agree recent warming is human-caused, and major assessments treat that conclusion as settled for practical policy purposes.
- Scientific institutions use consensus processes because expert agreement, when evidence-based and transparent, is a reliable way to summarize the state of knowledge. The National Academies’ consensus reports are designed to synthesize available evidence and expert judgment for decision-makers.
- Some scientific conclusions become so strongly supported that they are extremely unlikely to be overturned in their core form, even if details are refined—e.g., climate change being human-caused, germ theory, evolution, or plate tectonics.
Evidence against the claim- The claim says “settled forever,” which is too absolute. Science is inherently provisional: even strong consensus can be revised, refined, or replaced if better evidence appears. The National Academies describes scientific knowledge as based on evidence and subject to change with new observations and interpretations.
- History shows that majority scientific views can change. The Nobel Prize organization notes that Marshall and Warren’s discovery of Helicobacter pylori transformed understanding of gastritis and peptic ulcers, overturning the older emphasis on stress/lifestyle as primary causes and leading to antibiotic treatment.
- Plate tectonics is another example: USGS describes how Wegener’s continental drift idea initially had little support but later evidence from seafloor mapping and geophysics led to the modern plate tectonics consensus.
- Consensus is not the same as proof, and “most scientists agree” is weaker than “the evidence is strong.” A majority can be premature, field-dependent, or based on incomplete data. Scientific consensus is valuable because of the evidence behind it, not because agreement makes a claim permanently true.
- Therefore, the statement as written is misleading: broad agreement can justify treating a conclusion as settled for current purposes, but not “forever.”
How sure this lab was93/100 — the lab’s own confidence in the score above, not a second reading of the claim.
SourcesNASAgov · 95 · National Academies Pressother · 95 · National Academies of Sciences, Engineering, and Medicineother · 94 · Nobel Prizeother · 90 · U.S. Geological Surveygov · 94 · Environmental Research Lettersjournal · 88
Google · EQUAL WEIGHT · 32.1s
gemini-3.1-pro-preview
85%BALONEY
ROUND-1 BREAKDOWN
Evidence offered for the claim- A strong scientific consensus represents the most reliable, rigorously tested knowledge available at any given time, meaning the likelihood of foundational theories (like gravity, cell theory, or evolution) being entirely discarded is extraordinarily low.
- Colloquially, science communicators and policymakers sometimes use the shorthand phrase 'the science is settled' to emphasize that a theory—such as anthropogenic climate change—has passed sufficient empirical testing to justify immediate societal action, rather than waiting for an impossible standard of absolute certainty.
Evidence against the claim- The scientific method is fundamentally rooted in falsifiability and empirical testing. Epistemologically, science never 'settles' anything forever; theories are always open to revision, refinement, or rejection in the face of new, reproducible evidence.
- The history of science is filled with 'paradigm shifts' (a concept popularized by philosopher Thomas Kuhn) where an overwhelming scientific consensus was eventually overturned. Examples include the transition from the geocentric to the heliocentric model of the solar system, the rejection of the miasma theory of disease in favor of germ theory, and the updating of classical Newtonian mechanics by Einstein's theory of relativity.
- Major scientific organizations and philosophers of science explicitly state that scientific knowledge is tentative. The willingness to change conclusions based on new data is the primary feature that distinguishes science from dogma.
How sure this lab was98/100 — the lab’s own confidence in the score above, not a second reading of the claim.
SourcesStanford Encyclopedia of Philosophyedu · 95 · Stanford Encyclopedia of Philosophyedu · 95 · NASAgov · 98
Perplexity · EQUAL WEIGHT · 7.9s
sonar-pro
68%BALONEY
ROUND-1 BREAKDOWN
Evidence offered for the claim- Scientific consensus is meaningful evidence: it reflects the collective assessment of the relevant expert community and is based on the quality and quantity of available evidence.[https://www.nationalacademies.org/read/27894/chapter/4]
- For well-established theories, consensus is usually supported by multiple independent lines of confirming evidence, making abrupt reversal unlikely.[https://www.nationalacademies.org/read/27894/chapter/4]
- Some scientific conclusions, such as core aspects of germ theory, evolution, and conservation of energy, are sufficiently well supported that they are considered settled within defined domains, even though research continues.[https://www.nationalacademies.org/read/27894/chapter/4]
Evidence against the claim- The word "forever" makes the claim false: the National Academies states that scientific uncertainty can never reach zero, knowledge is partial, and new evidence can call a current consensus into question.[https://www.nationalacademies.org/read/27894/chapter/4]
- Scientific consensus is historically contingent and can change as evidence, methods, and interpretations change; research on consensus formation documents trajectories in which propositions are contested, revisited, and sometimes reformulated.[https://pmc.ncbi.nlm.nih.gov/articles/PMC3163460/]
- Scientific theories can remain highly reliable while being revised or replaced in some domains. Newtonian mechanics and classical gravity were superseded or generalized by later theories, and even Einstein's theory is not complete in every regime.[https://www.livescience.com/human-behavior/politics/there-is-such-a-thing-as-settled-science-anyone-who-says-otherwise-is-trying-to-manipulate-you-opinion]
- The National Academies distinguishes well-established science from emerging science and explicitly says that conclusions should be judged against the evidence available at the time, rather than treated as permanently final.[https://www.nationalacademies.org/read/27894/chapter/4]
How sure this lab was97/100 — the lab’s own confidence in the score above, not a second reading of the claim.
SourcesNational Academies of Sciences, Engineering, and Medicinegov · 98 · American Sociological Association journal article available through PubMed Centraljournal · 92 · Live Sciencenews · 78
xAI · EQUAL WEIGHT · 97.1s
grok-4.6
74%BALONEY
ROUND-1 BREAKDOWN
Evidence offered for the claim- Strong, independently replicated consensus on highly tested findings (e.g., heliocentrism, germ theory, DNA as genetic material) has proven extremely reliable for prediction and application, so reversal of the core claim is practically improbable on human timescales.
- Bodies such as NASA treat very high expert agreement (e.g., ~97% of publishing climate scientists on human-caused warming) as indicating that the core science is robust enough to inform policy.
Evidence against the claim- Science is methodologically provisional: conclusions remain open to revision or falsification by new evidence and are never declared settled forever (Popper, scientific method).
- 'Most scientists agree' is a weak threshold (potentially just over 50%) and historically insufficient: medical consensus long held peptic ulcers were caused by stress/acid until H. pylori was established, recognized by the 2005 Nobel Prize.
- Major consensus reversals include rejection then acceptance of continental drift/plate tectonics and the supersession of Newtonian gravity by general relativity (still useful as an approximation, not final).
- Recent rapid revisions (COVID-19 transmission/origins/treatments, nutrition guidelines, cosmology) show even active consensus is not permanent.
- The absolute 'forever' clause is incompatible with how science actually progresses via anomalies, better measurements, and paradigm change.
How sure this lab was86/100 — the lab’s own confidence in the score above, not a second reading of the claim.
SourcesNASAgov · 95 · The Nobel Prizeother · 98 · Stanford Encyclopedia of Philosophyedu · 94 · Encyclopædia Britannicaother · 87
Station 2 · The Masters
Primary Master · drawn at random
gemini-3.1-pro-preview
85% BALONEY
The evaluators universally found the claim to be false due to the absolutist phrase 'settled forever.' While scientific consensus is a highly reliable indicator of the best available evidence at a given time and is heavily relied upon by institutions for policy and decision-making, the fundamental nature of the scientific method requires that all conclusions remain provisional and open to testing. History is replete with examples of dominant scientific consensuses—such as classical Newtonian mechanics or the understanding of peptic ulcers—that were later overturned or radically revised by new, reproducible evidence. Treating a consensus as permanently 'settled' violates core scientific epistemology.
The evaluators completely agreed on both the value and the limitations of scientific consensus. They consistently noted that while consensus is practically useful and indicative of strong evidence, science is inherently self-correcting and no theory is immune to revision. All evaluators independently provided historical examples of paradigm shifts to demonstrate that majority agreement does not guarantee permanence.
Blind Validation Master · never saw the Primary’s work
sonar-pro
94% BALONEY
The statement is overwhelmingly false as written because “forever” makes an absolute claim that scientific evidence and methodology cannot justify. The reports consistently agree that broad expert consensus is valuable evidence and often makes a conclusion highly reliable for current decisions, particularly when supported by independent replication and multiple lines of evidence. However, consensus is a proxy for the underlying evidence, not proof or permanent finality. Scientific conclusions remain revisable, and the reports cite major historical changes including the shift from geocentrism to heliocentrism, miasma theory to germ theory, rejection and later acceptance of continental drift, the reinterpretation of Newtonian mechanics by relativity, and the discovery of H. pylori as a cause of peptic ulcers. The phrase “most scientists agree” is also underspecified: it could mean a narrow majority, agreement outside the relevant field, or a consensus based on incomplete or unreliable evidence. A more accurate claim would be that strong, independently supported scientific consensus is usually the best available guide and may be treated as settled for practical purposes, while remaining open to revision. Because the original wording presents consensus as making science permanently settled, it belongs in the radioactive-baloney range.
AI EVALUATORS SHOWED MODERATE DISAGREEMENT — MASTERS SCORED 85% AND 94% · REVIEW THE EVIDENCE CAREFULLY
How the number was reached · scoring model v2 · how our models work
- Median of 5 counted lab scores: 87, 72, 85, 68, 74 → 74.
The Evidence
Evidence offered for the claim
- Consensus indicates reliable knowledge DEBUNKED — Expert agreement often summarizes multiple independent lines of rigorous evidence, meaning that foundational theories (like heliocentrism or germ theory) are practically very unlikely to be entirely overturned.
- Shorthand for actionable science DEBUNKED — Policymakers and communicators frequently use the shorthand 'the science is settled' to emphasize that a theory has passed sufficient empirical testing to inform immediate societal action.
Evidence against the claim
- Falsifiability is fundamental The scientific method fundamentally relies on testing and falsifiability. Major scientific institutions state that knowledge is always provisional, and uncertainty never reaches absolute zero.
- Historical paradigm shifts Scientific history documents numerous instances where an overwhelming majority was overturned by new data, including the transition to plate tectonics, the acceptance of H. pylori as the cause of ulcers, and the discovery of quasicrystals.
- Consensus is not proof 'Most scientists agree' is a proxy for evidence, not a replacement for it. Replication research has shown that published consensus can occasionally be premature, field-dependent, or based on incomplete data.
Sources · Reliability · Why Accepted or Discounted
| Source | Type | Reliability | Ruling |
|---|
| NASA | gov | 95 | ACCEPTED — Authoritative government science agency illustrating how consensus is measured and applied in practice. |
| National Academies Press | other | 95 | ACCEPTED — Highly respected academic publishing body detailing epistemological foundations of science. |
| National Academies of Sciences, Engineering, and Medicine | other | 94 | ACCEPTED — Major scientific institution explaining its internal evidence-synthesis and consensus processes. |
| Nobel Prize | other | 90 | ACCEPTED — Official press release documenting a famous paradigm shift in medicine. |
| U.S. Geological Survey | gov | 94 | ACCEPTED — Authoritative government resource summarizing the historical shift to plate tectonics. |
| Environmental Research Letters | journal | 88 | ACCEPTED — Peer-reviewed scientific journal documenting measurements of consensus. |
| Stanford Encyclopedia of Philosophy | edu | 95 | ACCEPTED — Gold-standard philosophical reference on falsifiability and the scientific method. |
| Stanford Encyclopedia of Philosophy | edu | 95 | ACCEPTED — Reputable academic reference on the history and philosophy of paradigm shifts. |
| NASA | gov | 98 | ACCEPTED — Authoritative agency overview on the scientific consensus regarding anthropogenic climate change. |
| National Academies of Sciences, Engineering, and Medicine | gov | 98 | ACCEPTED — Rigorous institutional report defining the nature of scientific evidence and uncertainty. |
| PubMed Central | journal | 92 | ACCEPTED — Peer-reviewed sociological study on how scientific consensus forms and changes. |
| Live Science | news | 78 | ACCEPTED — Popular science article offering a nuanced perspective on 'settled' science and its rhetorical uses. |
| Stanford Encyclopedia of Philosophy | edu | 94 | ACCEPTED — Highly reliable academic reference defining the core tenets of the scientific method. |
| Encyclopædia Britannicaunverified | other | 87 | ACCEPTED — Standard encyclopedia providing basic context on scientific methodology. |
| U.S. Geological Survey | gov | 95 | ACCEPTED — Trusted historical account of geological consensus development. |
| Nobel Prize Organisation | other | 92 | ACCEPTED — Primary historical source documenting the H. pylori discovery that overturned an established medical consensus. |
| Nobel Prize Organisation | other | 92 | ACCEPTED — Primary historical source for the discovery of quasicrystals, illustrating resistance to new findings. |
| NIH / NIDDK | gov | 95 | ACCEPTED — Authoritative medical body detailing the current understanding of peptic ulcers. |
| Scienceunverified | journal | 93 | ACCEPTED — Landmark peer-reviewed study analyzing replication failure rates in published literature. |
| PLOS Medicine | journal | 85 | ACCEPTED — Widely cited meta-research paper discussing vulnerabilities in published scientific consensus. |
Every link was opened when this claim was inspected. A dead link was discarded before publication; “unverified” means the site refused our automated check and the link is the one the panel cited. A source without a URL was never kept.
Serve It Fresh
Ready to Serve10 servings
Serve it the way you want it
The Verdict
“If most scientists agree, the science is settled forever.”74% Radioactive Baloney — Dangerously ToxicVerified blind by 5 frontier AIs · Baloney Inspection Report: baloney.ai/baloney/if-most-scientists-agree-the-science-is-settled-forever