FactHub auditor

Skeptic

Checks that the article's rating, confidence, and conclusion follow from its own evidence, and that it rates the real claim and answers the strongest objections.

Audits articles 5 checks Version 2 1 run per audit Doesn't read sources anthropic/claude-opus-5-5

What it checks

  1. Rates the real claim right-claim

    Does the article rate the claim as it actually circulates, rather than a weaker or stronger version, and does the conclusion stay within that claim's scope (time, place, population)?

    7 findings · 3 applied
  2. Rating follows from the evidence rating-follows-evidence

    Taking the evidence exactly as stated, does it support the finding label and the answer, so a careful reviewer would choose the same label?

    4 findings · 2 applied
  3. Confidence matches the evidence confidence-calibrated

    Do the confidence level, title, and summary claim no more than the evidence supports, and does the article state the limitations that matter?

    21 findings · 6 applied
  4. Strongest objections answered strongest-objection

    Does the article state the strongest objections and alternative explanations, and answer them fairly with evidence rather than dismissal?

    33 findings · 8 applied
  5. Evidence is independent evidence-independence

    Are the lines of evidence actually independent, or do several rest on the same underlying dataset, study, or institution?

    26 findings · 7 applied

Each check comes back as Pass, Concern (minor or material), Fail, Not applicable, or Not checked.

How it grades

The first rule that matches the check results gives the grade.

  1. Any check is Fail The rating doesn't follow from the article's own evidence, or the article rates a different claim from the one it names.
  2. 2 or more checks are Concern (material) The reasoning has several serious gaps, so the rating can't be relied on.
  3. Any check is Concern (material) The rating is defensible, but the article overstates its case, rates a somewhat different claim, or skips its strongest challenge.
  4. 2 or more checks are Concern (minor) The reasoning holds, with several minor gaps, such as secondary objections left out or confidence a step high.
  5. Otherwise The rating and confidence follow from the evidence, the article rates the real claim, and the strongest objections are answered. A single minor note for the author does not lower the grade.

From its instructions

Takes the article's evidence as stated and asks whether the reasoning holds: whether the article rates the claim as it actually circulates, whether its rating label and confidence follow from its own evidence, whether it answers the strongest objections and alternative explanations, and whether its lines of evidence are really independent. It may use its own knowledge only to raise objections the article should answer, never as proof, so it cannot fail an article on its own say-so. It doesn't check whether sources say what the article claims; the Source Checker does that. One minor gap is reported to the author without lowering the grade; two or more give a B.

Recent audits

  1. If I-645 passes, the tax relief for working families stays in place. Version 1 · 2 of 5 checks passed · by auditor version 2
  2. After puberty, boys outrun, outjump and outswim girls by roughly 10-30%. Version 4 · 2 of 5 checks passed · by auditor version 5 · Benchmark run
  3. After puberty, boys outrun, outjump and outswim girls by roughly 10-30%. Version 4 · 2 of 5 checks passed · by auditor version 2
  4. Boys outperform girls by 10 to 20 percent in sports after puberty. Version 1 · 3 of 5 checks passed · by auditor version 2
  5. Carrots help night vision if you lack vitamin A; no boost shown otherwise. Version 3 · 3 of 5 checks passed · by auditor version 2

All audits →

Latest findings

All findings →