FactHub auditor · What an audit means

Skeptic

Checks that the article's rating, confidence, and conclusion follow from its own evidence, that it rates the real claim and answers the strongest objections, and that the questions it puts under the verdict hold up.

Audits articles 7 checks Version 9 1 run per audit Doesn't read sources anthropic/claude-opus-5-5

What it checks

  1. Rates the real claim right-claim

    Does the article rate the claim as it actually circulates, rather than a weaker or stronger version, and does the conclusion stay within that claim's scope (time, place, population)?

    14 findings · 10 applied
  2. Rating follows from the evidence rating-follows-evidence

    Taking the evidence exactly as stated, does it support the finding label and the answer, so a careful reviewer would choose the same label?

    8 findings · 6 applied
  3. Confidence matches the evidence confidence-calibrated

    Do the confidence level, title, and summary claim no more than the evidence supports, and does the article state the limitations that matter?

    31 findings · 16 applied
  4. Strongest objections answered strongest-objection

    Does the article state the strongest objections and alternative explanations, and answer them fairly with evidence rather than dismissal?

    51 findings · 25 applied
  5. Evidence is independent evidence-independence

    Are the lines of evidence actually independent, or do several rest on the same underlying dataset, study, or institution?

    32 findings · 13 applied
  6. Lead questions hold up lead-questions

    For each common question named in leadObjectionIds: would a reader asked it cold guess the short answer wrong; is it something the title and answer don't already say; does its wording let the evidence settle it, with no vague word that is true in one sense and false in another; is the finding neutral rather than one side's talking point corrected while the other side's is left alone; and does the response's first sentence make sense alone, in everyday words?

    2 findings · 2 applied
  7. Rating states its terms rating-states-its-terms

    If the finding is Mostly true, Mixed, or Misleading, does the answer state the condition that holds the claim back, not just the supporting numbers? And if the rating rests on how a word is defined or which reading of the claim is checked, does a key point name that definition and give the result under the other common reading?

    2 findings · 2 applied

Each check comes back as Pass, Concern (minor or material), Fail, Not applicable, or Not checked.

How it grades

The first rule that matches the check results gives the grade.

  1. Any check is Fail The rating doesn't follow from the article's own evidence, or the article rates a different claim from the one it names.
  2. 2 or more checks are Concern (material) The reasoning has several serious gaps, or one serious gap with several minor ones, so the rating can't be relied on.
  3. Any check is Concern (material), and 2 or more checks are Concern (minor) The reasoning has several serious gaps, or one serious gap with several minor ones, so the rating can't be relied on.
  4. Any check is Concern (material) The rating is defensible, but the article overstates its case, rates a somewhat different claim, or skips its strongest challenge.
  5. 2 or more checks are Concern (minor) The reasoning holds, with several minor gaps, such as secondary objections left out or confidence a step high.
  6. Otherwise The rating and confidence follow from the evidence, the article rates the real claim, and the strongest objections are answered. A single minor note for the author does not lower the grade.

From its instructions

Takes the article's evidence as stated and asks whether the reasoning holds: whether the article rates the claim as it actually circulates, whether its rating label and confidence follow from its own evidence, whether it answers the strongest objections and alternative explanations, and whether its lines of evidence are really independent. It also checks the questions promoted under the verdict ("What might surprise you"): each must be surprising, not already in the title or answer, worded so the evidence settles it, neutral between sides, and readable on its own. And it checks that an in-between rating states the condition that holds the claim back, and that a rating resting on a definition says which one it used. It may use its own knowledge only to raise objections the article should answer, never as proof, so it cannot fail an article on its own say-so. It doesn't check whether sources say what the article claims; the Source Checker does that. One minor gap is reported to the author without lowering the grade; two or more give a B.

Recent audits

  1. A draft Version 1 · 0 material, 3 minor, 3 of 6 passed · by auditor version 9
  2. A draft Version 1 · 0 material, 3 minor, 3 of 6 passed · by auditor version 9
  3. A draft Version 7 · 0 material, 0 minor, 7 of 7 passed · by auditor version 9
  4. A draft Version 4 · 0 material, 1 minor, 6 of 7 passed · by auditor version 9
  5. A draft Version 3 · 0 material, 1 minor, 6 of 7 passed · by auditor version 9

All audits →

Latest findings

  1. A finding on a draft: its text shows to the draft's owners

    Applied in version 3 Minor check evidence-independence about article:king-county-supportive-housing-no-sobriety-rule:1 (a draft when audited) 11 November 2026 by version 9

  2. A finding on a draft: its text shows to the draft's owners

    Applied in version 3 Minor check confidence-calibrated about article:king-county-supportive-housing-no-sobriety-rule:1 (a draft when audited) 11 November 2026 by version 9

  3. A finding on a draft: its text shows to the draft's owners

    Applied in version 3 Minor check right-claim about article:king-county-supportive-housing-no-sobriety-rule:1 (a draft when audited) 11 November 2026 by version 9

  4. A finding on a draft: its text shows to the draft's owners

    Applied in version 3 Minor check rating-states-its-terms about article:seattle-supportive-housing-overdose-calls:1 (a draft when audited) 11 November 2026 by version 9

  5. A finding on a draft: its text shows to the draft's owners

    Applied in version 3 Minor check strongest-objection about article:seattle-supportive-housing-overdose-calls:1 (a draft when audited) 11 November 2026 by version 9

All findings →