Agreement between the first coding and a blind second coder

What this dataset measures

Stories were collected from the outlets named in the audit design: mainstream and public media (18 outlets), conservative outlets (6) and progressive advocacy media (2), found through outlet search and tag pages, sitemaps, web search and syndicated copies. The aim was every substantive I-645 story we could find from each outlet in the window, with at least two per outlet where they existed. Pure link roundups, letters to the editor, and video-only pages with no text were excluded.

Geography

Washington State, United States

Unit of observation

One story (news, explainer, fact check, broadcast write-up, opinion piece or editorial)

Update cadence

One-time release; later versions only for corrections

Curator

FactHub Research Team

Curated data tables

Agreement between the first coding and a blind second coder

Percent agreement and Cohen's kappa on a stratified random sample of 20 news stories (14 mainstream, 6 conservative; seed 645).

Unit of observation: Coded field

media_recode_agreement.csv text/csv · 1,022 bytes

All 24 rows 6 columns Download the file

All 24 rows

field	stories stories	agree stories	percent_agreement percent	cohens_kappa	disagreements
O1	20	20	100	1	—
O2	20	20	100	1	—
O3a	20	17	85	0.7	SR-10M:0→1 R4:0→1 M4:0→1
O3b (first pass, before re-pass)	20	18	90	0.46	K08:0→1 K07:0→1
O3b (after re-pass)	20	20	100	1	—
O4 (first pass, before re-pass)	20	18	90	0.62	K07:0→1 PUB-12:0→1
O4 (after re-pass)	20	20	100	1	—
O5	20	18	90	0.46	K08:1→0 K03:1→0
O6	20	20	100	—	—
O7	20	19	95	0.89	ST-EXPAND:0→1
O8	20	20	100	—	—
O9	20	20	100	—	—
first_word	20	19	95	0.9	R4:none→no
last_word	20	17	85	0.72	PUB-20:neutral→yes R4:none→yes PUB-01:neutral→no
rebuttal	20	17	85	0.63	EH-WSS:Y→N C2:Y→N C1:N→Y
source_lean	20	20	100	1	—
any_unchecked_no_claim	20	18	90	0.69	R1:false→true M4:true→false
any_unchecked_yes_claim	20	19	95	0.9	R1:true→false
claim_N13B	20	17	85	0.33	R1:absent→unchecked S10:checked→absent M4:unchecked→absent
claim_N460K	20	20	100	1	—
claim_NMEALS	20	20	100	1	—
claim_YCOME	20	19	95	0.9	K08:unchecked→absent
claim_YPENS	20	20	100	1	—
claim_YLEAVE	20	20	100	1	—

6 field definitions

Field	Type	Meaning	Units	Value type
field	string	Coded field; O3b and O4 shown before and after the voice re-pass.	—	derived
stories	integer	Stories compared.	stories	derived
agree	integer	Stories where both coders agreed.	stories	derived
percent_agreement	number	Agreement.	percent	derived
cohens_kappa	number	Cohen's kappa (blank when one category only).	—	derived
disagreements	string	story_id:first→second for each disagreement.	—	derived

Supporting files

Source snapshots, manifests, quality reports, documentation, and other materials used to produce or review this release.

• media_derived.json Derived table · application/json · 41.5 KB (42,472 bytes)

  The summary tables as JSON (omission, claims, sourcing, chartRows), used for the saved analyses.

• media_recode_agreement.json Quality report · application/json · 2.7 KB (2,746 bytes)

  Agreement results as JSON.

• media_recode_second_coder.md Quality report · text/markdown · 10 KB (10,263 bytes)

  The blind second coder's codes and justifications for the 20 sampled stories.

• media_coding_codebook.md Data dictionary · text/markdown · 7.9 KB (8,114 bytes)

  The codebook: every column, claim code and omission fact, with definitions.

• media_coding_overrides.json Transformation manifest · application/json · 1.2 KB (1,191 bytes)

  The voice re-pass for O3b and O4 and the N460K correction, applied after the reliability check.

• media_audit.js Transformation manifest · text/javascript · 8.7 KB (8,906 bytes)

  Merges the coders' notes into the stories table and applies the overrides.

• media_audit_derived.js Transformation manifest · text/javascript · 8.7 KB (8,949 bytes)

  Computes every summary table from the stories table.

• media_recode_agreement.js Transformation manifest · text/javascript · 4.4 KB (4,480 bytes)

  Computes percent agreement and Cohen's kappa against the second coder.

• media_dataset_parts.json Documentation · application/json · 49.4 KB (50,565 bytes)

  Sources, table and field definitions, and headline rows for this release, generated by media_dataset_build.js.

• media_dataset_build.js Transformation manifest · text/javascript · 15.2 KB (15,559 bytes)

  Builds the sources, table definitions and headline rows.

Sources

1. Seattle Times Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

   Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

   Reliability: News report. News publisher whose coverage is the subject of the coding; its I-645 stories (June 1 to Oct 21, 2026) are rows in the stories table. It has no stake in the counts beyond its own coverage.

2. Spokesman-Review Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

   Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

   Reliability: News report. News publisher whose coverage is the subject of the coding; its I-645 stories (June 1 to Oct 21, 2026) are rows in the stories table. It has no stake in the counts beyond its own coverage.

3. The Columbian Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

   Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

   Reliability: News report. News publisher whose coverage is the subject of the coding; its I-645 stories (June 1 to Oct 21, 2026) are rows in the stories table. It has no stake in the counts beyond its own coverage.

4. Everett Herald Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

   Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

   Reliability: News report. News publisher whose coverage is the subject of the coding; its I-645 stories (June 1 to Oct 21, 2026) are rows in the stories table. It has no stake in the counts beyond its own coverage.

5. Tri-City Herald Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

   Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

   Reliability: News report. News publisher whose coverage is the subject of the coding; its I-645 stories (June 1 to Oct 21, 2026) are rows in the stories table. It has no stake in the counts beyond its own coverage.

6. Washington State Standard Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

   Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

   Reliability: News report. News publisher whose coverage is the subject of the coding; its I-645 stories (June 1 to Oct 21, 2026) are rows in the stories table. It has no stake in the counts beyond its own coverage.

7. KUOW Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

   Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

   Reliability: News report. News publisher whose coverage is the subject of the coding; its I-645 stories (June 1 to Oct 21, 2026) are rows in the stories table. It has no stake in the counts beyond its own coverage.

8. Cascade PBS Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

   Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

   Reliability: News report. News publisher whose coverage is the subject of the coding; its I-645 stories (June 1 to Oct 21, 2026) are rows in the stories table. It has no stake in the counts beyond its own coverage.

9. The Stranger Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

   Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

   Reliability: News report. News publisher whose coverage is the subject of the coding; its I-645 stories (June 1 to Oct 21, 2026) are rows in the stories table. It has no stake in the counts beyond its own coverage.

10. Deseret News Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

    Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

    Reliability: News report. News publisher whose coverage is the subject of the coding; its I-645 stories (June 1 to Oct 21, 2026) are rows in the stories table. It has no stake in the counts beyond its own coverage.

11. Cascadia Daily News Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

    Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

    Reliability: News report. News publisher whose coverage is the subject of the coding; its I-645 stories (June 1 to Oct 21, 2026) are rows in the stories table. It has no stake in the counts beyond its own coverage.

12. KOMO Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

    Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

    Reliability: News report. News publisher whose coverage is the subject of the coding; its I-645 stories (June 1 to Oct 21, 2026) are rows in the stories table. It has no stake in the counts beyond its own coverage.

13. KING 5 Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

    Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

    Reliability: News report. News publisher whose coverage is the subject of the coding; its I-645 stories (June 1 to Oct 21, 2026) are rows in the stories table. It has no stake in the counts beyond its own coverage.

14. KIRO 7 Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

    Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

    Reliability: News report. News publisher whose coverage is the subject of the coding; its I-645 stories (June 1 to Oct 21, 2026) are rows in the stories table. It has no stake in the counts beyond its own coverage.

15. KXLY Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

    Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

    Reliability: News report. News publisher whose coverage is the subject of the coding; its I-645 stories (June 1 to Oct 21, 2026) are rows in the stories table. It has no stake in the counts beyond its own coverage.

16. Fox 13 Seattle Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

    Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

    Reliability: News report. News publisher whose coverage is the subject of the coding; its I-645 stories (June 1 to Oct 21, 2026) are rows in the stories table. It has no stake in the counts beyond its own coverage.

17. Axios Seattle Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

    Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

    Reliability: News report. News publisher whose coverage is the subject of the coding; its I-645 stories (June 1 to Oct 21, 2026) are rows in the stories table. It has no stake in the counts beyond its own coverage.

18. KREM Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

    Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

    Reliability: News report. News publisher whose coverage is the subject of the coding; its I-645 stories (June 1 to Oct 21, 2026) are rows in the stories table. It has no stake in the counts beyond its own coverage.

19. The Center Square Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

    Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

    Reliability: News report. News publisher whose coverage is the subject of the coding; its I-645 stories (June 1 to Oct 21, 2026) are rows in the stories table. It has no stake in the counts beyond its own coverage.

20. Seattle Red/KTTH Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

    Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

    Reliability: News report. News publisher whose coverage is the subject of the coding; its I-645 stories (June 1 to Oct 21, 2026) are rows in the stories table. It has no stake in the counts beyond its own coverage.

21. MyNorthwest/KIRO Newsradio Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

    Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

    Reliability: News report. News publisher whose coverage is the subject of the coding; its I-645 stories (June 1 to Oct 21, 2026) are rows in the stories table. It has no stake in the counts beyond its own coverage.

22. Washington Policy Center Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

    Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

    Reliability: Advocacy group. Free-market think tank that argued for I-645 (repeal); its staff are quoted as Yes-side sources. Its own I-645 posts are coded rows (opinion).

23. The Daily Wire Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

    Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

    Reliability: News report. News publisher whose coverage is the subject of the coding; its I-645 stories (June 1 to Oct 21, 2026) are rows in the stories table. It has no stake in the counts beyond its own coverage.

24. Lynnwood Times Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

    Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

    Reliability: News report. News publisher whose coverage is the subject of the coding; its I-645 stories (June 1 to Oct 21, 2026) are rows in the stories table. It has no stake in the counts beyond its own coverage.

25. NPI Cascadia Advocate Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

    Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

    Reliability: Party to the dispute. Northwest Progressive Institute, part of the coalition opposing I-645; it published the No coalition's legal memo. Its blog posts are coded rows (opinion).

26. The Stand Publisher of coded I-645 stories. Each story's exact URL is in the stories table (column url); the publisher's site is listed here.

    Retrieved October 21, 2026. All rights reserved (publisher copyright). Story text is not redistributed. The dataset holds coding, story metadata and quotes of 25 words or fewer.

    Reliability: Party to the dispute. News site of the Washington State Labor Council, AFL-CIO, which opposes I-645 and whose president debated for the No side. Its posts are coded rows.

27. Washington Research Council, "New brief: I-645: Repealing the State Individual Income Tax" (Sept. 22, 2026) Anchor for facts O3b (no current appropriation relies on the tax) and O4 (the 2027-29 outlook assumes the revenue). Free copy

    Retrieved October 21, 2026. All rights reserved. Linked only.

    Reliability: Other. Business-funded nonpartisan fiscal research group. Its brief states both that no current appropriation relies on the tax and that the four-year outlook assumes $2.698 billion from it in 2027-29.

28. Washington Secretary of State, 2026 General Election Voters' Pamphlet (Edition 10, Spokane) Anchor for fact O1 (the Attorney General's explanatory statement says the tax relief would remain in place) and for the ballot sentence text.

    Retrieved October 21, 2026. Washington State government publication. Linked only.

    Reliability: Government. Official state publication. The explanatory statement and ballot disclosure are written by the Attorney General's office; the statewide text is the same in every edition.

29. Economic and Revenue Forecast Council, enacted 2026 supplemental budget outlook (Apr. 22, 2026) Anchor for fact O4: the outlook counts ESSB 6346 revenue in 2027-29.

    Retrieved October 21, 2026. Washington State government publication. Linked only.

    Reliability: Government. State council that adopts the official four-year balanced-budget outlook; lists ESSB 6346 at +$2,309.18 million (net) for 2027-29.

Limitations and reproducibility

Partially reproducible: The coded rows are judgments by trained coders following media_coding_codebook.md; a reader can re-read any story at its URL and recode it. Every summary table is recomputed exactly from the stories table by media_audit_derived.js (node media_audit_derived.js), and the reliability table by media_recode_agreement.js from media_recode_second_coder.md. media_audit.js, which merged the coders' working notes, is included for transparency, but those working notes are not published.

• Not a random sample. Stories were found by search, so outlets whose sites block automated reading (Tacoma News Tribune, Bellingham Herald, The Olympian) are missing and the Tri-City Herald has one story. No AP story on I-645 was found.
• TV stories were coded from their web text, not the broadcast video.
• Coding involves judgment. A blind second coder recoded 20 randomly drawn news stories: agreement was 85-100% on every field (Cohen's kappa 0.33-1.0), 100% on which side had more sources and on the relief-survives fact.
• The second-coder check showed the first coding missed several mentions of two budget facts (O3b, O4) and one claim (N460K) that appeared only inside advocates' quotes. Every news story's text was then searched for them and each mention was classified by voice (media_coding_overrides.json). O3b and O4 counts include both voices; O3b_voice and O4_voice separate them.
• The relief fact (O1) and the other omission facts were not split by voice; they mostly appear in reporter voice.
• Outlet groups follow the audit design, not a measurement of each outlet's slant. The Stranger is grouped as mainstream though it is an openly progressive weekly; MyNorthwest/KIRO Newsradio is grouped as conservative though two of its rows are straight news.
• Per-outlet counts are small (1 to 14 stories) and describe this sample, not the outlet's overall coverage.
• An omitted fact is not necessarily an error: short event stories cannot carry every fact. Omission rates measure what readers of these stories were told, not intent.
• Sample size for the news groups: 58 mainstream news stories, all read in full. An earlier internal count used 53-54 because five KING 5 and KREM stories had not yet been read (they were later read in a browser) and one paywalled Columbian story was only partly read; that story is now excluded from the news groups so every fact shares one denominator.