Part IV

Methods and Replication

How the dataset was built and checked, and how to re-derive every registered number yourself.

The dataset behind the Note’s Part II did not exist before this project. Court opinions were retrieved, screened by a frozen prompt, and classified by several separately run AI models from different providers, with disagreements resolved under prespecified adjudication rules; the headline finding has its own separate ensemble, audited blind by a different vendor’s models. The models classify — answering fixed questions about each opinion; the author directed the project and made all legal and interpretive judgments. Validation measures reproducibility across independent classifiers, not accuracy against a human-coded benchmark, and every layer is published with its instruments.

The method, documented

• Methodology — pipeline stages, model roster, prompts, and the publication boundary, with a guide to running the method on a different legal question.
• System map — the one-page data-flow map, with a written text equivalent.
• Validation — the published validation layers and what each does and does not establish.
• Frozen prompts — the classification instruments as run.

Reproduce it

One command runs the deterministic release gate — no network access, API keys, or spend — and re-derives the registered series from the committed record:

python scripts/run_release_checks.py

• Reproduction guide — what is deterministic, what needs model access, and the commands for each analysis.
• The release gate — every check, what a green run establishes, and what it deliberately does not.
• Verify one claim, end to end — a complete worked example from printed sentence to generating artifact.
• Sample and denominator definitions — executable predicates for every population the paper counts.

Where replication stops

Full case texts are not redistributed; source identifiers and hashes are preserved so each text can be re-obtained. End-to-end corpus reconstruction requires upstream working files retained privately; the frozen canonical record published in the repository is the replication baseline. The evidence and limits page states what remains judgment rather than mechanics.