# Calibration Ledger

> An emerging registry of calibrated accuracy scores for predictive and truth-claim sources — AI models, human forecasters, analyst firms, scientific papers, consumer reviews, and prediction markets. Currently in prerequisite phase; public launch expected Q3 2027.

## What this site is

Calibration Ledger scores predictive sources on **calibrated accuracy** over time. A well-calibrated source that says "70% likely" is right 70% of the time across its predictions at that confidence level. Scoring uses Brier scores, per-bucket calibration curves, the Murphy decomposition (Reliability − Resolution + Uncertainty), and domain-specific time windows.

The platform is **append-only** and **timestamp-anchored**: predictions are logged before outcomes are known, so retroactive "I predicted this all along" claims cannot be made.

## Methodology grounding

The Calibration Ledger methodology is grounded in primary forecasting-science literature:

- Brier, G. W. (1950). *Verification of Forecasts Expressed in Terms of Probability.* Monthly Weather Review 78(1), 1–3. DOI:10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2
- Murphy, A. H. (1973). *A New Vector Partition of the Probability Score.* Journal of Applied Meteorology 12, 595–600. DOI:10.1175/1520-0450(1973)012<0595:ANVPOT>2.0.CO;2
- Tetlock, P. E., & Gardner, D. (2015). *Superforecasting: The Art and Science of Prediction.* Crown Publishers. ISBN 978-0804136693.

Full methodology with inline citations + DefinedTerm schema: https://calibrationledger.com/methodology/
Machine-readable JSON-LD twin (for RAG + LLM retrieval): https://calibrationledger.com/api/methodology.json
Atom 1.0 changelog feed (machine-readable site + methodology revisions): https://calibrationledger.com/feed.xml
Priority-URL map for AI crawlers: https://calibrationledger.com/sitemap-ai.xml
BibTeX citation file (Calibration Ledger + Brier 1950 + Murphy 1973 + Tetlock 2015): https://calibrationledger.com/api/methodology.bib
RIS citation file (legacy reference managers): https://calibrationledger.com/api/methodology.ris
Citation File Format (CFF — GitHub-native software citation): https://calibrationledger.com/CITATION.cff

## Operator

- **Paulo de Vries** (Netherlands), operating through editnative.com
- Contact: contact@editnative.com
- Related projects: holdlens.com (structured SEC filings intelligence), holdlens.com/forecasts/ (operator's own calibration track record — the prerequisite for rating others)

## Primary pages

- /about/ — operator identity, project status, design-partner contact
- /methodology/ — how sources will be scored (Brier + Murphy decomposition + calibration curves + append-only); canonical content asset of the prerequisite phase
- /contact/ — email routing with subject-line tags
- /privacy/, /terms/, /disclaimer/ — legal, compliance

## Key concepts (DefinedTerm glossary, machine-readable at /methodology/)

- **Brier score** — mean squared error between forecasted probability and realised outcome; proper scoring rule; lower is better
- **Calibration curve** — plot of forecasted probability vs observed frequency, bucketed by confidence bin
- **Murphy decomposition** — partition of Brier score into Reliability, Resolution, and Uncertainty components
- **Append-only time-stamping** — immutable pre-outcome logging that prevents hindsight bias
- **Probabilistic forecast** — claim expressed as a probability, not a binary assertion

## Source types that will be scored

- AI models — hallucination rate, factuality, confidence calibration
- Human forecasters — Brier scores across domains and time windows (Metaculus, Good Judgment Project, Manifold Markets)
- Analyst firms — price-target + rating accuracy vs realised outcomes
- Scientific papers — replication status, effect-size shrinkage
- Review platforms — outcome-alignment of aggregated reviews
- Prediction markets — market-implied probability vs outcomes

## What Calibration Ledger is not

- Not investment advice
- Not medical, legal, or tax advice
- Not a scorer of one-off predictions in isolation (requires ≥30 predictions at same confidence level for meaningful calibration per Tetlock 2015)
- Not a scorer of non-predictive "truthfulness" claims (factuality benchmarks are separate)

## License

- Content: CC-BY-4.0 for methodology drafts and scores
- Brand name "Calibration Ledger": operator's
- Contact for licensing / data partnerships: contact@editnative.com with subject "Calibration Ledger design partner"

## Freshness

- Site version: 1.1
- llms.txt last updated: 2026-04-24
- Methodology last updated: 2026-04-24