# FableTest — A Constitutional Test Suite

**Milestone #3 of FablePool: Adversarial Scenarios as Regression Tests.**

The premise of this repository: a constitution is load-bearing infrastructure, and
infrastructure that is never tested fails exactly when it is needed. FableTest treats
constitutional text the way good engineering teams treat code — every proposed amendment
is run against a corpus of adversarial scenarios before it can merge, and every exploit
ever discovered becomes a permanent regression test.

Optimism in the defaults. Paranoia in the tests.

## What is in this repository

| Path | Contents |
|---|---|
| `constitution/parameters.yaml` | The live governance configuration the kernel parameterizes (thresholds, quorum, notice periods, emergency limits). Amendments change this file or the kernel text it binds to. |
| `fabletest/` | The scenario DSL loader (`model.py`), the adjudication engine (`engine.py`), the empathy metric (`metrics.py`), the attack taxonomy (`taxonomy.py`), and the CI harness (`harness.py`, `cli.py`). |
| `scenarios/` | The adversarial corpus: 200+ fully specified attacks, one YAML file per taxonomy family, each scenario documented with its historical or game-theoretic precedent. |
| `docs/dsl-spec.md` | Normative specification of the scenario DSL: actors, resources, moves, verdicts, rule-id namespace. |
| `docs/attack-taxonomy.md` | The seven attack families, their signatures, and what each family stresses. |
| `docs/empathy-metric.md` | How every scenario is scored first on how the worst-off participant fares. |
| `docs/runbook.md` | The process for turning a newly discovered exploit into a permanent regression test. |
| `tests/` | Meta-tests: the test suite for the test suite (DSL parsing, engine semantics, harness behavior, corpus integrity). |
| `.github/workflows/constitutional-tests.yml` | The CI gate. A PR that changes constitutional text and fails any scenario is blocked. |

## Quickstart

```bash
# Python >= 3.10
pip install -e ".[dev]"

# Run the entire corpus against the current constitution
fabletest run

# Run a single family or a single scenario
fabletest run --family treasury_drain
fabletest run --id FC-004

# Validate scenario files against the DSL schema without executing them
fabletest validate scenarios/

# Evaluate a *proposed* parameter change before opening a PR
fabletest run --parameters my-proposed-parameters.yaml

# Meta-tests (the suite's own unit tests)
pytest
```

The first build on a clean machine should generate the dependency lockfile with the
tooling of your choice (`pip-compile`, `uv lock`, etc.); no lockfile is committed.

## How the CI gate works

1. A pull request modifies `constitution/parameters.yaml` (a userland change) or kernel
   text (a major-version change requiring supermajority ratification — see Milestone #2).
2. CI runs `fabletest run --parameters <proposed>` over the entire scenario corpus.
3. Each scenario replays a sequence of *legal moves* by adversarial actors against the
   proposed text. The engine adjudicates every move against the kernel invariants and the
   proposed parameters.
4. A scenario **passes** when the engine's verdict matches the scenario's `expected`
   block — including the empathy floor: the worst-off participant's welfare must not
   fall below the asserted minimum.
5. Any failure blocks the merge. The failure report names the scenario, the precedent it
   encodes, the first rule that should have blocked the attack, and the welfare of the
   worst-off participant under the proposed wording.

A failing scenario is not a bug in the test. It is a proof, in moves anyone can read,
that a faction can do the bad thing under your wording.

## The one scoring rule above all others

Every scenario is graded **first** on how the worst-off participant fares under stress
(`docs/empathy-metric.md`). A wording that technically blocks an attack but immiserates a
minority along the way still fails. The bet underneath the project is that people are
good at heart and the tooling is broken — so the suite measures the tooling by what it
does to the person with the least power in the room.

## Contributing a scenario

Found an exploit — in self-play, in review, or in the news? Follow `docs/runbook.md`:
reproduce it as a move sequence, minimize it, classify it into the taxonomy, assert the
expected verdict and empathy floor, document the precedent, and open a PR. Exploits
become tests; tests become permanent.

## Provenance

- Milestone #1 delivered the constitutional kernel v0.1 and design doc.
- Milestone #2 delivered the amendment pipeline (governance-as-code tooling).
- **Milestone #3 (this one)** delivers the scenario DSL, the harness, the taxonomy, the
  200+ scenario corpus, and the runbook.

License: Apache-2.0 for code, CC-BY-SA-4.0 for constitutional text and scenarios.