formalmethods.dev

§ 1.6

Property-based testing & falsification

Let the machine try to break your spec — then let it explain itself.

see also · Runtime verification · Specification as the test oracle · Coverage as a negative metric

Random executions, one property

Property-based testing replaces hand-written cases with a generator plus a predicate: invent inputs at random, keep the property that must hold for all of them. A few thousand runs covers cases nobody would type — the empty trace, everything at once, the pathological order.

Here the artefacts are traces of propositions, judged by an LTL formula. The fuzzer invents runs; the formula votes.

Shrinking: counterexamples that explain

A raw failure is noise — 12 random steps when 2 would do. Shrinking keeps deleting a step or an atom while the trace still violates; what survives is a minimal, readable reproduction.

A passing campaign proves nothing — you sampled and stayed lucky. But a shrunk failure teaches you something no example-based test would.

  • Seeds make runs reproducible — same seed, same failures.
  • Coverage-guided fuzzers bend generation toward unexplored paths.
  • Surviving 10⁵ random traces is evidence, not proof.

try it · falsify it with seeds — lab C →