formalmethods.dev

§ 1.1

Specification as the test oracle

Before you can test a system, decide what counts as a failure.

see also · Linear temporal logic · Model checking · Property-based testing & falsification

The oracle problem

A test needs two things: an input, and a way to tell pass from fail. The oracle is the expensive half — expected behaviour lives in tickets and heads, where it rots.

A formal spec replaces all of that with one precise sentence: G (request -> F response) — always, a request is eventually answered. One line, an oracle for infinitely many tests.

From property to verdict

Once the property is machine-checkable, the loop inverts: generate executions, ask the spec whether each one passes. This lab does that three ways — exhaustively (model checking), randomly (property-based testing), and after the fact (runtime verification on logs).

  • G p — safety: p holds in every reachable state.
  • F p — liveness: p eventually happens.
  • p U q — precedence: p holds until q, which really occurs.
  • G (request -> F response) — the request/response contract.

try it · check a property on a run — lab A →