§ 04 · take it with you
The doing-guide.
The operating manual: write a property, pick the instrument by question, read a verdict without lying to yourself. One tools/call away for agents.
SKILLS.md — doing formal methods with the lab
The one idea
Testing with formal methods = write the obligation once, let machines generate and judge the cases.
obligation (LTL) → cases (runs) → verdict (holds / violated / incomplete)Everything below is a recipe for one of those three steps.
Step 1 — Write the property
LTL grammar (as this lab parses it):
p ::= atom | true | false | !p | p && p | p || p | p -> p | p <-> p
| X p | G p | F p | p U p | p W p | p R p (atoms: [A-Za-z_][A-Za-z0-9_]*)Precedence loosest→tightest: <->, ->, ||, &&, then U W R (left-assoc), then ! X G F.
Property patterns and when to reach for them:
| You want to say | Write | Kind |
|---|---|---|
| nothing bad ever happens | G !bad | safety |
| something good eventually happens | F good / G F good | liveness |
| every request is answered | G (request -> F response) | response |
| grant only when ready | G (grant -> ready) | invariant |
| after grant, ready next step | G (grant -> X ready) | step |
p holds until q takes over | p U q (must happen) / p W q (may never) | precedence |
| alternate forever | G (a -> X (!a U b)) | rhythm |
Rules of thumb:
- One obligation per formula. If you wrote
&&between two independent thoughts, you will get two tools' worth of value by checking them separately. - Say when, not how.
F response— notretry then response. The model owns "how"; the property owns "when". - If you find yourself writing
G (!a && !b && ...), ask whetherG !(a || b || ...)or an auxiliary atom reads better. Future-you reads these.
Step 2 — Pick the instrument
The four tools answer four different questions. Pick by question, not by habit:
| Question | Tool / instrument | Cost |
|---|---|---|
| Does this exact run satisfy the property? | check_ltl (lab A) | cheap, finite semantics (LTLf) |
| Does every possible run of the model satisfy it? | model_check (lab B) | exhaustive over finite models |
| Can random runs break it? | falsify_property (lab C) | cheap, shrinks what it finds |
| Which states/transitions did my tests never touch? | analyze_coverage (lab D) | cheap, negative metric |
Decision recipe:
- You have a log/test trace and a property →
check_ltl. Truth table tells you which step broke it. - You have a model (states, transitions, labels) →
model_check. Counterexample = a ready-made failing test. - Model too big to check? Bound the depth (
model_checkwithdepth) — bounded model checking: violations are real, "incomplete" is honest. - No model, just a spec you distrust →
falsify_propertyover the atoms. A surviving campaign is evidence, not proof. - Wrote tests against a model →
analyze_coveragewith the traces. It proves gaps, never correctness. - Don't know the vocabulary →
explain_concept(no args: index; withconcept: chapter). - Need the whole manual →
read_skills_guide(returns this very file, verbatim). - Coming from C / Python / Rust → read the matching chapter under Approaches (
/approaches): native code, the artifact, and which tool works it.
Step 3 — Read the verdict honestly
Every tool labels exactly what it proved. Quote it verbatim; never upgrade:
- `holds` — only from exhaustive enumeration (depth = every state, no truncation). Means: this model cannot violate it. As strong as the model is faithful.
- `violated` — sound even when truncated: the counterexample lasso is a genuine failing run. Turn it into a regression test.
- `incomplete` — depth or path budget stopped the search. It means unknown, not passing.
- `not falsified` — random traces stayed lucky. Evidence, not proof.
A failing fuzzer campaign plus a holds model check on the same formula is the dream result: the machine found nothing you can't explain, and the exhaustive search found nothing at all.
Step 4 — Build the model (when you need one)
Kripke structure, as JSON:
{
"atoms": ["ready", "coin", "dispense"],
"initial": ["idle"],
"states": [
{ "id": "idle", "atoms": ["ready"], "next": ["paid"] },
{ "id": "paid", "atoms": ["coin"], "next": ["vend"] },
{ "id": "vend", "atoms": ["dispense"], "next": ["idle"] }
]
}atoms= the vocabulary; label each state with the ones true there.next= may-transitions: nondeterminism is a feature — checking the structure checks every resolution (that's the adversary).- A state with no
nextstutters forever — that's howF xcatches hangs. - Strip everything irrelevant: no variables, no timers, only what the property mentions. If the model is big, it's wrong.
Calling over MCP
POST /api/mcp — stateless JSON-RPC 2.0 (Streamable HTTP). No session handshake:
curl -s https://formalmethods.dev/api/mcp \
-H 'content-type: application/json' \
-d '{"jsonrpc":"2.0","id":1,"method":"tools/call",
"params":{"name":"check_ltl",
"arguments":{"formula":"G (request -> F response)",
"trace":[["request"],["timeout"]]}}}'initializeandtools/listare answered but nothing is remembered.- Bad args → JSON-RPC error
-32602; a tool that ran and failed →result.isError: true. - Read
result.structuredContent(agents) orresult.content[0].text(humans). - Seed everything:
falsify_propertyreturns the seed it used — record it in the bug report.
Recipes
Spec-first feature: write property → model_check against the model → take counterexamples as tests → analyze_coverage → ship the model + property + generated traces in the repo.
Triage a flaky bug: write the property you believe → falsify_property with a fixed seed → shrink output is your minimal reproduction → check_ltl on the shrunken trace shows exactly which step sins.
Check a log/CI run: same property, check_ltl over the trace of events — runtime verification without a monitor.
Hunt liveness bugs: G F something + a model with a stuttering dead end. If the checker says violated with a lasso, the loop in the counterexample is the hang.
Honesty checklist (before you report a result)
- Named the semantics: LTLf (finite) or infinite (lasso)?
- Quoted the tool's verdict verbatim — no upgrades?
- Seeded and recorded every random campaign?
- Turned every counterexample into a kept test?
- Labelled bounded/unfalsified results as evidence, not proof?
If you caught yourself saying "it passes" about anincompleteornot falsifiedresult — stop, rerun, say what actually happened. The lab is built so you never have to lie about a verdict; use that.