Skip to content
Poisson Labs Docs
GitHub poissonlabs.ai

Examples

Find where an agent fails with Break

Break ships five small agents to try it on. Each has a scripted model in place of a real one: a function that decides what the agent does next from the request and from what its tools returned. So each runs with no key and no network, and does the same thing every time. poisson-break init --example NAME writes one into the folder you are in. init will not write over a file that is already there, so each example below is made in a new folder of its own.

Before you start: Install and Your machine.

This is a help desk for an online shop. It looks an order up, and for a refund request it also reads the refund policy. When the look-up fails it gives up; when the policy cannot be read it answers without it.

Terminal window
mkdir desk
cd desk
poisson-break init --example desk

hunt prints its plan, runs the desk with no change (the control), then tries one change at a time and keeps the ones under which the desk failed.

Terminal window
poisson-break hunt --yes
Outputexcerpt, 11 of 20 lines · exit 0
⋯ 9 lines
found 3 candidates in 42 runs
control 12 runs · 0 failed (0.000)
search 30 runs · 10 conditions tried on 3 cases each
condition failed kind
1 tool:lookup_order#1=timeout 3 of 3 failed failed
2 tool:lookup_order#1=rate_limited 3 of 3 failed failed
3 tool:lookup_order#1=error 3 of 3 failed failed
not looked at: faults turned off: slow, cut_off, malformed, stale, lost_reply, planted_instruction; 0 single changes and 33 pairs of them untried
these are candidates, not findings: a few runs each, chosen for having failed
wrote break-runs/desk-1/hunt.json, proposals.jsonl, candidates.json and every run in store/
next: poisson-break confirm
All output20 lines
plan a hunt of desk on 6 cases, at most 120 runs
target desk_agent:desk (rollout)
tools lookup_order (read), check_policy (read)
may change tool behaviour (4 faults), inputs (2 operators)
strategy screen
stages control 12 runs, search and shrink up to 108
cap 120 runs, 2 hours
estimate at most 120 runs and 2 hours, with no prices to put in dollars, from the settings' limits
note: Break calls your tools for real: run it against a test environment, never against anything that matters
found 3 candidates in 42 runs
control 12 runs · 0 failed (0.000)
search 30 runs · 10 conditions tried on 3 cases each
condition failed kind
1 tool:lookup_order#1=timeout 3 of 3 failed failed
2 tool:lookup_order#1=rate_limited 3 of 3 failed failed
3 tool:lookup_order#1=error 3 of 3 failed failed
not looked at: faults turned off: slow, cut_off, malformed, stale, lost_reply, planted_instruction; 0 single changes and 33 pairs of them untried
these are candidates, not findings: a few runs each, chosen for having failed
wrote break-runs/desk-1/hunt.json, proposals.jsonl, candidates.json and every run in store/
next: poisson-break confirm

The desk never failed in its 12 control runs. Of the 10 conditions the hunt tried, three made it fail on every case they were tried on, and all three are on the first call to lookup_order. Those three are candidates, kept because they failed in a few runs each. confirm runs each of them again on every case, three times, beside a control arm that runs the same cases with no change, and asks Trace whether the difference holds.

Terminal window
poisson-break confirm --yes
Outputexcerpt, 8 of 15 lines · exit 0
⋯ 7 lines
confirmed 3 of 3 candidates fail more than the control (0.000)
condition verdict numbers
1 tool:lookup_order#1=timeout confirmed 1.000 +1.000
2 tool:lookup_order#1=rate_limited confirmed 1.000 +1.000
3 tool:lookup_order#1=error confirmed 1.000 +1.000
wrote break-runs/desk-1/findings/1.toml, 2.toml, 3.toml
reference 54 runs
next: poisson-break findings 1
All output15 lines
plan confirm 3 candidates against a control
design 6 cases × 3 repeats, once with no change and once under each candidate · 72 runs
reference up to 54 more runs, 18 for each candidate that is confirmed
cap 120 runs, 2 hours
Trace will say if this is too small to see a difference of 0.200, and how many repeats would be enough
estimate 126 runs, about 0 s; no prices, so no dollars, from the control stage's 0 s a run
note: Break calls your tools for real: run it against a test environment, never against anything that matters
confirmed 3 of 3 candidates fail more than the control (0.000)
condition verdict numbers
1 tool:lookup_order#1=timeout confirmed 1.000 +1.000
2 tool:lookup_order#1=rate_limited confirmed 1.000 +1.000
3 tool:lookup_order#1=error confirmed 1.000 +1.000
wrote break-runs/desk-1/findings/1.toml, 2.toml, 3.toml
reference 54 runs
next: poisson-break findings 1

All three hold, so the desk ends with three findings: a timeout, a rate limit and an error on the order look-up each end the request. findings 1 reads the first of them.

Terminal window
poisson-break findings 1
Output13 lines · exit 0
ok finding 1: tool:lookup_order#1=timeout
fails 1.000 under it, 0.000 under the control
difference +1.000
p 3.1e-08, adjusted 3.1e-08, q 0.050
kinds failed 18
signals gave_up 18, retry_storm 0, over_budget 0
judged by break
flagged none
left out not re-run on the old version 0, not re-run on the new version 0, no outcome on the old version 0, no outcome on the new version 0
cases 6 cases × 3 repeats, in confirm stage 1
arms desk-1-confirm-1 against desk-1-confirm-control
reference desk-1-reference-1
next: poisson-break replay 1 --hunt desk-1

replay 1 runs that finding once more and stores nothing. It lists every tool call the desk made, with the change that landed on it. Reading a finding goes through every line of both.

Terminal window
poisson-break replay 1
Outputexcerpt, 3 of 4 lines · exit 0
⋯ 1 line
failed finding 1 on case status-4821, seed 56661626: failed
tool call change real tool ran
1 lookup_order 1 tool:lookup_order#1=timeout no
All output4 lines
note: Break calls your tools for real: run it against a test environment, never against anything that matters
failed finding 1 on case status-4821, seed 56661626: failed
tool call change real tool ran
1 lookup_order 1 tool:lookup_order#1=timeout no

2. full-loop: hunt, confirm, put a fix in, recheck

Section titled “2. full-loop: hunt, confirm, put a fix in, recheck”

agent.py is an order desk that looks an order up once, and agent_fixed.py is the same desk with a failed look-up tried again. This example goes round the whole loop.

Terminal window
mkdir full-loop
cd full-loop
poisson-break init --example full-loop
Terminal window
poisson-break hunt --yes
Outputexcerpt, 11 of 20 lines · exit 0
⋯ 9 lines
found 3 candidates in 27 runs
control 12 runs · 0 failed (0.000)
search 15 runs · 5 conditions tried on 3 cases each
condition failed kind
1 tool:lookup_order#1=error 3 of 3 failed failed
2 tool:lookup_order#1=timeout 3 of 3 failed failed
3 tool:lookup_order#1=rate_limited 3 of 3 failed failed
not looked at: faults turned off: slow, cut_off, malformed, stale, lost_reply, planted_instruction; 0 single changes and 4 pairs of them untried
these are candidates, not findings: a few runs each, chosen for having failed
wrote break-runs/order-desk-1/hunt.json, proposals.jsonl, candidates.json and every run in store/
next: poisson-break confirm
All output20 lines
plan a hunt of order_desk on 6 cases, at most 120 runs
target agent:order_desk (rollout)
tools lookup_order (read)
may change tool behaviour (4 faults), inputs (1 operator)
strategy screen
stages control 12 runs, search and shrink up to 108
cap 120 runs, 2 hours
estimate at most 120 runs and 2 hours, with no prices to put in dollars, from the settings' limits
note: Break calls your tools for real: run it against a test environment, never against anything that matters
found 3 candidates in 27 runs
control 12 runs · 0 failed (0.000)
search 15 runs · 5 conditions tried on 3 cases each
condition failed kind
1 tool:lookup_order#1=error 3 of 3 failed failed
2 tool:lookup_order#1=timeout 3 of 3 failed failed
3 tool:lookup_order#1=rate_limited 3 of 3 failed failed
not looked at: faults turned off: slow, cut_off, malformed, stale, lost_reply, planted_instruction; 0 single changes and 4 pairs of them untried
these are candidates, not findings: a few runs each, chosen for having failed
wrote break-runs/order-desk-1/hunt.json, proposals.jsonl, candidates.json and every run in store/
next: poisson-break confirm
Terminal window
poisson-break confirm --yes
Outputexcerpt, 8 of 15 lines · exit 0
⋯ 7 lines
confirmed 3 of 3 candidates fail more than the control (0.000)
condition verdict numbers
1 tool:lookup_order#1=error confirmed 1.000 +1.000
2 tool:lookup_order#1=timeout confirmed 1.000 +1.000
3 tool:lookup_order#1=rate_limited confirmed 1.000 +1.000
wrote break-runs/order-desk-1/findings/1.toml, 2.toml, 3.toml
reference 54 runs
next: poisson-break findings 1
All output15 lines
plan confirm 3 candidates against a control
design 6 cases × 3 repeats, once with no change and once under each candidate · 72 runs
reference up to 54 more runs, 18 for each candidate that is confirmed
cap 120 runs, 2 hours
Trace will say if this is too small to see a difference of 0.200, and how many repeats would be enough
estimate 126 runs, about 0 s; no prices, so no dollars, from the control stage's 0 s a run
note: Break calls your tools for real: run it against a test environment, never against anything that matters
confirmed 3 of 3 candidates fail more than the control (0.000)
condition verdict numbers
1 tool:lookup_order#1=error confirmed 1.000 +1.000
2 tool:lookup_order#1=timeout confirmed 1.000 +1.000
3 tool:lookup_order#1=rate_limited confirmed 1.000 +1.000
wrote break-runs/order-desk-1/findings/1.toml, 2.toml, 3.toml
reference 54 runs
next: poisson-break findings 1

confirm ends with three findings, as the help desk’s does. The fix is the copy of agent_fixed.py over the agent.

Terminal window
cp agent_fixed.py agent.py

recheck runs each finding’s condition again on the agent as it is now. With --check the exit code carries the answer: the command exits 4 when a finding fails more than it did.

Terminal window
poisson-break recheck --yes --check
Outputexcerpt, 6 of 12 lines · exit 0
⋯ 6 lines
rechecked 3 of 3 findings now fail less
condition verdict numbers
1 tool:lookup_order#1=error fails less 1.000 before · 0.000 now
2 tool:lookup_order#1=timeout fails less 1.000 before · 0.000 now
3 tool:lookup_order#1=rate_limited fails less 1.000 before · 0.000 now
gap 0.0 days since the reference runs
All output12 lines
plan recheck 3 findings on the agent as it is now
findings 1 tool:lookup_order#1=error, 2 tool:lookup_order#1=timeout, 3 tool:lookup_order#1=rate_limited
runs 54: each finding over the cases and repeats of its reference run
cap 120 runs, 2 hours
estimate 54 runs, about 0 s; no prices, so no dollars, from the control stage's 0 s a run
note: Break calls your tools for real: run it against a test environment, never against anything that matters
rechecked 3 of 3 findings now fail less
condition verdict numbers
1 tool:lookup_order#1=error fails less 1.000 before · 0.000 now
2 tool:lookup_order#1=timeout fails less 1.000 before · 0.000 now
3 tool:lookup_order#1=rate_limited fails less 1.000 before · 0.000 now
gap 0.0 days since the reference runs

Each of the three findings failed in every run of its reference run and in none now, and the recheck exited 0. Checking a fix explains each part of it.

This is a refund desk. Its issue_refund tool asks a payments service to pay a customer back, which must not happen in a test. break.toml classes the tool write and names a stand-in, payments_fake:issue_refund, and Break calls the stand-in in the tool’s place in every run.

Terminal window
mkdir stand-in
cd stand-in
poisson-break init --example stand-in
Terminal window
poisson-break hunt --yes
Outputexcerpt, 9 of 18 lines · exit 0
⋯ 9 lines
found 1 candidate in 27 runs
control 12 runs · 0 failed (0.000)
search 15 runs · 5 conditions tried on 3 cases each
condition failed kind
1 tool:issue_refund#1=lost_reply 3 of 3 failed duplicate_write
not looked at: faults turned off: slow, rate_limited, empty, cut_off, malformed, stale, planted_instruction; 0 single changes and 6 pairs of them untried
these are candidates, not findings: a few runs each, chosen for having failed
wrote break-runs/refund-desk-1/hunt.json, proposals.jsonl, candidates.json and every run in store/
next: poisson-break confirm
All output18 lines
plan a hunt of refund_desk on 6 cases, at most 120 runs
target refund_desk:refund_desk (rollout)
tools lookup_order (read), issue_refund (write, stand-in)
may change tool behaviour (3 faults), inputs (0 operators)
strategy screen
stages control 12 runs, search and shrink up to 108
cap 120 runs, 2 hours
estimate at most 120 runs and 2 hours, with no prices to put in dollars, from the settings' limits
note: Break calls your tools for real: run it against a test environment, never against anything that matters
found 1 candidate in 27 runs
control 12 runs · 0 failed (0.000)
search 15 runs · 5 conditions tried on 3 cases each
condition failed kind
1 tool:issue_refund#1=lost_reply 3 of 3 failed duplicate_write
not looked at: faults turned off: slow, rate_limited, empty, cut_off, malformed, stale, planted_instruction; 0 single changes and 6 pairs of them untried
these are candidates, not findings: a few runs each, chosen for having failed
wrote break-runs/refund-desk-1/hunt.json, proposals.jsonl, candidates.json and every run in store/
next: poisson-break confirm
Terminal window
poisson-break confirm --yes
Outputexcerpt, 6 of 13 lines · exit 0
⋯ 7 lines
confirmed 1 of 1 candidate fails more than the control (0.000)
condition verdict numbers
1 tool:issue_refund#1=lost_reply confirmed 1.000 +1.000
wrote break-runs/refund-desk-1/findings/1.toml
reference 18 runs
next: poisson-break findings 1
All output13 lines
plan confirm 1 candidate against a control
design 6 cases × 3 repeats, once with no change and once under each candidate · 36 runs
reference up to 18 more runs, 18 for each candidate that is confirmed
cap 120 runs, 2 hours
Trace will say if this is too small to see a difference of 0.200, and how many repeats would be enough
estimate 54 runs, about 0 s; no prices, so no dollars, from the control stage's 0 s a run
note: Break calls your tools for real: run it against a test environment, never against anything that matters
confirmed 1 of 1 candidate fails more than the control (0.000)
condition verdict numbers
1 tool:issue_refund#1=lost_reply confirmed 1.000 +1.000
wrote break-runs/refund-desk-1/findings/1.toml
reference 18 runs
next: poisson-break findings 1

There is one finding. lost_reply lets the refund go through and then reports an error. The desk answers an error by sending the refund again, so the customer is paid twice, and Break’s duplicate_write check sees the second call. The replay shows the two refund calls.

Terminal window
poisson-break replay 1
Outputexcerpt, 5 of 6 lines · exit 0
⋯ 1 line
failed finding 1 on case damaged-4821, seed 32408516: duplicate_write
tool call change real tool ran
1 lookup_order 1 none yes
2 issue_refund 1 tool:issue_refund#1=lost_reply yes
3 issue_refund 2 none yes
All output6 lines
note: Break calls your tools for real: run it against a test environment, never against anything that matters
failed finding 1 on case damaged-4821, seed 32408516: duplicate_write
tool call change real tool ran
1 lookup_order 1 none yes
2 issue_refund 1 tool:issue_refund#1=lost_reply yes
3 issue_refund 2 none yes

The last column reads yes for both issue_refund calls, although the write tool never runs. The column says whether a call was passed on to the tool or answered by the fault alone. Where a tool has a stand-in, Break passes the call to the stand-in and records the stand-in’s run as the tool’s. So yes on these two rows means the stand-in ran. Under lost_reply the stand-in runs first and the desk is then given an error in place of the receipt. The example’s own issue_refund, which would ask the payments service to pay, is not called in any run. The rollout way in says how a tool is classed.

4. own-check: a failure only your own rule can see

Section titled “4. own-check: a failure only your own rule can see”

This order desk always answers. When its look-up fails it sends a confident reply anyway and reports resolved, so none of Break’s own checks sees anything wrong. The folder’s checks.py holds the team’s own rule, named in break.toml as [fails] own: a reply that says where the order is, sent after every look-up failed, is made up.

Terminal window
mkdir own-check
cd own-check
poisson-break init --example own-check
Terminal window
poisson-break hunt --yes
Outputexcerpt, 11 of 20 lines · exit 0
⋯ 9 lines
found 3 candidates in 24 runs
control 12 runs · 0 failed (0.000)
search 12 runs · 4 conditions tried on 3 cases each
condition failed kind
1 tool:lookup_order#1=error 3 of 3 failed own:made_up
2 tool:lookup_order#1=timeout 3 of 3 failed own:made_up
3 tool:lookup_order#1=rate_limited 3 of 3 failed own:made_up
not looked at: faults turned off: slow, cut_off, malformed, stale, lost_reply, planted_instruction; 0 single changes and 0 pairs of them untried
these are candidates, not findings: a few runs each, chosen for having failed
wrote break-runs/order-desk-1/hunt.json, proposals.jsonl, candidates.json and every run in store/
next: poisson-break confirm
All output20 lines
plan a hunt of order_desk on 6 cases, at most 120 runs
target desk:order_desk (rollout)
tools lookup_order (read)
may change tool behaviour (4 faults), inputs (0 operators)
strategy screen
stages control 12 runs, search and shrink up to 108
cap 120 runs, 2 hours
estimate at most 120 runs and 2 hours, with no prices to put in dollars, from the settings' limits
note: Break calls your tools for real: run it against a test environment, never against anything that matters
found 3 candidates in 24 runs
control 12 runs · 0 failed (0.000)
search 12 runs · 4 conditions tried on 3 cases each
condition failed kind
1 tool:lookup_order#1=error 3 of 3 failed own:made_up
2 tool:lookup_order#1=timeout 3 of 3 failed own:made_up
3 tool:lookup_order#1=rate_limited 3 of 3 failed own:made_up
not looked at: faults turned off: slow, cut_off, malformed, stale, lost_reply, planted_instruction; 0 single changes and 0 pairs of them untried
these are candidates, not findings: a few runs each, chosen for having failed
wrote break-runs/order-desk-1/hunt.json, proposals.jsonl, candidates.json and every run in store/
next: poisson-break confirm
Terminal window
poisson-break confirm --yes
Outputexcerpt, 8 of 15 lines · exit 0
⋯ 7 lines
confirmed 3 of 3 candidates fail more than the control (0.000)
condition verdict numbers
1 tool:lookup_order#1=error confirmed 1.000 +1.000
2 tool:lookup_order#1=timeout confirmed 1.000 +1.000
3 tool:lookup_order#1=rate_limited confirmed 1.000 +1.000
wrote break-runs/order-desk-1/findings/1.toml, 2.toml, 3.toml
reference 54 runs
next: poisson-break findings 1
All output15 lines
plan confirm 3 candidates against a control
design 6 cases × 3 repeats, once with no change and once under each candidate · 72 runs
reference up to 54 more runs, 18 for each candidate that is confirmed
cap 120 runs, 2 hours
Trace will say if this is too small to see a difference of 0.200, and how many repeats would be enough
estimate 126 runs, about 0 s; no prices, so no dollars, from the control stage's 0 s a run
note: Break calls your tools for real: run it against a test environment, never against anything that matters
confirmed 3 of 3 candidates fail more than the control (0.000)
condition verdict numbers
1 tool:lookup_order#1=error confirmed 1.000 +1.000
2 tool:lookup_order#1=timeout confirmed 1.000 +1.000
3 tool:lookup_order#1=rate_limited confirmed 1.000 +1.000
wrote break-runs/order-desk-1/findings/1.toml, 2.toml, 3.toml
reference 54 runs
next: poisson-break findings 1

The kind own:made_up is the word the check returned, and the three findings rest on it. findings 1 says so in its judged by line.

Terminal window
poisson-break findings 1
Output13 lines · exit 0
ok finding 1: tool:lookup_order#1=error
fails 1.000 under it, 0.000 under the control
difference +1.000
p 3.1e-08, adjusted 3.1e-08, q 0.050
kinds own:made_up 18
signals gave_up 18, retry_storm 0, over_budget 0
judged by your check
flagged none
left out not re-run on the old version 0, not re-run on the new version 0, no outcome on the old version 0, no outcome on the new version 0
cases 6 cases × 3 repeats, in confirm stage 1
arms order-desk-1-confirm-1 against order-desk-1-confirm-control
reference order-desk-1-reference-1
next: poisson-break replay 1 --hunt order-desk-1

Adding a check of your own explains how a check is written.

This is an order desk, with a fault the team wrote in faults.py: error_page, under which a tool returns a page of HTML with no error raised. The agent’s module imports faults.py, which registers the fault, and break.toml names it in [vary] tools beside three of Break’s own.

Terminal window
mkdir own-fault
cd own-fault
poisson-break init --example own-fault
Terminal window
poisson-break hunt --yes
Outputexcerpt, 9 of 18 lines · exit 0
⋯ 9 lines
found 1 candidate in 24 runs
control 12 runs · 0 failed (0.000)
search 12 runs · 4 conditions tried on 3 cases each
condition failed kind
1 tool:lookup_order#1=error_page 3 of 3 failed crashed
not looked at: faults turned off: slow, empty, cut_off, malformed, stale, lost_reply, planted_instruction; 0 single changes and 0 pairs of them untried
these are candidates, not findings: a few runs each, chosen for having failed
wrote break-runs/order-desk-1/hunt.json, proposals.jsonl, candidates.json and every run in store/
next: poisson-break confirm
All output18 lines
plan a hunt of order_desk on 6 cases, at most 120 runs
target shop_agent:order_desk (rollout)
tools lookup_order (read)
may change tool behaviour (4 faults), inputs (0 operators)
strategy screen
stages control 12 runs, search and shrink up to 108
cap 120 runs, 2 hours
estimate at most 120 runs and 2 hours, with no prices to put in dollars, from the settings' limits
note: Break calls your tools for real: run it against a test environment, never against anything that matters
found 1 candidate in 24 runs
control 12 runs · 0 failed (0.000)
search 12 runs · 4 conditions tried on 3 cases each
condition failed kind
1 tool:lookup_order#1=error_page 3 of 3 failed crashed
not looked at: faults turned off: slow, empty, cut_off, malformed, stale, lost_reply, planted_instruction; 0 single changes and 0 pairs of them untried
these are candidates, not findings: a few runs each, chosen for having failed
wrote break-runs/order-desk-1/hunt.json, proposals.jsonl, candidates.json and every run in store/
next: poisson-break confirm
Terminal window
poisson-break confirm --yes
Outputexcerpt, 6 of 13 lines · exit 0
⋯ 7 lines
confirmed 1 of 1 candidate fails more than the control (0.000)
condition verdict numbers
1 tool:lookup_order#1=error_page confirmed 1.000 +1.000
wrote break-runs/order-desk-1/findings/1.toml
reference 18 runs
next: poisson-break findings 1
All output13 lines
plan confirm 1 candidate against a control
design 6 cases × 3 repeats, once with no change and once under each candidate · 36 runs
reference up to 18 more runs, 18 for each candidate that is confirmed
cap 120 runs, 2 hours
Trace will say if this is too small to see a difference of 0.200, and how many repeats would be enough
estimate 54 runs, about 0 s; no prices, so no dollars, from the control stage's 0 s a run
note: Break calls your tools for real: run it against a test environment, never against anything that matters
confirmed 1 of 1 candidate fails more than the control (0.000)
condition verdict numbers
1 tool:lookup_order#1=error_page confirmed 1.000 +1.000
wrote break-runs/order-desk-1/findings/1.toml
reference 18 runs
next: poisson-break findings 1

There is one finding, and it names the team’s fault. The hunt puts each fault on the first call to lookup_order only. Break’s error, timeout and rate_limited each reach the desk as an exception raised from that call. The desk catches an exception from a look-up and calls the tool again, up to three times, and the second call has no fault on it and returns the order record. So those three do it no harm. error_page raises nothing: the first call returns a page of HTML as its result, and the desk has nothing to catch. It takes the page for an order record and splits it into the record’s two fields. That line fails on the page, it sits outside the desk’s try, and the run ends crashed. The replay shows the call.

Terminal window
poisson-break replay 1
Outputexcerpt, 3 of 4 lines · exit 0
⋯ 1 line
failed finding 1 on case status-4821, seed 56661626: crashed
tool call change real tool ran
1 lookup_order 1 tool:lookup_order#1=error_page no
All output4 lines
note: Break calls your tools for real: run it against a test environment, never against anything that matters
failed finding 1 on case status-4821, seed 56661626: crashed
tool call change real tool ran
1 lookup_order 1 tool:lookup_order#1=error_page no

Adding a fault explains how a fault is written.

Every output above is what the command printed, stored in the site repository under src/data/examples/find-where-an-agent-fails-with-break/ as poisson-break 0.1.0 printed it from commit 37da225, each example in a new folder of its own. The run left out 5 next: lines, each the poisson-trace fix-check command for one finding’s full report, because each holds a full path on the machine the run was made on. Those lines are not in the stored output, so the blocks above do not show them.