Break is available: it finds where your agent fails before your users do. Read the launch
Prove is available: it tells you whether a change to your agent changed how often it fails. Read the launch
Poisson Labs

News · 2026-11-16

Poisson Break is out

Finds the tool failures that break an agent, shrinks each to the smallest change that still breaks it, and confirms it against an untouched control.

Poisson Break 0.1.0, 48 seconds, no sound.
The video’s text
  1. 0:00$ pip install poisson-break · poisson labs · break 0.1.0 · Your evals run with every tool working. · Break finds which tool failures break your agent.
  2. 0:06100% pass rate on your benchmark. · In production, the refund reply gets lost and the agent pays twice.
  3. 0:10Poisson Break · Break searches for the tool failures you didn't think to test.
  4. 0:14target: your agent, called as a function · changes: 10 tool faults, injected into live tool calls · control: the same cases with nothing changed · confirm: fresh seeds, kept only if it fails more than the control · shrink: cuts a failure down to the smallest change that still fails · grading: a plain-code check by default, not an LLM judge
  5. 0:19Ten ways a backend breaks in production. · timeout: the tool raises a timeout at once · rate_limited: an error saying to wait a stated number of seconds · error: the tool raises an error · slow: the real result, after a stated delay · empty: an empty result of the real result's own kind · cut_off: the real result cut to a stated length · malformed: the real result with its structure broken · stale: the result of the previous call to the same tool · lost_reply: an error after the tool has already acted, so a retry repeats the action · planted_instruction: the real result with one added sentence telling the agent to repeat a marker word · Every change has a name, and every finding replays from its seed. · tool:<tool>#<call>=<fault>
  6. 0:24One lost reply. 18 of 18 runs refunded twice. · refund_desk:refund_desk (example: stand-in) · 6 cases × 3 repeats · control · nothing changed: 0 of 18 failed (0.000) · under finding: 18 of 18 failed (1.000) · culprit: tool:issue_refund#1=lost_reply · failure rate: 0.000 → 1.000 (+1.000 vs control) · kind: duplicate_write · verdict: confirmed · fresh seeds · every case in both arms · exact replay: poisson-break replay 1 --hunt refund-desk-1
  7. 0:32failed finding 1 on case damaged-4821, seed 32408516: duplicate_write · tool call change real tool ran · 1 lookup_order 1 none yes · 2 issue_refund 1 tool:issue_refund#1=lost_reply yes · 3 issue_refund 2 none yes · in this example issue_refund is a stand-in, called in the tool's place: no payments service is called
  8. 0:39a finding is a TOML file · re-runs in CI with poisson-break recheck --yes --check
  9. 0:42If it's going to break, break it here.
  10. 0:45poisson labs · break 0.1.0 · pip install poisson-break · poissonlabs.ai/break

An agent can pass every eval you have and still fall over the first time one of its tools times out. An eval mostly runs the agent with its tools working. In production a tool returns an error, answers late, hits a rate limit, or sends back half a result, and the question is what the agent does next.

Break tests it. It runs an agent you already have, many times, and makes one of its tool calls fail in one of ten ways: an error, a timeout, a slow answer, a rate limit, an empty, cut-off, malformed or stale result, a reply lost after the tool already acted, or an instruction planted in the result. Each change has a name you can read and type, of the form tool:<tool>#<call>=<fault>. The change replays exactly from the condition and the run’s seed. Whether the agent’s whole run replays depends on the agent and its model being deterministic.

A search like that turns up conditions that only look bad. One that is tried on a few runs, and kept because those runs failed, has numbers that run high. So nothing a hunt finds is a result. Break first cuts each candidate down to the smallest change that still breaks the agent. Then it runs every candidate again on fresh seeds, on every case, beside a control with nothing changed, and keeps only the ones that fail more than the control. The test is the exact test from Trace, the tool that checks whether a fix held, corrected across the candidates. What it keeps is a finding, with its failure rate beside the control’s and the difference with its interval. A candidate the runs could not settle is reported as too small to tell, with the number of repeats that would decide.

To start:

pip install poisson-break
poisson-break init --example desk
poisson-break hunt --yes
poisson-break confirm --yes

init --example desk writes a small help desk into the folder, with its cases and break.toml; the example needs no key and no network once it is installed. On your own agent, init asks where your agent and its test cases are and, for each tool it finds, whether calling it changes anything, and writes break.toml. Run Break against a test environment: it calls your tools for real, and it refuses a tool that changes the outside world until you say how that tool is made safe to repeat.

A finding is a plain file. After you change the agent, poisson-break recheck runs each finding again on the changed agent and prints Trace’s answer: fails less, fails more, no significant difference or too small to tell. The same command runs in CI as a standing check.

Break’s own checks use no model. A failure is something the agent did that code can see: it crashed, hung, ran out of turns, called a forbidden tool, followed a planted instruction, repeated a write after a lost reply, or missed a case’s own expectation. A check of your own may call a model, and a finding that rests on it says so in its judged by line. Break computes no statistic of its own; the test is Trace’s, and the correction across the candidates tested together comes from Prove, the tool that tells you whether a difference is real. It opens no connection of its own and sends nothing anywhere.

0.1.0 is an alpha. It runs an agent it can call as a Python function that wraps its own tools. It has no adapter for an agent framework yet, and its search does not learn. A hunt that finds nothing says what it tried and what it did not look at. It never says the agent is safe.

Break needs Python 3.11 or later, on macOS or Linux. It is open source, under the MIT licence.

MATH ASINFRASTRUCTURE.

Talk to us