Tools · open source · In development
Break
Find where an agent fails, before a user does.
Break runs an agent you already have, many times, while it changes what the agent meets in production. It keeps only the changes that make the agent fail more often than an untouched control, by Trace’s test, cuts each down to the smallest change that still breaks it, and hands it over as a replayable condition a fix can be checked against.
In development. Ask about it
A hunt, in five stages
- 1 ControlEvery case, a few times, with nothing changed: the failure rate with no help, which tools get called, what one run costs.
- 2 SearchA strategy proposes changes and tries each on a few cases. Candidates only.
- 3 ShrinkSmaller versions of each candidate, down to the smallest change that still fails.
- 4 ConfirmEvery case again, unchanged and under each candidate, on fresh seeds. Only this stage decides what is a finding.
- 5 ReferenceEach finding once more on fresh seeds: the “before” a later fix is compared with.
Stages 2 and 3 are discovery: nothing they produce is a result. Stage 4 is fixed before its first run and uses seeds discovery never touched, which is why a finding can be believed.
What it is for
What breaks it before a user does?
A hunt before a release, and a short list of the conditions that fail, worst first.
What happens when my tools misbehave?
Every way a tool fails in production, tried against your agent, with each failure rate beside the control’s.
I fixed it. Did I?
Each finding re-run on the changed agent, with Trace’s answer: fails less, fails more, no significant difference, or too small to tell.
Keep what we found.
Findings are files. They re-run in CI as a standing check.
What it changes
- What users ask
- The request and how the conversation goes.
- How your tools behave
- Errors, timeouts, slow answers, rate limits, empty, cut-off or malformed results, a reply lost after the tool already acted, and an instruction planted in a tool result. MCP tools included.
- How model calls fail
- The agent’s own calls to its model, failing the way they do in production.
How it decides something broke
No model grades a run.
Every run ends ok, failed or not counted, and a failure is something the agent did that code can see: it crashed, hung, ran out of turns, called a forbidden tool, followed a planted instruction, repeated a write after a lost reply, reported a failing outcome, missed a case’s expectation, or failed your own check. Your own check may call a model; a finding that rests on it says so wherever it is printed.
Break computes no test of its own. The test against the control is Trace’s, and the correction across findings is Prove’s.
Who it is for, and what it is not
Engineers who build and ship tool-using agents on the OpenAI Agents SDK, LangGraph or CrewAI, and the people responsible for those agents staying up. You run the agent in a test environment and say what a bad outcome is. No statistics needed, and no account with anyone.
- Not a catalogue of jailbreak prompts or a content-safety scanner.
- Not a guardrail: it tests; it does not defend at run time.
- Not a certificate: a hunt that finds nothing says what it tried, never that the agent is safe.
- Not for production traffic: it calls your tools for real, in a test environment, and refuses a tool that changes the outside world until you say how to make it safe to repeat.
Where it sits
Tasks makes controlled tasks. Break finds where an agent fails. Trace says whether a change really moved the failure rate. Prove runs and stores everything and is the statistics underneath.
Break is open source and runs on your machine, sending nothing anywhere. Monte, the lab’s learned search, is part of Poisson Cloud: a model of where agents fail that points a hunt at the conditions most likely to break the next version. More on the platform page.