Poisson Labs · Tools
Five tools. One evidence trail.
Generate controlled conditions, hunt for failure boundaries, trace where trajectories die, prove whether a change held on paired seeds, and turn failures directly into post-training environments. Every tool outputs an exact statistical verdict—what improved, what regressed, and what is still underpowered below the minimum detectable effect (MDE).
They run locally or on your own cloud compute, work alongside the simulators and runners you already use, and send nothing home. Every check is graded by deterministic code and statistics—never by another model.
pip install poisson-tasks poisson-prove Trace is in private wheel · Break & Gym in development · View the studies and changelog.
-
Tasks
Available
Measure how controlled difficulty changes outcomes.
Generates seeded, algorithmically parameterized tasks and conditions across calibrated difficulty knobs—on built-in domains or your own tool schemas—and measures pass rates with 95% confidence intervals at every level.
Open source The Tasks page →
-
Trace
Private wheel
Find where runs fail, and whether a fix made them fail less.
Ingests stored trajectories from your existing runners step by step to map survival rates and pinpoint where rollouts die. Compares candidate fixes against a re-run control to report verified improvements and the exact minimum detectable effect.
Open source The Trace page →
-
Prove
Available
Measure whether a change held, condition by condition.
Evaluates your agent, policy, or workflow before and after a change across every regime on identical seeds. Applies multiple-testing correction across all conditions at once to separate real regressions from run-to-run variance.
Open source The Prove page →
-
Break
In development
Search for the exact conditions that cause failures.
Sweeps across prompts, tool faults, and physical simulation parameters to map your system's failure boundary. Keeps only regimes that fail significantly more often than an untouched control and exports each as a replayable seed.
Open source The Break page →
-
Gym
In development
Train a system on its own failures, and prove whether training helped.
An open-source post-training environment built on Tasks, Break, Trace, and Prove. Draws practice distributions directly from isolated failure regimes, grades rollouts with deterministic code verifiers the policy cannot see, and validates every checkpoint against baseline.
Open source The Gym page →
How they fit
Each tool runs standalone or as a continuous pipeline. Tasks and Break define the seeded conditions. Prove executes paired before-and-after rollouts across those conditions and stores the trajectories. Trace analyzes step-by-step survival on Prove's runs—or imports existing results directly from DeepEval and Ragas without re-running your system, with OpenTelemetry, LangSmith, and JUnit readers built as each team is onboarded.
Poisson Cloud, in development, will run this entire verification loop automatically in CI on every pull request or checkpoint promotion; the single-tenant version, self-hosted in a design partner's environment, is in private preview (the platform page). The statistical estimators behind every verdict are documented on the methods page, and CLI docs are mirrored directly from each repository.
What happens to what they find
A failure discovered by the tools doesn't stay in a static report. It becomes a permanent regression gate in CI, a verifiable preference pair for post-training, and prior evidence for the next adversarial search.
-
1Into your CIAvailable
Every failure becomes a regression test.
Each condition that broke your system is committed as a reproducible artifact: the exact parameter regime, seed, and baseline failure rate. Prove re-runs it on every pull request so a patched failure mode stays fixed.
-
2Into trainingIn development
Every fix becomes trainable data.
The same condition on the same seed—failing before a change and passing after—forms a clean preference pair with a deterministic code verifier. For robotics and RL policies, training distributions shift toward verified failure boundaries, as we did by hand in our Go1 retrain.
-
3Into MonteIn research
Every hunt sharpens the surrogate.
Labeled failure trajectories and their exact parameter regimes will train Monte: our surrogate failure model on Poisson Cloud that will predict where a system will break next, guiding Break to sample high-risk regimes first. It will learn only from workspaces that explicitly opt in.