Poisson Labs · Platform · Poisson Cloud
Continuous verification for systems that can't afford regressions.
Poisson Cloud is the hosted verification and failure-discovery platform we are building on our open-source measurement engine. It will run paired statistical checks on your AI agents, robot policies, and workflows on every pull request or checkpoint promotion—reporting exactly what improved, what regressed, and what is still underpowered, and blocking merges when a condition gets significantly worse.
Poisson Cloud is in development. Today, single-tenant verification is in private preview: a self-hosted deployment that runs the verification pipeline inside a design partner's own environment—on your compute, keys, and VPC—against real workloads and CI/CD workflows.
5 conditions got worse.
- 8 safer
- 5 worse
- 1 no significant change
checkpoint 2250 against checkpoint 5999, 14 conditions, 100 trials each, same seeds on both sides. Too small to see: anything under 8pp where nothing moved.
report.md microduck-walking@v1
Empirical run from our bipedal locomotion study (14 conditions × 100 paired trials), shown as the pull-request gate reports it. The gate is in private preview, single tenant; Poisson Cloud is in development.
What it runsTasks & Prove, available · Trace, private wheel · Break, in development
A single leaderboard average hides localized failures. The pipeline chains our open-source primitives into continuous verification: Break searches for unknown failure regimes, Trace isolates the exact step where trajectories die, and Prove runs paired hypothesis tests across every condition to govern releases.
-
1Break
Search for the conditions under which your system fails.
Testing only the edge cases you already thought of leaves your production boundary unmapped. Break systematically perturbs what your system encounters—user prompts, tool latencies, malformed schemas, injected instructions, or physical simulation parameters—and retains only the regimes that fail significantly more often than an untouched control. It distills each into a minimal, replayable seed so Prove tests your real failure boundary.
In developmentThe Break page →
-
2Trace
Find the exact step at which runs die.
An episode pass rate never tells you where a trajectory went off the rails. Trace aligns every stored rollout on a unified step clock, computing step-by-step survival curves and confidence intervals across tool calls, model generations, and state transitions to pinpoint the exact step where runs diverge.
Private wheelAbout Trace →
Both runs, identical
- 0 user message
- 1 agent message
- 2 user message
- 3 agent
get_reservation_details
Call 4: the runs part. The agent is given the same 3,659 tokens in both runs. It writes 96 tokens in one and 129 in the other.
Run R 20260915T210140Z-4847
- 4 agent message
- 5 user message
- 6 agent
transfer_to_human_agents - 7 agent message
- 8 user message
Passed solved · COMMUNICATE 1 · DB 1
Run R′ 20260915T211148Z-965d
- 4 agent message
- 5 user message
- 6 agent
cancel_reservation - 7 agent message
- 8 user message
- 9 agent
transfer_to_human_agents - 10 agent message
- 11 user message
Failed unsolved · COMMUNICATE 1 · DB 0
Paired rollout from the R.1 study: one τ²-bench task, identical seed and temperature 0, run twice. Both runs reach call 4 on the exact same 3,659 input tokens, then diverge there: Run R′ goes on to invoke cancel_reservationand fails. Across every stored run, Trace aggregates step-level outcomes to compute survival intervals. -
3Prove
Prove whether a change moved the failure rate.
A patch or retrain that lifts overall accuracy frequently breaks specific regimes. Prove evaluates your system before and after a change across every condition on identical seeds, running paired tests per condition with family-wise error control across the entire sweep—separating true improvements and regressions from run-to-run noise.
AvailableAbout Prove →
5 worse 1 no significant change 8 safer coloured only at q<0.05, corrected across every condition
LATERAL PUSH, m/s → ↑ FRICTION μ 100 TRIALS PER CELL PER VERSIONA real run: a walking policy at two checkpoints, 14 conditions, on the same seeds in both, tested by McNemar's test per condition and corrected across all 14. The later one is better in 8 (blue) and worse in 5 (black, hatched). report.md
What happens to what it findsThe piece that closes the loop.
A failure the pipeline discovers doesn't end as a line in a static report. It becomes a permanent regression gate on every pull request, verifiable preference data for post-training, and prior evidence for the next adversarial search.
-
1Into your CIAvailable
Every failure becomes a standing gate.
Each condition that broke your system is committed as a deterministic fixture: the parameter regime, the seed, and the baseline failure rate. It re-runs automatically on every pull request so fixed failure modes stay fixed.
-
2Into trainingIn development
Every fix becomes trainable data.
The same condition on the same seed—failing before a change and passing after—forms a verified preference pair. For robotics and RL policies, training distributions shift toward isolated failure boundaries, as we did by hand in our Go1 retrain.
-
3Into MonteIn research
Every hunt sharpens the surrogate.
Labeled failure trajectories and their exact parameter regimes will train Monte: our surrogate failure model that will predict where a system is most likely to break next, focusing Break's compute budget on high-risk regimes first. It will train only on workspaces that explicitly opt in.
Nothing counts as fixed until it is re-run. A retrained agent goes back through the same conditions on fresh seeds, and the result is a receipt like any other. The Go1 retrain is this loop, run once: the map showed two conditions training had missed, and the policy was retrained on exactly those.
Open-source core. Single-tenant preview. Hosted platform in development.
Our core measurement tools are open source and always will be: you can run them locally, inspect every statistical estimator, and never send a single token or trajectory to us.
Poisson Cloud, in development, is the hosted orchestration and intelligence layer around them: automated parallel batching across thousands of rollouts, CI pull-request gating, permanent statistical receipts, and, in research, Monte-guided failure search.
Single-tenant verification is in private preview with a select group of design partners: the same verification pipeline, self-hosted directly inside their cloud environments and tested on real workloads and CI/CD workflows, while we build the hosted platform.
Single-tenant verificationPrivate preview. Deployed inside your environment.
The single-tenant deployment runs inside your VPC or cluster against your own repositories, model keys, and simulators. Its lightweight instrumentation layer captures step-level spans directly from OpenAI Agents SDK, LangGraph, and CrewAI, and stores every rollout from your robotics simulators—packaging every trajectory for Trace survival analysis and Prove hypothesis testing.
Instead of relying on brittle prompt wrappers, Poisson captures the exact execution graph per episode—recording agent state transitions, model invocations, and tool calls at the span level without altering your runtime behavior. Full framework version pins and instrumentors are documented on the integrations page.
Try it on a robotLive. One rollout per world.
The same method on a system you can watch fail. This is the quadruped from the Go1 study: 400 conditions of floor friction and sideways push, run before and after a retrain. Pick a condition and push the robot. The map on the left is the study’s result, and it marks the cell you landed in.
0 worse 340 no significant change 60 safer coloured only at q<0.05, corrected across every condition
Push it yourself.
Pick a floor friction and a sideways push, and watch one rollout of the policy in that world. Worlds someone has already probed replay instantly. A new one runs on a GPU, about 3 seconds warm and 25 cold.
A 3D viewer, about 53 MB. It loads only when you ask.
The two maps use the same seeds, and the comparison is corrected across all 400 conditions. The replay is one rollout at one fixed seed; the counts beside it are over all 32 trials in that cell. A policy in a simulator, not a robot on a floor.
What it does while it runs
- It runs the compute.
- A real test is thousands of runs of your system. It sends them out in batches, collects them as they finish, and never rewrites a result once it is stored.
Out — 400 conditions × 32 trials is 12,800 rollouts, chunked into 200 batches of 64. Every shade in the upper strip is one batch, tinted by its place in that plan. Back — the same batches in the same tints, in the order they finish. The gradient is gone, and that scramble is the whole point: no batch waits for another and none of them share state, so the store fills in an order nothing chose. Each one is appended as a part and never rewritten. Only at the end, over the whole stack, does anything read it — Aalen–Johansen per cell, then Fisher exact and Benjamini–Hochberg across all 400, giving 60 safer · 0 worse.
Shown above: 12,800 rollouts (400 conditions × 32 trials) dispatched in 200 stateless batches of 64. Batches execute asynchronously across worker nodes and append to an immutable store as they finish, before Prove computes Aalen–Johansen survival and Benjamini–Hochberg corrected p-values across the complete stack. The arrival order drawn is illustrative.
- Automated boundary search with Break and Monte.
- Today, you define the condition grid or task distribution to sweep, and Prove verifies every cell. As Break and Monte roll out into the platform, Poisson Cloud automates the discovery of those conditions.
- Monte is the lab’s engine for that search.
- Randomly sampling a high-dimensional parameter space wastes compute on safe regimes where nothing fails. Monte is our surrogate failure-prediction engine for Poisson Cloud, in research. Trained on historical failure trajectories and parameter boundaries across prior checkpoints, Monte will predict high-probability failure regions and direct Break to sample those regimes first. Every candidate failure Monte surfaces will still be verified the exact same way: re-run against a paired control and tested via Trace and Prove.
- It guards every pull request.
- The same test runs on each change. If any condition comes back significantly worse, the check fails and names it. If the test was too small to tell, it says that instead of passing.
- name: Check the agent
run: |
poisson-prove sweep agent.toml \
--target my_agent:rollout \
--pair-with <main-run> --yes
poisson-prove check <main-run> <this-run> \
--event violated --fail-on worse- It keeps the record.
- Every result is kept as a receipt: what was run, on which seeds, what moved, and what the run could not have detected. Here is a real one.