Poisson Labs · Work with us
Work with us
Agents give different answers to the same input. We measure how often yours fails, with the bound on that number, on your agent and in your environment. Then we hand over the tools to keep measuring it.
Five shapes
-
We find where your agent fails and the step that causes it, on your own tasks and traces. You get the report, the runs behind every number, and the commands to rerun them.
From the Trace studyIn a synthetic test, one stage's failure rate was doubled and the choice sealed before any run. Trace called the pair different (p = 1.7e-9) and named review, the stage that had been altered. The study
-
We set the tools up on your stack with you: your tracer, your test runner, your framework, your compute. Your team runs them from the first day.
What it readsTrace reads what your stack already writes: DeepEval and Ragas today, and 10 more readers built as each team is onboarded. Every integration
-
03 A recurring check in CI
Every change gets the same comparison on the same seeds. A significant regression fails the check, and the result says what the run was too small to see.
From the Microduck studyA retrain of a walking policy, checkpoint 2250 against checkpoint 5999, cut the average fall rate from 50.5% to 20.8%. By McNemar's test on the same seeds, corrected across all 14 conditions, 8 got significantly safer and 5 got worse, so the check fails. The 1 that showed no significant change could not have shown a change under 8 points. The study
-
Failure rates with confidence bounds, before and after every change, on the same seeds. The report also states what the test could not have detected. It gives an auditor or an insurer numbers they can check and rerun.
Step by stepWhat one run can and cannot tell an underwriter, with the numbers and the files behind each step. How the evidence is built
-
05 The full product, self-hosted
The whole suite, single tenant, in your own account.
StatusThis is Poisson Cloud, in private preview with a small number of teams, run in your environment. Poisson Cloud
The report
What every report carries
- The verdict and the test
- The first lines say which conditions moved, which way, and the test and correction that made the call.
- What it couldn't have detected
- Every condition that showed no significant change carries its detection floor in its own row.
- The runs and the seed
- The report names both runs and the seed every episode was drawn from, so the sweep can be run again.
- The instruments
- The report comes from Tasks and Prove, open source, which your team keeps when the work ends.
# go1-g2-converged@20260813T155006Z 2026-09-03 a vs go1-g2-v1@20260818T230622Z-v1 2026-09-03 a: felllower fell rose in 0 and fell in 60 of 400 conditions; counted at the family-wise level: fell 0 rose and 34 fellfell: 60 lower, 0 higher(aalen_johansen + fisher_bh, BH q<0.05 over 400 tests, unpaired)before go1-g2-converged@20260813T155006Z 2026-09-03 a (go1-v0)after go1-g2-v1@20260818T230622Z-v1 2026-09-03 a (go1-v1)method: two separate samples (Fisher), named with --method… 47 lines: the omnibus test and the fault lines …## Cellspush_pct_bw=10.0: 20 tests, 2 rejected; cell MDE median 0.465 / max 0.465| cell | event | delta | q | MDE ||---|---|---|---|---|| mu=0.1, push_pct_bw=10.0 | fell | -0.344 | 0.003 | none detectable || mu=0.05, push_pct_bw=10.0 | fell | -0.750 | 2.9e-08 | none detectable |… 605 more lines: every condition, the intervals, what the verdict rests on, the runs and the seed …
Data and people
Your data
You control and own your data. Your keys, prompts and outputs stay in your environment. Run records and verdicts leave it only if you opt in, in a format we publish.
Get involved
Engineers and statisticians who want to work on this: try the tools, open an issue, or ask to become a collaborator. Contribute