Instrument

Proofload

Find where a policy or an agent breaks, and prove whether your change moved it.

Name the conditions. Proofload runs your system across them, before and after a change, and returns what moved, and what was too small for the run to detect. Your physics, your agent, your outcome. It brings the sweep, the seeds, the store and the statistics.

Open source, releasing soon. We are using it on our own work first. Write to be told when it ships, or to use it early.

PUSH, % BODYWEIGHT → ↑ FRICTION μ 32 TRIALS PER CELL PER VERSION

Go1 locomotion, v0 → v1. Hover any cell.

Fig. 1Go1 locomotion, v0 → v1. 400 conditions of floor friction and lateral push, 32 trials per cell, paired on seed. 60 conditions significantly safer, 0 worse. The worst-powered cell with no significant change could only detect a 34pp rise. report.md

The check on a real agent

A code agent on a real model, run across 328 conditions twice on the same seeds with nothing changed between the runs. One condition still moved.

failed_assertion: 0 safer, 1 worse, 327 no significant change. The one that moved is what the substrate does on its own, and it is the noise floor a real change has to clear. The worst-powered cell could not have seen a change under 34 points, and the report says so on the line that matters.

failed_assertion: 0 safer, 1 worse, 327 no significant change over 328 conditions. Hover any point.

Fig. 2Each condition's failure rate in the first run against the second, one point per condition. Agreement is the dashed diagonal. 327 conditions sit within noise of it; one does not, and the paired test names it. Runs 20260905T185407Z-0313 and 20260905T195009Z-18ac: 164 HumanEval tasks at two temperatures, 32 episodes per cell, gpt-4o-mini on Modal. report.md

How it works

Map

The conditions you name, a seed per condition and replicate derived from a run salt, every episode appended to a store with its config, seed, outcome and time. A second run with the same salt gets the same seeds, so two runs pair by construction.

Diff

Per condition, the event rate with censoring handled. Between two runs, a paired test with a false-discovery correction across every condition, and a verdict for each: safer, worse, or no significant change.

Say what it could not see

Every verdict carries the smallest effect that condition had the power to detect. "None worse" at a given N is a claim only above that number, and the report puts it on the line that matters. What was too small to see is said, not implied.

One function

import proofload

def rollout(cfg: dict, seed: int) -> proofload.Episode:
    result = run_my_system(task=cfg["task_id"], seed=seed)
    return proofload.Episode(
        outcome=result.terminal_state,  # your label
        t=result.steps,
        horizon=20,
    )

The head of a real report.md

# Go1 v0 -> v1

fell: 60 safer, 0 worse
(aalen_johansen + fisher_bh, BH q<0.05 over 400 tests, paired)

Worst-powered unchanged cell (push_pct_bw=100, mu=0.95, n=32/32) can only detect a 34pp rise. "0 worse" is an honest claim only above that.
Worst-powered cell overall: 34pp (push_pct_bw=10, mu=0.1).

It is a statistical instrument, not a simulator, an environment framework, a benchmark or a training loop. It never reaches into your process, and it never phones home.

Where it has been used

2026.09 Where a 14-DoF Biped Falls Two checkpoints of a Microduck walking policy under a lateral push: a cliff and a slope, with a crossover between 0.600 and 0.625 m/s. A retrain meant to widen the boundary was worse in 14 of 14 cells. Microducklocomotion read report receipt retrain report
2026.08 Map the Failure Boundary A Go1 locomotion policy across 400 conditions of floor friction and lateral push. Retrained on the two gaps the map exposed: 60 conditions significantly safer, none worse. Go1locomotion read report receipt

Empirical evidence under a stated test envelope, not formal verification. Conditions outside the envelope were not tested; conditions inside it can only resolve effects at or above the reported MDE.