Where does my controller break?

A flight controller that tracks well in calm air tells you little about the gust that knocks it off its path. This example finds that gust. It sweeps a simulated 27 g quadrotor through wind gusts with Prove and reads off, direction by direction, the gust speed at which half of its flights crash. That speed is v50, and the set of them is a boundary map.

The drone, the gusts and the seeds are the ones behind Where does a trained drone controller break?, and the code is that study’s public repository, so the map you draw sits beside the maps it stored. You run it from a clone. There is nothing to pip install and no account to make.

Everything here is in simulation: one drone model, one air model, one trajectory and these gusts. A speed in m/s does not carry to a real drone, which is why every speed is printed with a flat force ratio.

1. Clone the repository and set it up

You need Python 3.11, uv and git. The clone is about 200 MB, most of it the study’s stored flights.

git clone https://github.com/poisson-labs/drone-robustness-post1
cd drone-robustness-post1
uv sync

uv sync builds an environment from the repository’s lock file: the simulation stack, the two classical controllers, MuJoCo and Prove at the version that produced the study’s numbers.

2. Sweep the PID

The example is one script. --quick flies two gust directions, a headwind and a tailwind, at 11 gust speeds and two gust timings, 24 flights each: 1,056 flights, about five minutes on two cores.

uv run python examples/where-does-my-controller-break/where_does_my_controller_break.py --flyer pid --quick --jobs 2

Output15 lines · exit 0

cascaded PID: 2 directions x 11 speeds x 2 timings = 44 cells, 24 episodes each, 1,056 episodes; store examples/where-does-my-controller-break/prove-store

Beside the study's stored counts (N1 of RESULTS-P1.md db173c86fbcf): 44 of 44 cells have the same crash count; the counts differ by 0 episodes in all, of 1,056.

direction       v50, m/s                                timing 0  timing 0.125  PID     16 policies, lowest to highest    stored
az 0° el 0°      9.24                                   9.60      9.00          9.24    8.09 to 9.71                      9.24
az 180° el 0°   10.16                                   9.09      11.04         10.16   11.00 to 12.50                    10.16

crashes of episodes at each speed, both timings pooled (speed m/s · flat force ratio):
                2·0.03    3·0.06    4·0.09    5·0.13    6·0.17    7·0.22    8·0.28    10·0.41   12·0.57   14·0.75   16·0.95
az 0° el 0°     0/48      0/48      0/48      0/48      0/48      0/48      1/48      38/48     48/48     48/48     48/48
az 180° el 0°   0/48      0/48      0/48      0/48      0/48      0/48      0/48      22/48     47/48     48/48     48/48

wrote examples/where-does-my-controller-break/boundary-pid.svg and examples/where-does-my-controller-break/boundary-pid.json
In simulation: one drone model, one air model, one trajectory, these gusts. 24 episodes a cell cannot show a direction is safe.

The PID holds a headwind to 9.24 m/s and a tailwind to 10.16 m/s, and those are the study’s numbers to the digit. Prove derives each flight’s seed from the gust and a fixed salt, so flight 0 of a gust here starts where the study’s did and meets the same turbulence and the same sensor noise. A controller with no network in it is deterministic on top of that, so all 44 of your crash counts are the study’s.

The tailwind’s two timings disagree: 9.09 m/s when the gust peaks at the path’s fastest point and 11.04 m/s when it peaks in the tightest part of a lobe. Pooled, they give 10.16. A pooled number can hide a controller that is fine at one moment of its path and fragile at another, so the script prints all three.

The script also writes boundary-pid.svg beside itself: your two points on a map with the study’s PID and each of its sixteen trained policies.

3. Check your flights against the study’s, row by row

uv run python examples/where-does-my-controller-break/compare_with_study.py --flyer pid

Output5 lines · exit 0

where-pid-az0-el0: 528 of 528 rows equal to the study's on every field
where-pid-az180-el0: 528 of 528 rows equal to the study's on every field

pid: 1056 of 1056 rows equal on every field; 1056 of 1056 with the same outcome
every field equal

Every one of the 1,056 flights you flew equals the study’s stored flight on every field: how it ended and when, each crash rule’s first crossing, the tracking error and the rotor effort.

4. Sweep your own controller

A controller is a class with reset() and act(view). It reads the same noisy, delayed state every flyer gets and returns a thrust and three body rates. The rate loop, the mixer and the motors are the study’s, the same for every flyer. my_controller.py is a stand-in for yours: the study’s geometric controller with softer gains.

"""Your own controller, as a Prove target:

    uv run python examples/where-does-my-controller-break/where_does_my_controller_break.py --flyer my_controller:rollout --quick --jobs 2

A controller is a class with `reset()` and `act(view)`. `view.reading` is the noisy, delayed state every flyer gets at
100 Hz (position, velocity, rotation, body rate), and `view.reference` is the path's next five points, 0.1 s apart. `act`
returns a collective thrust in newtons and three body rates in rad/s; the study's rate loop, mixer and motors do the rest,
the same for every flyer. See `drone_study.stack.Controller` and `ControllerView`.

This one is a stand-in for yours: the study's geometric controller with softer gains than the study picked. Replace it.
"""
from drone_study.controllers import GeometricController, GeometricGains
from drone_study.rollout import rollout_for

rollout = rollout_for(GeometricController(GeometricGains(kp=6.0, kv=3.5, ki=2.0)))

Name it to the script as module:name.

uv run python examples/where-does-my-controller-break/where_does_my_controller_break.py --flyer my_controller:rollout --quick --jobs 2

Output13 lines · exit 0

my_controller:rollout: 2 directions x 11 speeds x 2 timings = 44 cells, 24 episodes each, 1,056 episodes; store examples/where-does-my-controller-break/prove-store

direction       v50, m/s                                timing 0  timing 0.125  PID     16 policies, lowest to highest
az 0° el 0°      7.45                                   7.50      7.39          9.24    8.09 to 9.71
az 180° el 0°    7.50                                   6.79      8.29          10.16   11.00 to 12.50

crashes of episodes at each speed, both timings pooled (speed m/s · flat force ratio):
                2·0.03    3·0.06    4·0.09    5·0.13    6·0.17    7·0.22    8·0.28    10·0.41   12·0.57   14·0.75   16·0.95
az 0° el 0°     0/48      0/48      0/48      0/48      0/48      6/48      46/48     48/48     48/48     48/48     48/48
az 180° el 0°   0/48      0/48      0/48      0/48      1/48      15/48     33/48     48/48     48/48     48/48     48/48

wrote examples/where-does-my-controller-break/boundary-my_controller-rollout.svg and examples/where-does-my-controller-break/boundary-my_controller-rollout.json
In simulation: one drone model, one air model, one trajectory, these gusts. 24 episodes a cell cannot show a direction is safe.

With those gains it breaks near 7.5 m/s in both directions, 1.8 m/s below the PID in a headwind and below every one of the sixteen policies.

5. Change it and sweep again

Open my_controller.py and stiffen the gains.

diff --git a/examples/where-does-my-controller-break/my_controller.py b/examples/where-does-my-controller-break/my_controller.py
index d4d9778..27aae76 100644
--- a/examples/where-does-my-controller-break/my_controller.py
+++ b/examples/where-does-my-controller-break/my_controller.py
@@ -12,4 +12,4 @@ This one is a stand-in for yours: the study's geometric controller with softer g
 from drone_study.controllers import GeometricController, GeometricGains
 from drone_study.rollout import rollout_for

-rollout = rollout_for(GeometricController(GeometricGains(kp=6.0, kv=3.5, ki=2.0)))
+rollout = rollout_for(GeometricController(GeometricGains(kp=12.0, kv=7.0, ki=2.0)))

Then sweep again with a new --store. The script keeps every flight in a Prove store and looks runs up by the flyer’s name, so a second run under the same name reads what is stored and flies nothing. That is what makes a repeat instant, and it also means an edited controller run into the old store prints the old controller’s map. Give each version of your controller its own store and its own output name.

uv run python examples/where-does-my-controller-break/where_does_my_controller_break.py --flyer my_controller:rollout --quick --jobs 2 --store examples/where-does-my-controller-break/prove-store-stiffer --out examples/where-does-my-controller-break/boundary-stiffer

Output13 lines · exit 0

my_controller:rollout: 2 directions x 11 speeds x 2 timings = 44 cells, 24 episodes each, 1,056 episodes; store examples/where-does-my-controller-break/prove-store-stiffer

direction       v50, m/s                                timing 0  timing 0.125  PID     16 policies, lowest to highest
az 0° el 0°      9.45                                   9.26      9.71          9.24    8.09 to 9.71
az 180° el 0°   11.02                                   11.09     10.96         10.16   11.00 to 12.50

crashes of episodes at each speed, both timings pooled (speed m/s · flat force ratio):
                2·0.03    3·0.06    4·0.09    5·0.13    6·0.17    7·0.22    8·0.28    10·0.41   12·0.57   14·0.75   16·0.95
az 0° el 0°     0/48      0/48      0/48      0/48      0/48      0/48      0/48      33/48     48/48     48/48     48/48
az 180° el 0°   0/48      0/48      0/48      0/48      0/48      0/48      0/48      1/48      46/48     48/48     48/48

wrote examples/where-does-my-controller-break/boundary-stiffer.svg and examples/where-does-my-controller-break/boundary-stiffer.json
In simulation: one drone model, one air model, one trajectory, these gusts. 24 episodes a cell cannot show a direction is safe.

The stiffer controller holds a headwind to 9.45 m/s and a tailwind to 11.02, about 2 m/s and 3.5 m/s past the softer one, and no flight of it crashes at 8 m/s or below. This is the map the script drew for it.

The stiffer stand-in controller's v50 for a headwind and a tailwind, drawn on a ring of gust directions beside the study's PID and its sixteen trained policies.

Both versions met the seeds the PID met, gust by gust, so the air was the same for all three. That makes the maps comparable in the plain sense. It does not make 24 flights a test: the counts say where each controller sits, and a claim that one beats another needs a test chosen beforehand and more flights than this.

Going further

Without --quick the script flies the eight level directions (4,224 flights) and draws a full ring; --all flies the study’s whole grid of sixteen directions. --jobs flies that many directions at once.
--flyer geometric flies the study’s geometric controller. --flyer pi0-7 flies one of its sixteen trained policies, which needs PyTorch: run uv sync --group gate first.
The repository’s README shows how to regenerate every number in the post from the stored flights with one command.

What this can and cannot tell you

It is a simulation of one small drone under one air model. Above about 6 m/s of airspeed that air model is an extrapolation.
Twenty-four flights a gust cannot show that a direction is safe.
One trajectory, two gust timings. A different path is a different map.
The two classical controllers are two designs with stated settings, and the trained policies never met wind in training. Where they sit beside each other says nothing about learned and classical control in general.

Stored output

Every output above is what the command printed on a fresh clone of the repository at commit 4b7d0e7, stored in the site repository under src/data/examples/where-does-my-controller-break/ with the three result files the script wrote. my_controller.py is stored there too, byte for byte as the repository gives it at that commit, with its sha256. The page cannot show an output that is not there.

One thing is changed in those outputs. The script prints the store’s path and the paths of the files it wrote in full, so on your machine they begin with wherever you cloned the repository. Here they are shown from the clone’s root. The same steps run again on a fresh clone on 2026-11-12 printed the same lines otherwise, and wrote the same three result files.