fell: 60 safer, 0 worse
(aalen_johansen + fisher_bh, BH q<0.05 over 400 tests, unpaired)
"no cell changed" is rejected (p = 0.000249938 < q 0.05) over 400 cells x 1 event
min-p of location and dispersion; 4000 within-cell permutations, arm_label, seed 0
p floor 2.50e-04: the smallest p 4000 permutations can express; a p at the floor is a bound, not a measurement
| event | mean delta (pp) | p | rms delta (pp) | p |
|---|---|---|---|---|
| fell | -10.0 | 2.50e-04 | 21.0 | 2.50e-04 |
informative cells: 155 of 400; 245 unchanged by every exchange
grouping mu (axis), 20 groups; cells per group min 20 / median 20 / max 20; 0 degenerate; 20 of 20 groups resolve 20pp at alpha 0.0025
"no cell in the group changed" tested by BH across 20 groups at q 0.05: 19 rejected
group p floor 2.50e-04: the same 4000 permutations; a group p at the floor is a bound, not a measurement
| mu | cells | informative | unchanged by every exchange | units (a/b) | group MDE (pp) | p | q | verdict |
|---|---|---|---|---|---|---|---|---|
| 0.05 | 20 | 12 | 8 | 640/640 | 10.7pp | 2.50e-04 | 3.12e-04 | rejected |
| 0.1 | 20 | 13 | 7 | 640/640 | 10.7pp | 2.50e-04 | 3.12e-04 | rejected |
| 0.15 | 20 | 12 | 8 | 640/640 | 10.7pp | 2.50e-04 | 3.12e-04 | rejected |
| 0.2 | 20 | 10 | 10 | 640/640 | 10.7pp | 2.50e-04 | 3.12e-04 | rejected |
| 0.25 | 20 | 9 | 11 | 640/640 | 10.7pp | 2.50e-04 | 3.12e-04 | rejected |
| 0.3 | 20 | 7 | 13 | 640/640 | 10.7pp | 2.50e-04 | 3.12e-04 | rejected |
| 0.35 | 20 | 8 | 12 | 640/640 | 10.7pp | 2.50e-04 | 3.12e-04 | rejected |
| 0.4 | 20 | 8 | 12 | 640/640 | 10.7pp | 2.50e-04 | 3.12e-04 | rejected |
| 0.45 | 20 | 8 | 12 | 640/640 | 10.7pp | 2.50e-04 | 3.12e-04 | rejected |
| 0.5 | 20 | 9 | 11 | 640/640 | 10.7pp | 2.50e-04 | 3.12e-04 | rejected |
| 0.55 | 20 | 9 | 11 | 640/640 | 10.7pp | 2.50e-04 | 3.12e-04 | rejected |
| 0.6 | 20 | 8 | 12 | 640/640 | 10.7pp | 2.50e-04 | 3.12e-04 | rejected |
| 0.65 | 20 | 6 | 14 | 640/640 | 10.7pp | 2.50e-04 | 3.12e-04 | rejected |
| 0.7 | 20 | 5 | 15 | 640/640 | 10.7pp | 2.50e-04 | 3.12e-04 | rejected |
| 0.75 | 20 | 5 | 15 | 640/640 | 10.7pp | 2.50e-04 | 3.12e-04 | rejected |
| 0.8 | 20 | 6 | 14 | 640/640 | 10.7pp | 0.00875 | 0.00921 | rejected |
| 0.85 | 20 | 6 | 14 | 640/640 | 10.7pp | 7.50e-04 | 8.82e-04 | rejected |
| 0.9 | 20 | 5 | 15 | 640/640 | 10.7pp | 2.50e-04 | 3.12e-04 | rejected |
| 0.95 | 20 | 5 | 15 | 640/640 | 10.7pp | 0.075 | 0.075 | not rejected |
| 1.0 | 20 | 4 | 16 | 640/640 | 10.7pp | 1.00e-03 | 0.00111 | rejected |
"this cell did not change" tested by BH within each of 19 rejecting groups at q 0.05: 62 of 380 rejected
MDE is ADR-018's smallest detectable increase from arm A's rate, capped at the headroom, so on a cell at ceiling it reads the headroom and is not a power number.
mu=0.05: 20 tests, 9 rejected; cell MDE median 2pp / max 12pp
| cell | event | delta (pp) | q | MDE |
|---|---|---|---|---|
| mu=0.05, push_pct_bw=80.0 | fell | -43.8 | 6.57e-05 | 2pp |
| mu=0.05, push_pct_bw=70.0 | fell | -31.2 | 0.00663 | 3pp |
| mu=0.05, push_pct_bw=90.0 | fell | -25.0 | 0.0106 | 2pp |
| mu=0.05, push_pct_bw=30.0 | fell | -71.9 | 2.55e-09 | 2pp |
| mu=0.05, push_pct_bw=50.0 | fell | -50.0 | 9.84e-06 | 2pp |
| mu=0.05, push_pct_bw=60.0 | fell | -37.5 | 3.93e-04 | 2pp |
| mu=0.05, push_pct_bw=10.0 | fell | -75.0 | 7.19e-09 | 12pp |
| mu=0.05, push_pct_bw=40.0 | fell | -71.9 | 2.55e-09 | 2pp |
| mu=0.05, push_pct_bw=20.0 | fell | -81.2 | 6.03e-11 | 2pp |
mu=0.1: 20 tests, 10 rejected; cell MDE median 2pp / max 34pp
| cell | event | delta (pp) | q | MDE |
|---|---|---|---|---|
| mu=0.1, push_pct_bw=80.0 | fell | -56.2 | 5.78e-06 | 3pp |
| mu=0.1, push_pct_bw=90.0 | fell | -43.8 | 6.57e-05 | 2pp |
| mu=0.1, push_pct_bw=10.0 | fell | -34.4 | 7.71e-04 | 34pp |
| mu=0.1, push_pct_bw=100.0 | fell | -34.4 | 7.71e-04 | 2pp |
| mu=0.1, push_pct_bw=70.0 | fell | -75.0 | 1.26e-08 | 16pp |
| mu=0.1, push_pct_bw=30.0 | fell | -46.9 | 1.94e-04 | 32pp |
| mu=0.1, push_pct_bw=60.0 | fell | -81.2 | 4.44e-10 | 6pp |
| mu=0.1, push_pct_bw=50.0 | fell | -37.5 | 0.00661 | 32pp |
| mu=0.1, push_pct_bw=20.0 | fell | -50.0 | 9.84e-06 | 33pp |
| mu=0.1, push_pct_bw=40.0 | fell | -59.4 | 5.31e-07 | 30pp |
mu=0.15: 20 tests, 5 rejected; cell MDE median 3pp / max 34pp
| cell | event | delta (pp) | q | MDE |
|---|---|---|---|---|
| mu=0.15, push_pct_bw=90.0 | fell | -59.4 | 2.39e-06 | 3pp |
| mu=0.15, push_pct_bw=80.0 | fell | -43.8 | 0.00483 | 23pp |
| mu=0.15, push_pct_bw=70.0 | fell | -68.8 | 3.21e-08 | 26pp |
| mu=0.15, push_pct_bw=60.0 | fell | -59.4 | 7.97e-07 | 30pp |
| mu=0.15, push_pct_bw=50.0 | fell | -25.0 | 0.019 | 34pp |
mu=0.2: 20 tests, 4 rejected; cell MDE median 7pp / max 34pp
| cell | event | delta (pp) | q | MDE |
|---|---|---|---|---|
| mu=0.2, push_pct_bw=90.0 | fell | -53.1 | 3.32e-04 | 16pp |
| mu=0.2, push_pct_bw=60.0 | fell | -40.6 | 3.53e-04 | 34pp |
| mu=0.2, push_pct_bw=80.0 | fell | -81.2 | 4.44e-10 | 12pp |
| mu=0.2, push_pct_bw=70.0 | fell | -43.8 | 8.79e-04 | 33pp |
mu=0.25: 20 tests, 6 rejected; cell MDE median 2pp / max 34pp
| cell | event | delta (pp) | q | MDE |
|---|---|---|---|---|
| mu=0.25, push_pct_bw=60.0 | fell | -25.0 | 0.0423 | 34pp |
| mu=0.25, push_pct_bw=110.0 | fell | -21.9 | 0.0423 | 2pp |
| mu=0.25, push_pct_bw=70.0 | fell | -50.0 | 2.46e-05 | 33pp |
| mu=0.25, push_pct_bw=80.0 | fell | -65.6 | 5.70e-07 | 26pp |
| mu=0.25, push_pct_bw=100.0 | fell | -50.0 | 6.88e-05 | 3pp |
| mu=0.25, push_pct_bw=90.0 | fell | -43.8 | 0.00496 | 25pp |
mu=0.3: 20 tests, 5 rejected; cell MDE median 7pp / max 34pp
| cell | event | delta (pp) | q | MDE |
|---|---|---|---|---|
| mu=0.3, push_pct_bw=90.0 | fell | -81.2 | 4.44e-10 | 12pp |
| mu=0.3, push_pct_bw=70.0 | fell | -43.8 | 1.97e-04 | 34pp |
| mu=0.3, push_pct_bw=60.0 | fell | -31.2 | 0.00426 | 34pp |
| mu=0.3, push_pct_bw=80.0 | fell | -50.0 | 3.05e-04 | 30pp |
| mu=0.3, push_pct_bw=110.0 | fell | -21.9 | 0.0433 | 2pp |
mu=0.35: 20 tests, 4 rejected; cell MDE median 12pp / max 34pp
| cell | event | delta (pp) | q | MDE |
|---|---|---|---|---|
| mu=0.35, push_pct_bw=90.0 | fell | -53.1 | 1.62e-04 | 29pp |
| mu=0.35, push_pct_bw=80.0 | fell | -65.6 | 5.70e-07 | 26pp |
| mu=0.35, push_pct_bw=60.0 | fell | -28.1 | 0.0102 | 34pp |
| mu=0.35, push_pct_bw=70.0 | fell | -40.6 | 3.53e-04 | 34pp |
mu=0.4: 20 tests, 3 rejected; cell MDE median 11pp / max 34pp
| cell | event | delta (pp) | q | MDE |
|---|---|---|---|---|
| mu=0.4, push_pct_bw=90.0 | fell | -37.5 | 0.0342 | 29pp |
| mu=0.4, push_pct_bw=100.0 | fell | -40.6 | 0.0222 | 12pp |
| mu=0.4, push_pct_bw=80.0 | fell | -31.2 | 0.0265 | 34pp |
mu=0.45: 20 tests, 1 rejected; cell MDE median 16pp / max 34pp
| cell | event | delta (pp) | q | MDE |
|---|---|---|---|---|
| mu=0.45, push_pct_bw=80.0 | fell | -53.1 | 1.64e-05 | 32pp |
mu=0.5: 20 tests, 2 rejected; cell MDE median 15pp / max 34pp
| cell | event | delta (pp) | q | MDE |
|---|---|---|---|---|
| mu=0.5, push_pct_bw=80.0 | fell | -43.8 | 3.94e-04 | 34pp |
| mu=0.5, push_pct_bw=90.0 | fell | -43.8 | 0.00176 | 33pp |
mu=0.55: 20 tests, 5 rejected; cell MDE median 9pp / max 34pp
| cell | event | delta (pp) | q | MDE |
|---|---|---|---|---|
| mu=0.55, push_pct_bw=70.0 | fell | -28.1 | 0.0102 | 34pp |
| mu=0.55, push_pct_bw=110.0 | fell | -31.2 | 0.0106 | 3pp |
| mu=0.55, push_pct_bw=80.0 | fell | -50.0 | 4.92e-05 | 33pp |
| mu=0.55, push_pct_bw=90.0 | fell | -40.6 | 0.00987 | 31pp |
| mu=0.55, push_pct_bw=100.0 | fell | -56.2 | 1.08e-04 | 16pp |
mu=0.6: 20 tests, 3 rejected; cell MDE median 13pp / max 34pp
| cell | event | delta (pp) | q | MDE |
|---|---|---|---|---|
| mu=0.6, push_pct_bw=70.0 | fell | -25.0 | 0.0317 | 34pp |
| mu=0.6, push_pct_bw=80.0 | fell | -31.2 | 0.0265 | 34pp |
| mu=0.6, push_pct_bw=90.0 | fell | -71.9 | 3.39e-08 | 23pp |
mu=0.65: 20 tests, 3 rejected; cell MDE median 14pp / max 34pp
| cell | event | delta (pp) | q | MDE |
|---|---|---|---|---|
| mu=0.65, push_pct_bw=90.0 | fell | -40.6 | 0.0111 | 32pp |
| mu=0.65, push_pct_bw=100.0 | fell | -50.0 | 0.00258 | 22pp |
| mu=0.65, push_pct_bw=80.0 | fell | -25.0 | 0.0317 | 34pp |
mu=0.7: 20 tests, 0 rejected; cell MDE median 13pp / max 34pp
mu=0.75: 20 tests, 1 rejected; cell MDE median 16pp / max 34pp
| cell | event | delta (pp) | q | MDE |
|---|---|---|---|---|
| mu=0.75, push_pct_bw=100.0 | fell | -40.6 | 0.0482 | 25pp |
mu=0.8: 20 tests, 0 rejected; cell MDE median 23pp / max 34pp
mu=0.85: 20 tests, 0 rejected; cell MDE median 22pp / max 34pp
mu=0.9: 20 tests, 0 rejected; cell MDE median 16pp / max 34pp
mu=1.0: 20 tests, 1 rejected; cell MDE median 15pp / max 34pp
| cell | event | delta (pp) | q | MDE |
|---|---|---|---|---|
| mu=1.0, push_pct_bw=110.0 | fell | -34.4 | 0.0471 | 6pp |
depth 3 (3 configured); outer-nodes false discovery rate at most 0.432 = 2 x 3 x 0.05 x 1.44
Yekutieli (2008) Corollary A.2 assumes each p-value is independent of its ancestors; this tree's group and root statistics aggregate their own cells, so that assumption is not established here.
| push_pct_bw | mu | n (a/b) | rate_a (pp) | rate_b (pp) | delta (pp) | p | q | verdict | MDE | interval (pp) |
|---|---|---|---|---|---|---|---|---|---|---|
| 90 | 0.95 | 32/32 | 3.1 | 15.6 | 12.5 | 0.196 | 0.777 | unchanged | 25pp | [-10.0, 36.4] |
| 120 | 0.5 | 32/32 | 93.8 | 100.0 | 6.2 | 0.492 | 1 | unchanged | 6pp | [-12.9, 27.5] |
| 80 | 1 | 32/32 | 0.0 | 6.2 | 6.2 | 0.492 | 1 | unchanged | 23pp | [-12.9, 27.5] |
| 120 | 0.9 | 32/32 | 90.6 | 93.8 | 3.1 | 1 | 1 | unchanged | 9pp | [-19.8, 26.5] |
| 20 | 0.25 | 32/32 | 0.0 | 3.1 | 3.1 | 1 | 1 | unchanged | 23pp | [-15.7, 23.2] |
| 130 | 0.7 | 32/32 | 100.0 | 100.0 | 0.0 | 1 | 1 | unchanged | 2pp | [-18.5, 18.5] |
| 180 | 0.4 | 32/32 | 100.0 | 100.0 | 0.0 | 1 | 1 | unchanged | 2pp | [-18.5, 18.5] |
| 150 | 0.85 | 32/32 | 100.0 | 100.0 | 0.0 | 1 | 1 | unchanged | 2pp | [-18.5, 18.5] |
| 170 | 0.1 | 32/32 | 100.0 | 100.0 | 0.0 | 1 | 1 | unchanged | 2pp | [-18.5, 18.5] |
| 160 | 0.1 | 32/32 | 100.0 | 100.0 | 0.0 | 1 | 1 | unchanged | 2pp | [-18.5, 18.5] |
Worst-powered unchanged cell (push_pct_bw=100, mu=0.95, n=32/32) can only detect a 34pp rise. "0 worse" is an honest claim only above that.
Worst-powered cell overall: 34pp (push_pct_bw=10, mu=0.1).
intervals: 99.2500% (alpha* = 0.0075 = q x 60 / 400 over the flat BH family), basis score_rd
interval excludes zero, verdict unchanged: 6 cells
verdict significant, interval covers zero: 0 cells
| method | pairing | safer | worse | unchanged | max MDE | q |
|---|---|---|---|---|---|---|
| fisher_bh · fell | unpaired | 60 | 0 | 340 | 34pp | 0.05 |
| mcnemar_bh · fell | paired | 55 | 0 | 345 | 34pp | 0.05 |
| logrank_bh (cause-specific) · fell | unpaired | 88 | 0 | 312 | 34pp | 0.05 |
This is empirical evidence under a stated test envelope, not formal verification. Cells outside the envelope were not tested; cells inside it were tested at the stated replicate count and can only detect effects at or above the reported MDE.