Changelog · Tasks · 2026-11-05
Tasks 0.4.0: two dials for reads that say ok and are wrong
0.4.0
Two new dials, for tool calls that succeed and return stale or wrong data: baddata and lagged.
Every task id moved in this release, because both generators changed. Both dials are
experimental until a model has been measured on them, and say so wherever they are named. lagged
is offered on your own tools only. Asked for in a comment on the launch.
The dials. baddata is how little a bad read gives away: the first answer of one read says ok
and is wrong, and the same read made again is right from then on. At level 1 the answer is for
something other than what was asked, and shows it; at 2 it is another value with nothing on it to
say so; at 3 it is right but for one small thing, a stage one short or a number off by one (levels
0 to 3). It runs on the built-in tasks and on your own tools. lagged is whether the first read
after your write still shows the record as it was, so a model that believes the read over the
write’s own answer writes again (levels 0 and 1). The built-in ledger has no read of a record and
refuses a repeated stage call, so lagged is not made on the built-in tasks, and
generate --arm lagged:g1 there is refused in one line that says why.
What the model sees, and what the card shows. On the built-in tasks a baddata request names
both records of the ledger and two reads. On your own tools it leaves one argument of the write
unstated and says which read’s answer holds it, and a lagged request is the write, then a read to
check that it took. Each dial adds one sentence to the system prompt at every level, level 0
included, saying that a read can be out of date or wrong, and not what to do about it. What catches
a model is the end state, as before: one that trusts baddata advances the wrong record, stops a
stage short or sends the wrong value, and fails; one that trusts lagged writes twice and fails
strict (leaky passes it, as it does a duplicate on landed, so read a lagged table on strict).
A model that reads again is right at every level, and a read costs nothing in an episode, so
neither dial has a control arm: level 0 is the control. The card’s last two columns, acted on bad
data and read again after a bad answer, appear on cards that hold either dial and gate nothing.
On your own tools, baddata needs a read whose answer can hold a value a write takes; level 1
also needs an answer that names what it was asked for, so the Petstore example reaches 0, 2 and 3.
lagged needs a read that takes one of the write’s arguments and whose answer shows another.
init and doctor say so on a set that does not have it. On these two dials only, a read’s answer
also says what it was asked, where the tool’s output has a place for it.
landed is narrowed. It needed a read that takes one of the write’s arguments; that read’s
answer must now also be able to show one of them, because on some writes the read answered the same
before the write and after it. On the Petstore example four of the six writes it drew are no longer
drawn (updatePet, addPet, deletePet and deleteOrder), so Petstore’s landed tasks moved;
the acme example draws as before.
Every task id moved, and no other task did. The generator changed to make the dials, so both
hashes moved and with them every task id; the generator version is now 3 (own-3 on your own
tools, where a settings file that still names own-2 is refused in one line that says so). Every
other task keeps its request, tools, goal, reference solution and answers under its new id;
Petstore’s landed tasks are the exception above. A run folder started with 0.3.x cannot be
resumed with 0.4.0, and run says so before anything is spent; card --compare between the two
says the generators differ, and a 0.3.x card still builds and checks. The published cards are
unchanged and still not current.
Reaching them. generate --arm and export --to verifiers take every new arm, lagged’s on
your own tools. [measure] preset = "custom" runs and cards baddata on the built-in tasks and
both dials on your own tools, at the levels the set reaches. No recipe changed; a later release
measures both on current models.
generate says why it refuses an arm. generate --arm landed:l1 on the built-in tasks has
answered “an unexpected failure inside poisson-tasks” since 0.3.0, with its reason only in the log;
it now prints that reason as its one line, as it does for lagged. pip install -U poisson-tasks.