Tools · open source · In development
Gym
The gym is an open-source training environment for agents, built on Tasks, Break, Trace and Prove. An episode is a seeded task in a sandboxed world: the agent acts through tools, the world changes, and at the end a check the agent cannot see reads the world’s state and the actions taken and gives the reward.
The gym runs many such episodes for an RL trainer, draws the next ones from where the agent currently fails, records every one, and reports for each checkpoint whether training helped, where, and by how much.
In development. There is nothing to install yet.
How it works
- 1 TasksSupplies the tasks: seeded worlds, the tools the agent acts through, and the checks that give the reward, which the agent never sees.
- 2 BreakSearches for the conditions where the current checkpoint fails. Those failures become the curriculum: what the agent practises next.
- 3 TraceThe gym records every episode as it runs. Trace reads the records and says at which step, and which kind of step, the remaining failures happen.
- 4 ProveCompares each checkpoint with its starting point on held-out conditions training never touched, and reports whether training helped, with a stated confidence.
No model grades an episode. Every reward comes from a programmatic check.
Why it exists
- Hard cases written by hand do not scale
- The gym’s tasks are generated from seeds, with difficulty set one dial at a time, and the hard ones are found by searching for where the current checkpoint fails rather than written in advance.
- Agents learn to game the reward
- An agent under training exploits any gap between the reward and the task. In the gym the reward comes from checks on the world’s end state and on the actions taken, never from a model grading text, and a second set of checks the reward never sees looks for episodes that pass without doing the work.
- Most runs never establish whether training helped
- A rising training reward does not say whether the agent got better at work it has not seen. The gym evaluates every checkpoint against its starting point on conditions declared before training starts, and Prove says which improved, which regressed and which showed no significant change, with what it couldn’t have detected.
What the first release includes
- The tool-use agent pack
- Seeded multi-step tasks over a chain of tool calls, built on Tasks, with its end-state checks and negative controls.
- A local sandbox
- Each episode in a fresh, isolated world on your own machine, with limits on time and memory, so a stuck episode cannot stall training.
- One trainer adapter
- The gym plugs into an RL training framework labs already use. It does not train models itself.
- A curriculum from Break’s search
- The conditions where the current checkpoint fails decide which tasks it practises next.
- A Prove report on the training run
- Each checkpoint against its starting point, condition by condition, at a stated confidence and beside the smallest change each condition could have detected.
Packs planned after tool use
A pack is one kind of agent work: its tasks, its world and its checks. Coding and debugging comes next; the order of the rest is not settled.
- Coding and debugging
- Bugs planted in real code at pinned commits, fixed inside a container. Hidden tests the agent cannot read or edit decide the reward.
- Data and SQL agents
- A database per episode with seeded schemas and data. The result and the database’s end state decide the reward, and writing where only reading was asked is a failure.
- Multi-agent
- Several agents, or an agent with scripted counterparts, in one world, checked on the state they leave it in together.
- Browser agents
- Self-hosted web apps in a headless browser, checked on the app’s back-end state and on the path the agent took to reach it.
- Computer use
- A virtual machine per episode, with scripted checks of its state afterwards.
- Simulated robotics policies
- Continuous-control policies in a physics simulator, where the simulator’s exact state is the check.
Open and hosted
The gym is open source. It runs on your own machines or your own cloud, needs no Poisson service and sends nothing anywhere. The episode loop, the sandbox, the checks, the packs we publish, the curriculum, every trainer adapter and the Prove report are all in it.
Poisson Cloud will offer the gym hosted, alongside the rest of the suite, with the compute to run it. More on the platform page.
Labs and teams training agents can write to work@poissonlabs.ai.