A mission answers four questions, each in a different place
Every part of long_tamp exists to answer one of these questions and hand a checked result to the next part. Click a question to see who answers it and what comes out.
Symbols above the seam, geometry below it
The mission stack reasons about which gripper holds what and which screws are in. The motion stack reasons about constraint graphs and paths. They meet at one interface, the refiner.
Click a package to see its job, what it imports (blue) and what imports it (amber). These are the real import edges at the review commit.
Follow one mission from instruction to motion
Two UR10 arms fasten plates to a jig, two screws per plate. Step through what runs, in order. The plan and the event log on the right fill in as you go.
TaskPlan (simplified)
What the event stream records
Paraphrased, with illustrative values. The real stream is JSONL, one record per status change (schema).
One small graph per phase instead of one huge graph
HPP plans manipulation on a constraint graph: a state per grasp combination, an edge per transition. One graph that covers every way a long sequence could go grows factorially. GraspSequencePlanner builds a minimal graph for each phase instead, so the work grows linearly.
Log scale. The screw assembly with 4 parts is dozens of phases.
The per-phase loop
The lookahead in step 3 tries targets on a copy of the grasp tracker and keeps one from which the next phase stays reachable. Without it, the IKEA table prototype failed about 1850 of 1850 solver calls on one phase.
Each failure is handled at the lowest level that can fix it
Pick a failure and watch it climb. Levels 1 to 4 are deterministic. A model only gets a say at level 5, and only about the goal.
Models propose, long_tamp checks
Language models are optional and never write steps or motion. Each use is a typed role: the model proposes, a deterministic checker accepts or explains why not, and the reasons go back for a bounded number of rounds. After that, a fallback that needs no model.
What must stay responsive runs in its own process
HPP planning holds Python's interpreter lock for seconds at a time. Anything sharing the mission's process stops while a step plans. Switch between the two layouts and watch the scene and the simulator.
They talk through the event stream (a JSONL file), pipes to the simulator, a local socket for scene frames, and HTTP for the viewer's controls, which act at step boundaries. Measured: MuJoCo in a thread ran 3.6× slower than real time while planning ran next to it.
Where your code plugs in
Each extension point is a small interface. Whatever you plug in passes through the same checks as the built-in parts.