Skip to content

Flow pause and resume

A flow pauses at a pause gate or when a tool inside a node needs an approval that is not decided yet. FlowRunner.resume(flow, checkpoint, decision) continues the same run.

sequenceDiagram
  participant N as paused node
  participant E as Executor
  participant P as approval port
  N->>E: call A (finished, journaled)
  N->>E: call B (confirm)
  E->>P: request B
  P-->>E: ApprovalPending
  Note over N: status="paused", checkpoint sealed
  Note over N: resume(decision for B)
  N->>E: call A again
  E-->>N: replayed result (A does not run)
  N->>E: call B again
  E-->>N: decision answers B once; B runs

What a paused result holds

Field Holds
status "paused"
pending the ApprovalRequest: tool (or node:<flow>.<node> for a gate), run and call ids; pending.arguments are the requested arguments (reprs redact them)
checkpoint where to continue: state, frontier, counters, and sealed private parts

What resume does

On re-run Behaviour
executor calls the node already finished replayed from the node's journal, keyed by tool, the canonical form of the validated arguments the caller passed (unset fields left out at every depth, so a default_factory id, even in a nested model, still replays) and order: no execution, run guards, ledger entries or events
finished or skipped Map items kept; not run again
calls inside a Map(fn=...) item replayed from that item's own journal (keyed by item index) while the item is unfinished
a nested flow run inside the node its node visits get child journals inside the outer node's journal: their calls replay too
a replayed failure raises an exception rebuilt from the result's ErrorInfo (same AdapterError class, message and attributes)
approval requests decision, and every earlier answer for the node, each answer once a request with the same run, tool and arguments (and call id for Call / Map); anything else asks the port
a denied tool decision ApprovalDenied in the node: a FlowError, or a skipped Map item
tasks that outlive the node get no answer and no journal
other pending records of the pause a Call / Map(tool=) item record stays pending: its call id repeats, so the port finds it again; any other (a node function's or Map(fn=) item's call, which gets a new call id) is denied and consumed with reason superseded and asked again on resume

A node needing two approvals finishes after two resumes. LangGraph re-runs a resumed node's calls instead of replaying them; the parity case replay_two_confirms records this as an intended difference.

Safety

Rule Effect
sealed every field, in a strict type-tagged canonical form, is covered by HMAC-SHA256 with the runner's checkpoint_key; a changed or foreign checkpoint (or a MAC that is not text) raises FlowError (checkpoint-tampered) and runs nothing
graph the checkpoint holds a digest of the graph: name, state model, start, ends, edges, and per node its name, kind, gate and action (module:qualname of a function, gate payload or route function; a Call's tool, into and static args in canonical form; a Map's tool and options); a changed graph is refused before the approval is spent
key default: a random per-process key, so checkpoints resume only in that process; pass the same checkpoint_key (bytes, 16+) to every runner that should resume them
once a second resume raises ApprovalAlreadyDecided and runs nothing
store when the port has a store (e.g. DeferredApproval), the pending record is decided and consumed; every earlier answer that store holds must be a consumed record with that decision (one it does not hold belongs to another port)
own store a node that runs tools on its own executor with its own DeferredApproval: the answer's record in that store is decided and consumed there; a record decided otherwise denies the call (ApprovalAlreadyDecided)
memory other ports rely on a process-wide memory of resumed approvals (bounded, 24 h); after an entry is evicted only a store can still refuse
no tool a pending that names no tool is refused

A function counts by its module:qualname only: a changed body under the same name, or another lambda in the same scope (same <lambda> qualname), is not detected. Answers still fail closed: one answers only a request with the same run, tool and arguments.

A refused settle (the store holds another decision, or it is unreachable) does not spend the checkpoint: resume again with the store's decision. ToolResult.to_dict() and pickled ApprovalPending drop the arguments, so a pending rebuilt from them asks the port again. A pending result pickles and deep-copies; the port's store stays in the process.

Sealable state

Only these values can be sealed; the encoding never confuses types.

Value Canonical form
None, bool, int, finite float, str, list as is; an int past ~600 digits is tagged as hex (up to 1 MiB)
dict with str keys not starting with $ as is; other dicts become a sorted pair list
bytes, bytearray base64, tagged (invalid UTF-8 is fine)
tuple, set, frozenset, NaN / infinity tagged (sets sorted)
date, timedelta, Decimal, UUID, Enum tagged
datetime, time tagged with fold and the zone: datetime.timezone as offset seconds (and its name), zoneinfo.ZoneInfo by key; other tzinfo types are not sealable
pydantic models tagged class name and fields (recursively); private attributes (_x) are not sealed
anything else FlowRunner(seal_encoder=fn): fn(value) returns a sealable stand-in

A flow that never pauses may hold anything. When a run would pause, everything the checkpoint needs is sealed first; if any part cannot be, the pause fails closed with FlowError (state-not-sealable, naming the path):

Part Check
gate the state and the gate payload are checked before the port is asked: no record is made
tool approval every pending record the paused visits made (concurrent calls, Map items) is denied and consumed in the store its port reaches (the runner's executor or the node's own); with no store, an ERROR log names the approval id and an approval.decided event (withdrawn=False) is emitted
sibling updates, Map item values not dropped: they fail the pause the same way (nothing runs again)
call results a result that cannot be journaled leaves a marker (the call ran); the pause fails naming the call. Pass seal_encoder and seal_decoder to replay custom results

Storing checkpoints

Form Use
in memory pass the checkpoint back to resume
runner.dump_checkpoint(cp) / runner.load_checkpoint(flow, text) JSON text for any storage; the MAC and graph are checked before anything is built; custom values need seal_decoder=fn(type_name, value) to load
pickle trusted storage only: unpickling runs code before the MAC is checked

Loading builds only the classes the state model's field annotations reach (through Optional, unions, lists, dicts, tuples and nested models) and their subclasses already loaded (a Dog in an Animal field; a bare BaseModel or Enum annotation opens nothing), plus ApprovalRequest and ApprovalDecision; a replayed result may also hold classes its tool's return annotation reaches. Any other class name goes to seal_decoder (the value is the decoded fields or enum value), so the decoder must refuse names it does not know; without one it is refused before any object is built, and dump_checkpoint refuses such a checkpoint up front.

Limits

Limit Use instead
a port without a store, resumed in another process or after the memory evicted the entry, can resume twice DeferredApproval (its store refuses)
pickled checkpoints from untrusted storage dump_checkpoint / load_checkpoint
checkpoints grow with journaled results (measured: 200 results of 10 KB, about 2 MB as pickle or JSON) keep large data out of tool results, or in external storage
custom types in state or results seal_encoder (and seal_decoder to load JSON or replay them)