What are best practices for designing node boundaries?

What you need to know

The test for a boundary

Ask: "Would I ever want to…"

• retry this step alone? (a flaky API call)
• resume from just after it? (an expensive model call)
• pause before it? (a refund, an email)
• see it separately in a stream or trace? (a "searching…" status)
• cache its result? (an embedding lookup)

If yes, it is a node.

Rules that follow

• Name nodes with verbs: retrieve, grade_documents, call_model, issue_refund.
• Keep model calls, tool calls and database writes in separate nodes, so a retry repeats only the failing part.
• Put irreversible actions in their own node, after any interrupt(), with an idempotency key.
• Have the node write a decision field (is_valid, route); let the edge read it.

Why re-runs happen

When a run resumes after an interrupt or a failure, completed nodes are skipped because their writes are in the checkpoint, but the interrupted or failed node starts again from its first line. Anything it did before the failure happens twice.

A real-life example

A refund agent's first version had one node, process_refund, that looked up the order, asked a human for approval with interrupt(), then called the payment gateway. It also sent a "we're reviewing your refund" SMS at the top. On resume, the node re-ran from the start and the customer got the SMS twice — 1,800 duplicate SMS in the first week.

The fix split it into lookup_order → notify_customer → approve → issue_refund. Now the SMS node has already completed before the pause, so it is not repeated, and issue_refund passes idempotency_key=f"refund-{order_id}" to the gateway.

Follow-up questions to expect

• "How many nodes is too many?" — When nodes do trivial transformations and the trace is mostly noise. Merge pure steps; split only where a boundary buys retry, resume, pause or visibility.
• "Should a node call an LLM and a tool?" — Better split. The agent node decides, a tool node executes, so a tool failure does not repeat the paid model call.
• "Where does validation go?" — In its own node or at the end of the producing node, writing an error field that an edge routes on.