How do you implement dynamic routing based on classifier output or tool results?

What you need to know

1from typing import Literal2from pydantic import BaseModel, Field34class Route(BaseModel):5    intent: Literal["billing", "technical", "cancel", "other"]6    confidence: float = Field(ge=0, le=1)78router_llm = init_chat_model("openai:gpt-4.1-mini").with_structured_output(Route)910def classify(state):11    r = router_llm.invoke(state["messages"])12    return {"intent": r.intent, "confidence": r.confidence}1314def pick(state) -> str:15    if state["confidence"] < 0.6:16        return "clarify"17    return {"billing": "billing_agent", "technical": "tech_agent",18            "cancel": "retention_team"}.get(state["intent"], "general_agent")1920builder.add_conditional_edges("classify", pick,21    ["billing_agent", "tech_agent", "retention_team", "general_agent", "clarify"])

Why this shape

• Constrained output — the Literal becomes an enum in the JSON schema, so free-text labels like "Billing issue" cannot appear.
• Visible decision — intent and confidence are in the checkpoint and the trace.
• Safe default — .get(..., "general_agent") handles a label added later.
• Low-confidence path — clarify asks the user a question instead of guessing.
• Cheap model — the classifier runs on every message; a small model is usually accurate enough and much faster.

Routing on tool results

1def check_payment(state):2    result = payment_api.status(state["txn_id"])   # e.g. {"state": "PENDING"}3    return {"payment_state": result["state"]}45builder.add_conditional_edges("check_payment", lambda s: {6    "SUCCESS": "confirm", "PENDING": "wait_and_retry", "FAILED": "refund"7}.get(s["payment_state"], "human_review"))

A real-life example

A UPI payments app's support bot routes "money debited but not received" messages. The classifier writes intent="failed_txn"; a check_payment node calls the transaction status API; the edge routes PENDING to a node that says "banks settle within 48 hours" and schedules a follow-up, FAILED to an automatic reversal check, and anything unknown to a human. Before this, a single agent prompt tried to handle all cases and told 11% of PENDING users their money was lost. Explicit routing on the API's status removed that class of error.

Follow-up questions to expect

• "How do you evaluate the router?" — A labelled set of a few hundred real messages, measured by accuracy per class and a confusion matrix; re-run it on every prompt or model change.
• "Embeddings or an LLM for routing?" — Embedding similarity is cheaper for many stable intents; an LLM handles nuance and new phrasing better. Some teams use embeddings first and an LLM for uncertain cases.
• "What about multi-intent messages?" — Return a list of labels and route to several nodes in parallel, or handle the primary intent and queue the rest.