Build production-grade tool-calling agents by modeling state transitions explicitly with LangGraph and OpenAI function calling instead of brittle prompt loops.
Why LangGraph for tool-calling agents
Simple ReAct loops collapse once you need conditional branching, human-in-the-loop approval, or parallel tool calls. LangGraph forces you to declare the state machine, so cycles, interrupts, and persistence become first-class concerns rather than afterthoughts.
The library compiles your nodes and edges into a runnable graph that can be checkpointed to Postgres or Redis. This matters when an agent must resume after a tool fails at 3 a.m. or when you need to replay an execution for debugging.
For observability, wire the same graph into OpenTelemetry tracing so token usage and tool latency appear alongside your API routes.
Define state and tools
Start with a TypedDict that carries messages, intermediate results, and any flags the graph needs. Keep the schema minimal; adding fields later requires migration of checkpoints.
Register tools as Pydantic models so OpenAI receives strict JSON schemas. Each tool should be idempotent where possible; side effects belong in the tool node, not the LLM call.
Build the agent graph
Use StateGraph to wire an agent node (LLM with bound tools) to a tool executor node. Add an edge from agent to tools when tool_calls exist, then back to agent. Terminate with END when the LLM returns a final answer without tool calls.
from langgraph.graph import StateGraph, END
from langchain_openai import ChatOpenAI
graph = StateGraph(AgentState)
graph.add_node("agent", call_model)
graph.add_node("tools", tool_node)
graph.set_entry_point("agent")
graph.add_conditional_edges("agent", should_continue, {"continue": "tools", "end": END})
graph.add_edge("tools", "agent")
app = graph.compile(checkpointer=MemorySaver())Implement the tool node and error handling
The tool node must catch exceptions, attach error messages to state, and decide whether to retry or surface the failure. Never let an unhandled exception kill the entire graph run; wrap every tool invocation in try/except and return a ToolMessage with the error text.
For external APIs that may return 429s, add exponential backoff inside the tool itself. Record the attempt count in state so the agent can decide to switch tools after N failures.
Persistence and human checkpoints
LangGraph checkpoints let you pause before expensive actions. Store thread_id in your own database so a user can resume the exact conversation later. This pattern pairs naturally with a lightweight API layer that exposes resume endpoints.
Set interrupt_before on the tool node for any action that mutates production data. The graph will stop and return the current state; your frontend can then present an approval UI before you invoke .invoke again with the same thread_id.
Production cost and failure modes
Token spend grows with graph depth. Add a max_iterations counter in state and terminate early with a clear error message. Monitor average tool calls per run; anything above four usually signals prompt or tool-design problems.
At scale, move the graph executor behind a queue. Failed runs should be retried with the same checkpoint rather than starting over. Combine this with idempotent webhook patterns so external events do not duplicate work inside the agent.