LangGraph isn't cheaper than LangChain — unless you opt out of its defaults
Cost-audit series, episode 4. This series began with an AI agent that burned 136M tokens overnight →.
When LangChain deprecated ConversationBufferMemory (the subject of episode 1 in this series), the official migration path was LangGraph. The pitch: explicit state management, you control exactly what flows where. More expressive, more controllable.
It is — but only if you reach for the controls. The default state model in LangGraph has the same unbounded-growth problem as the memory it replaced. Teams migrating to escape ConversationBufferMemory's cost curve often land on an identical curve, with new graph complexity on top.
This audit shows exactly where the default grows, what it costs, and what opt-outs exist.
The default: MessagesState + add_messages
The quickstart in LangGraph's own docs uses this pattern:
from langgraph.graph import StateGraph, MessagesState
def my_node(state: MessagesState):
messages = state["messages"]
response = llm.invoke(messages) # sends ALL messages to the LLM
return {"messages": [response]}
graph = StateGraph(MessagesState)
graph.add_node("agent", my_node)
MessagesState is a TypedDict with a single key, messages, backed by the add_messages reducer. Here's what that reducer does:
# langgraph/graph/message.py — add_messages (def at line 18; merge loop below)
def add_messages(left: Messages, right: Messages) -> Messages:
# ... (coerces left/right to lists of BaseMessage) ...
left_idx_by_id = {m.id: i for i, m in enumerate(left)}
merged = left.copy()
ids_to_remove = set()
for m in right:
if (existing_idx := left_idx_by_id.get(m.id)) is not None:
if isinstance(m, RemoveMessage):
ids_to_remove.add(m.id)
else:
merged[existing_idx] = m # same id → update in place
else:
merged.append(m) # new id → APPEND (the list grows)
merged = [m for m in merged if m.id not in ids_to_remove]
return merged
Source: langgraph/graph/message.py
This is not a summarizer, not a window, not a trimmer. It is an append-only list. Every message ever added to state stays in state — and every node that reads state["messages"] sees the full list.
This is ConversationBufferMemory with a graph wrapper.
The cost math
Assume a conversational agent: 10 turns, 150 tokens per user message, 200 tokens per assistant reply (modest — a short answer each time).
After 10 turns, state["messages"] contains 20 messages = (10 × 150) + (10 × 200) = 3,500 tokens of accumulated history.
For the 11th call, the node sends all 3,500 tokens of prior history as context, then generates a new reply. Each further turn adds another 350 tokens (150 user + 200 assistant), so the 12th call sends 3,850, the 13th 4,200, and so on.
Total input tokens for a 20-turn conversation:
| Turn |
Messages in state |
Input tokens (messages + system) |
| 1 |
0 prior |
150 + 400 (system) |
| 5 |
4 prior turns |
1,550 + 400 |
| 10 |
9 prior turns |
3,300 + 400 |
| 15 |
14 prior turns |
5,050 + 400 |
| 20 |
19 prior turns |
6,800 + 400 |
| Total |
|
~77,500 tokens input |
(Each call = 400 system + 150 current user + (turn−1) × 350 accumulated history.)
A naive estimate (flat 550 tokens/call × 20 calls) = 11,000 tokens.
Actual with add_messages default = ~77,500 tokens. 7× over.
With claude-haiku-4-5 ($0.80/M input, $4/M output) for a chatbot doing 500 conversations/day:
-
Naive estimate: 11,000 × 500 × 30 × $0.80/M = $132/month
-
Actual: 77,500 × 500 × 30 × $0.80/M = $930/month
That's $798/month of silent overspend on input tokens alone, just from the default accumulation — before you add nodes, tools, or memory.
Multiplier 1: multi-node graphs (each node pays the full state)
LangGraph's value over a simple chat loop is composing multiple nodes — a router, a tool-caller, a summarizer, a responder. Each node that reads state["messages"] pays the full token cost of the accumulated message list.
graph = StateGraph(MessagesState)
graph.add_node("router", route_node) # reads state["messages"]
graph.add_node("tool_caller", tool_node) # reads state["messages"]
graph.add_node("responder", respond_node) # reads state["messages"]
For a 3-node graph where each node reads messages, a single user turn that passes through all three nodes costs 3× the message-list tokens. After 10 turns with 3,500 accumulated tokens, one user message costs: 3 × 3,500 = 10,500 tokens just for message history, before any node-specific prompts.
Multiplier 2: interrupt_before / interrupt_after (human-in-the-loop)
LangGraph's human-in-the-loop feature pauses graph execution at a node boundary. When the graph resumes, it deserializes the full checkpointed state and re-injects it into the next node:
# langgraph/pregel/__init__.py — Pregel.astream (v0.2.60), the entrypoint
# that drives interruptible execution. Verbatim signature:
async def astream(
self,
input: Union[dict[str, Any], Any],
config: Optional[RunnableConfig] = None,
*,
stream_mode: Optional[Union[StreamMode, list[StreamMode]]] = None,
output_keys: Optional[Union[str, Sequence[str]]] = None,
interrupt_before: Optional[Union[All, Sequence[str]]] = None,
interrupt_after: Optional[Union[All, Sequence[str]]] = None,
debug: Optional[bool] = None,
subgraphs: bool = False,
) -> AsyncIterator[Union[dict[str, Any], Any]]:
...
Source: langgraph/pregel/__init__.py — search the file for async def astream (defined ~line 1683; interrupt_before/interrupt_after are the pause controls). When a run resumes after an interrupt, Pregel reloads the pending state from the checkpointer (the aget_tuple/aget_state path returns the full checkpoint blob — every message included) before continuing at the next node.
The cost: full state deserialization on every resume. If a workflow interrupts 3 times before completion (a common approval flow), and the state has 5,000 tokens of messages, the resumption overhead alone is 3 × 5,000 = 15,000 extra tokens — paid every time, even if the approval is just a "yes."
Multiplier 3: parallel fan-out (Send API)
LangGraph's Send API dispatches parallel subgraph invocations, each receiving a copy of state:
from langgraph.types import Send
def fanout_node(state: MessagesState):
return [
Send("worker_a", {"messages": state["messages"], "task": "summarize"}),
Send("worker_b", {"messages": state["messages"], "task": "critique"}),
Send("worker_c", {"messages": state["messages"], "task": "expand"}),
]
Source: langgraph/types.py
Each Send carries the full state["messages"] to the worker node. With 3 workers and 5,000 tokens of history: 15,000 tokens dispatched in the fan-out alone. If those workers themselves call an LLM, each call pays the full 5,000-token history again. Compare to the CrewAI quadratic problem from episode 3 — this is the same failure mode, different API.
The opt-outs (LangGraph actually provides them)
Unlike ConversationBufferMemory (which had no good trim story), LangGraph ships built-in tools to fix this. Teams just don't use them by default.
Trim messages before every LLM call
from langchain_core.messages import trim_messages
def my_node(state: MessagesState):
trimmed = trim_messages(
state["messages"],
max_tokens=2000, # hard cap
strategy="last", # keep the most recent
token_counter=llm, # use the model's tokenizer
include_system=True, # always keep the system message
allow_partial=False,
)
response = llm.invoke(trimmed) # sends trimmed history, not full list
return {"messages": [response]}
Source: langchain_core/messages/utils.py
This keeps the full history in state (for checkpointing, human inspection) while capping what the LLM actually sees. Applying a 2,000-token cap on a 20-turn conversation reduces input tokens from ~77,500 to ~40,000 (2,000 tokens × 20 calls). ~48% cost reduction, one line change.
Pass only what the node needs
Instead of giving every node the full state["messages"], scope what each node receives:
class MyState(TypedDict):
messages: Annotated[list[BaseMessage], add_messages]
last_tool_result: str # structured, compact
task_description: str # set once, doesn't grow
def tool_caller_node(state: MyState):
# This node only needs the task + last result — not the full conversation
prompt = f"Task: {state['task_description']}\nContext: {state['last_tool_result']}"
response = llm.invoke(prompt)
return {"last_tool_result": response.content}
The tool_caller_node never touches state["messages"] — it pays zero for message accumulation. Only the nodes that genuinely need conversational context receive it.
Summarize periodically (the right way)
def maybe_summarize(state: MessagesState):
messages = state["messages"]
if len(messages) > 20: # threshold: tune to your cost tolerance
summary = llm.invoke([
SystemMessage("Summarize this conversation in 3 sentences."),
*messages,
])
return {
"messages": [
SystemMessage(f"Conversation summary: {summary.content}"),
messages[-2], # keep the last human message
messages[-1], # keep the last assistant message
]
}
return {} # no change needed
This collapses the accumulated history into a single system message at the summarization trigger. After summarization, the effective context is ~200 tokens (summary + last exchange) instead of 3,500+. Insert maybe_summarize as an always-on node before expensive LLM calls.
Measuring your own graph
LangGraph's built-in tracing (via LangSmith) shows per-node token usage, but it's behind a paid tier for production volumes. For a free alternative that works with any JSONL export:
npm install -g @wartzar-bee/tokenscope
npx @wartzar-bee/tokenscope langgraph-session.jsonl
Output shows the session total, how much of each call is re-sent accumulated context versus new work, and where the growth is steepest — the same view that surfaced the 136M-token burn in episode 1.
Summary
LangGraph's MessagesState + add_messages default is ConversationBufferMemory under a new name. The graph model gives you explicit controls that the old memory API lacked — but the controls are opt-in. Without trim_messages, selective state reads, or periodic summarization, migration from LangChain to LangGraph buys you graph expressiveness at the same (or higher, for multi-node graphs) token cost.
| Pattern |
Cost impact |
Fix |
MessagesState default |
7× over naive estimate at 20 turns |
trim_messages before every LLM call |
| Multi-node graph, all nodes read messages |
N× multiplier (N = node count) |
Scope state: only pass what each node needs |
interrupt/resume |
Full state re-injected on every resume |
Summarize before checkpointing at long sessions |
Send fan-out |
Parallel full-state copies |
Pass minimal substate to each worker |
The migration from LangChain to LangGraph is worth it — but only once you understand and explicitly opt out of these defaults. Otherwise you're paying for the complexity without the savings.
wartzar-bee builds tools for operating cost-efficient autonomous agents. tokenscope is free and open-source. Follow on dev.to →
Permalink
