Engine internals

LangGraph graph execution pipeline — nodes, routing, state, and tool execution.

Intended audience: Stakeholders, Business analysts, Solution architects, Developers, Testers

Learning outcomes by role

Stakeholders

Understand the multi-step AI pipeline that processes every chat request.

Business analysts

Map graph nodes to business capabilities (routing, planning, execution, synthesis).

Solution architects

Design per-node LLM configurations and understand recursion limits for capacity planning.

Developers

Trace a chat request through the LangGraph supervisor and grounded pipelines.

Testers

Build integration tests that exercise each graph node and routing edge.

Every chat request that reaches the Cadence engine is processed by a LangGraph state graph — a directed acyclic graph of nodes that classify intent, plan tool usage, execute tools, validate results, and synthesize a final response. This page documents the internal execution pipeline for the two production modes: supervisor and grounded.

Summary for stakeholders

Multi-step pipeline — A single chat turn passes through 3–8 nodes before producing a response. Each node can use a different LLM model.
Bounded execution — max_agent_hops and max_tool_rounds prevent runaway loops. The graph always terminates.
Error resilience — Every node routes to an error handler on failure, producing a graceful response instead of a crash.

Business analysis

Node-level cost control — Each node (router, planner, synthesizer, validator) can use a different model. Use cheap models for routing and expensive models for synthesis.
Validation as quality gate — The optional validator node runs a secondary LLM pass to check tool results before the planner proceeds.

Architecture and integration

Supervisor graph topology

The supervisor mode is the primary production pipeline. It implements a classifier-planner-executor pipeline:

flowchart TD
    START((START)) --> ROUTER["router<br/>intent classification"]
    ROUTER -->|tool_intent| PLANNER["planner<br/>tool selection"]
    ROUTER -->|conversational| RESPONDER["responder<br/>direct reply"]
    ROUTER -->|clarify| CLARIFIER["clarifier<br/>ask user"]
    ROUTER -->|error| ERROR["error_handler"]

    PLANNER -->|has_tool_calls| EXECUTOR["executor<br/>ToolNode execution"]
    PLANNER -->|call_synthesizer| SYNTHESIZER["synthesizer<br/>final response"]
    PLANNER -->|error| ERROR

    EXECUTOR -->|validation_on| VALIDATOR["validator<br/>LLM validation"]
    EXECUTOR -->|validation_off| PLANNER
    EXECUTOR -->|error| ERROR

    VALIDATOR -->|valid| PLANNER
    VALIDATOR -->|error| ERROR

    SYNTHESIZER --> END((END))
    RESPONDER --> END
    CLARIFIER --> END
    ERROR --> END

Grounded graph topology

Grounded mode adds a bootstrap node that loads anchor context for a specific resource, and uses a single scoped agent:

flowchart TD
    START((START)) --> BOOTSTRAP["bootstrap<br/>load anchor context"]
    BOOTSTRAP --> ROUTER["router<br/>intent classification"]
    BOOTSTRAP -->|error| ERROR["error_handler"]

    ROUTER -->|tool_intent| PLANNER["planner<br/>tool selection"]
    ROUTER -->|conversational| SYNTHESIZER["synthesizer<br/>direct reply"]
    ROUTER -->|error| ERROR

    PLANNER -->|has_tool_calls| EXECUTOR["executor<br/>ToolNode execution"]
    PLANNER -->|done| SYNTHESIZER
    PLANNER -->|error| ERROR

    EXECUTOR -->|validation_on| VALIDATOR["validator<br/>LLM validation"]
    EXECUTOR -->|validation_off| PLANNER
    EXECUTOR -->|error| ERROR

    VALIDATOR -->|valid| PLANNER
    VALIDATOR -->|error| ERROR

    SYNTHESIZER --> END((END))
    ERROR --> END

Supervisor node details

Router node

The router classifies the user’s intent using a cheap, fast LLM call. It does not receive tool schemas — only the message history and plugin descriptions.

Input: Message history, plugin descriptions
Output: routing_decision — one of tool_intent, conversational, clarify, error
Model: Created from router_node config; uses RoutingDecision structured output schema when the provider supports it
Routing: route_from_router maps the decision to the next node

When enabled_clarification_intent is false in mode_config, the clarifier node is omitted and clarify maps to planner.

Planner node

The planner selects which tools to call. It receives all plugin tools plus a synthetic call_synthesizer tool that signals “stop planning, generate response.”

Input: Message history, all tools (bound via bind_tools)
Output: AI message with tool_calls or a call_synthesizer invocation
Model: Created from planner_node config; tools bound with parallel_tool_calls from mode_config
Routing: route_from_planner — if call_synthesizer was called → synthesizer; if tool calls exist → executor; otherwise → error handler

The planner can loop: after the executor returns tool results, routing sends execution back to the planner for the next round. The loop is bounded by max_tool_rounds.

Executor node

The executor runs the planner’s tool calls using LangGraph’s ToolNode. It handles timeouts and builds structured tool_records for downstream nodes.

Input: AI message with tool_calls
Output: ToolMessage results, tool_results list, incremented tool_rounds
Timeout: node_execution_timeout from settings (default varies per mode)
Routing: route_from_executor — if validation is enabled → validator; otherwise → planner (for next round) or error handler

async def run_executor_core(state, *, tool_node, tool_collector, settings, ...):
    result = await asyncio.wait_for(tool_node.ainvoke(state), timeout=settings.node_execution_timeout)
    tool_results = await build_tool_records(state, tool_messages, tool_collector)
    return {**result, "tool_results": tool_results, "tool_rounds": state.get("tool_rounds", 0) + 1}

Validator node (optional)

The validator runs a secondary LLM pass (at temperature=0.0) to check whether tool results are correct and complete. It outputs a ValidationResponse with is_valid and reason.

Enabled by: enabled_llm_validation = true in mode_config
Input: Tool results from executor
Output: validation_result with is_valid boolean
Routing: route_from_validator — if valid → planner; otherwise → error handler

Synthesizer node

The synthesizer generates the final user-facing response from tool results and conversation history. It is the only node that produces visible output.

Input: Message history, tool results, whoami identity context
Output: Final AI response message, optional follow-up suggestions
Model: Created from synthesizer_node config
Routing: route_from_terminal → END (or error handler on failure)

Responder node

The responder handles conversational and meta queries that don’t require tools (greetings, “who are you?”, etc.).

Input: Message history, whoami identity context
Output: Direct AI response
Model: Reuses the synthesizer model
Routing: route_from_terminal → END

Clarifier node (optional)

The clarifier asks the user for more information when intent is ambiguous.

Enabled by: enabled_clarification_intent = true in mode_config
Output: Clarifying question message, optional suggestions
Routing: route_from_terminal → END

Error handler node

Every node routes to the error handler on failure. It produces a graceful error response using the error model.

Input: error_state with exception details
Output: Error response message
Routing: Always → END

Graph state

The supervisor graph state (SupervisorGraphState) carries all data between nodes (extends BaseGraphState from src/cadence/engine/impl/langgraph/state.py:12-25):

| Field | Type | Purpose | | --------------------- | ---------------------- | ------------------------------------------------------- | | messages | list[AnyMessage] | LangChain message history with add_messages reducer | | thread_id | str \| None | Conversation thread identifier | | user_intent | str \| None | Extracted user intent (set by router node) | | error_state | dict[str, Any] \| None | Error details when a node fails | | user_query | str \| None | Original user message | | user_intent_explain | str \| None | Router’s reasoning for the intent classification | | agent_hops | int \| None | Counter for recursive agent invocations | | current_agent | str \| None | Currently executing agent identifier | | validation_result | dict[str, Any] \| None | Validator output (is_valid, reason) | | routing_decision | str \| None | Router classification result | | language | str \| None | Detected user language | | tool_results | list[dict[str, Any]] \| None | Structured tool execution records | | tool_rounds | int | Counter for tool execution rounds |

The grounded graph state (GroundedGraphState) adds: resource_id, anchor_context, context_loaded, context_was_fresh, scope_rules. The messages field is shared from BaseGraphState.

Per-node LLM configuration

Each node can use a different LLM model via NodeConfig. The mode_config object supports per-node overrides:

{
  "mode_config": {
    "default_llm_config_id": "01936a8f-...",
    "router_node": {
      "llm_config_id": "cheap-model-id",
      "model_name": "gpt-4o-mini"
    },
    "planner_node": {
      "llm_config_id": "smart-model-id",
      "model_name": "gpt-4o"
    },
    "synthesizer_node": {
      "llm_config_id": "smart-model-id",
      "model_name": "gpt-4o"
    }
  }
}

NodeConfig.from_resolved_config extracts the default LLM config, then each node’s merge() applies node-specific overrides. This allows using a fast, cheap model for routing and a powerful model for synthesis.

Recursion limits

The graph recursion limit prevents infinite loops. _get_recursion_limit() is implemented per mode:

Supervisor: max_agent_hops from mode_config (default: 25).
Grounded: max_tool_rounds * 4 + max_agent_hops + 12 (accounts for bootstrap, tool rounds, and planner hops).

When the limit is reached, LangGraph raises GraphRecursionError, which is caught by the base orchestrator and converted to an error response.

Context compaction

Before the graph starts, the base orchestrator compacts message history based on token limits. The apply_context_policy function (from cadence/domain/common/context_policy.py) trims older messages to fit within the configured context window.

When enabled_auto_compact is true, a dedicated auto-compact model summarizes older messages before the graph runs, preserving context without exceeding token limits.

Tool collection

At build time (before the first message), the ToolCollector gathers all tools from loaded plugin bundles:

Each plugin bundle exposes LangChain-compatible tools via bundle.tools
The collector deduplicates and validates tool names
For the planner, a synthetic call_synthesizer tool is appended — calling it signals the planner to stop and route to the synthesizer
For grounded mode, UvTool instances from the SDK are bound to the planner model

Streaming

The LangGraphStreamingWrapper (src/cadence/engine/impl/langgraph/streaming.py) converts LangGraph’s astream(stream_mode=["messages", "custom", "updates"]) into Cadence StreamEvent objects for SSE delivery. The supported event types are defined in StreamEventType (src/cadence/infra/streaming/stream_event.py:14-29):

agent — when a node begins execution (includes node display metadata)
message — text tokens from LLM generation (only for nodes in the _TOKEN_STREAMING_NODES frozenset)
tool — tool execution start/result (when the tool’s stream_tool=True)
metadata — graph-level metadata (agent hops, tool rounds)
suggestion — follow-up suggestions from the synthesizer

error is not in StreamEventType; the chat router emits an event: error SSE line by hand when OrchestratorService.process_chat_stream raises an exception, carrying the CadenceException.code and message (or SY-9000 for unhandled exceptions).

The wire format is event: <type>\ndata: <json>\n\n produced by StreamEvent.to_sse().

Limitations

Google ADK is not registered — google_adk appears in validation patterns (e.g. CreateOrchestratorRequest accepts it and FRAMEWORK_SUPPORTED_PROVIDERS lists providers for it), but OrchestratorFactory._BACKEND_CONFIGS does not register any google_adk backends. All seven google_adk/<mode>/__init__.py files are create_not_implemented_orchestrator(...) placeholders that raise UnsupportedOperationError if invoked. The real Google ADK adapter (google_adk/adapter.py) and streaming wrapper (google_adk/streaming.py) exist on disk but the orchestrator classes are stubs.
FRAMEWORK_SUPPORTED_MODES is incomplete — the constant in core/constants/framework.py lists {supervisor, grounded} for langgraph and empty sets for openai_agents and google_adk. The actual (framework, mode) matrix is in _BACKEND_CONFIGS (langgraph: 6 modes; openai_agents: 7 modes; google_adk: none registered). The validator in domain/orchestrator/validator.py:145-147 uses the constant and would therefore reject openai_agents/supervisor even though the factory has a placeholder for it.
Only supervisor and grounded are real implementations — every other mode in _BACKEND_CONFIGS (coordinator, handoff, pipeline, reflection, ensemble, hierarchical for both langgraph and openai_agents) is a create_not_implemented_orchestrator(...) placeholder.
is_specialized is required for org agents — the inspector at domain/plugins/inspector.py:159-196 rejects bare BaseAgent subclasses; agents must extend BaseSpecializedAgent (for multi-agent topologies) or BaseScopedAgent (for grounded mode).
Single-turn tool execution — The executor runs all tool calls from a single planner turn. Parallel tool calls are supported but sequential tool dependencies require multiple planner rounds.
No mid-graph checkpointing — If the process crashes during graph execution, the conversation state is lost. Messages are only persisted after the graph completes.

Next steps

Orchestration modes Supervisor, grounded, and other mode topologies with mode_config reference.

Chat and engine Chat completion API, headers, and routing to orchestrator instances.

Hot-reload and AI App pool Pool tiers, async load/unload, and instance lifecycle.