Chat and engine
Chat completion headers, orchestrator vs central point, and engine prompt helpers.
Intended audience: Stakeholders, Business analysts, Solution architects, Developers, Testers
Learning outcomes by role
Stakeholders
- Summarize chat as the primary user-facing workload through Cadence HTTP APIs.
Business analysts
- Specify headers (including X-ORG-ID) and orchestrator selection rules for stories.
Solution architects
- Relate streaming responses, engine layer, and rate limits to client design.
Developers
- Follow chat routes and engine helpers under cadence.api.chat and related modules.
Testers
- Exercise streaming and non-streaming paths, errors, and org scoping.
Chat sends a user message to a chosen AI orchestrator instance or central point inside an organization-scoped request. Always send which org and which instance via headers; expect 400 if headers are incomplete. Implementation: cadence/api/chat/router.py (/api/chat/*).
Chat API
Section titled “Chat API”Chat API prefix /api/chat.
Headers
Section titled “Headers”| Header | Required | Role |
| ----------------------------------------- | ------------------------------------------ | ----------------------------------------------------------------- |
| X-ORG-ID | Yes (for completion + conversation routes) | Tenant scope; combined with org_context. |
| X-INSTANCE-ID or X-CENTRAL-ID | For POST /completion | Select orchestrator instance vs central point alias. |
| Auth | Bearer / API key | CHAT_USE / CHAT_HISTORY_READ via roles_allowed. |
POST /api/chat/completion — If both instance and central headers are missing → 400. Implementation branches to _handle_instance_chat or _handle_central_point_chat, then OrchestratorService with rate limits and quotas enforced in the domain layer. Note: the chat router does not use a roles_allowed decorator; it calls require_session and session.has_permission(CHAT_USE, org_id) inline. Public central points accept anonymous callers and use ANONYMOUS_USER_ID for user_id.
Conversations
Section titled “Conversations”GET /api/chat/conversations and GET /api/chat/conversations/{id}/messages require CHAT_HISTORY_READ and X-ORG-ID.
Data model
Section titled “Data model”Chat conversations and messages are persisted in PostgreSQL via the ConversationRepository and MessageRepository.
Conversation
Section titled “Conversation”A conversation groups messages for a user interacting with a specific orchestrator instance within an org.
| Field | Type | Notes |
| ------------- | ------------ | ---------------------------------------------------- |
| id | UUID7 PK | Primary key |
| title | string(500) | Nullable; set from first message or client-supplied |
| org_id | UUID7 FK | Owning organization (CASCADE) |
| user_id | UUID7 FK | Authenticated user (CASCADE) |
| instance_id | UUID7 FK | Orchestrator instance (SET NULL on instance delete) |
| created_at | DateTime(tz) | UTC default |
| updated_at | DateTime(tz) | UTC, onupdate |
| created_by | string(255) | Audit, nullable |
| updated_by | string(255) | Audit, nullable |
| is_deleted | boolean | Soft delete |
Message
Section titled “Message”Each message belongs to a conversation and a turn. The Message model docstring
(src/cadence/infra/persistence/postgresql/models.py:711-715) names three roles: user (human), assistant
(bot), and intent (orchestrator pre-LLM metadata, excluded from history reads). The actual storage layer in
MessageRepository._role_from_message produces user, assistant, tool, or "unknown" — the intent role
is documented in the model but is not currently produced by the SDK mapping in
src/cadence/data/messaging/message.py:27-34. History reads in
MessageRepository.get_messages_for_orchestrator filter with role IN ('user', 'assistant'), so tool and
intent (if used) are both excluded.
| Field | Type | Notes |
| -------------- | ------------ | -------------------------------------------------- |
| id | UUID7 PK | Primary key |
| org_id | UUID7 | Denormalized for org-scoped queries |
| conv_id | UUID7 | Conversation reference |
| turn_id | UUID7 | Groups messages in a single turn |
| user_id | UUID7 | |
| instance_id | UUID7 | |
| role | string(20) | user, assistant, tool, or intent |
| content | text | Message content |
| status | string(20) | |
| is_compacted | boolean | Set true when context policy compacts this message |
| source | string(20) | human (default) |
| metadata_ | JSONB | Python attribute metadata_; DB column metadata |
| created_at | DateTime(tz) | UTC; no updated_at (table is partitioned) |
History reads (via GET /api/chat/conversations/{id}/messages) filter to user + assistant roles only.
Message statistics
Section titled “Message statistics”MessageStatsService (cadence/domain/messaging/message_stats.py) aggregates daily and monthly message counts per user, org, and instance. Statistics use Redis as a real-time quota gate with PostgreSQL backfill for cold starts. Access via GET /api/stats/messages/me and GET /api/stats/messages.
See Database schema for the full table reference.
Engine API
Section titled “Engine API”The engine router exposes a single read-only endpoint, GET /api/engine/prompts (gated by
authenticated), that returns the default prompt text for a given (framework, mode) pair. Use the
framework and mode query parameters — for example, ?framework=langgraph&mode=supervisor returns the
supervisor template, and ?framework=langgraph&mode=grounded returns the grounded template. Other
combinations return 422. A sibling GET /api/engine/conversation-tones returns the conversation-tone
catalog used by orchestrator defaults. Treat these endpoints like internal documentation helpers;
harden network access if you expose the API on the public internet.
Limitations
Section titled “Limitations”- Chat paths may treat some callers as anonymous in specific flows; confirm expected behavior in the chat router before writing acceptance tests.
- Streaming (SSE) timing depends on the engine adapter; set client timeouts accordingly.