LLM configuration

Per-org LLM provider settings and the settings cascade (overview).

Intended audience: Stakeholders, Business analysts, Solution architects, Developers, Testers

Learning outcomes by role

Stakeholders

Relate per-org LLM settings to cost control and data residency narratives.

Business analysts

Document cascade levels (global, org, user) for configuration acceptance tests.

Solution architects

Map provider credentials and secrets handling to enterprise key management.

Developers

Apply TenantSetting keys and LLM config APIs when wiring models.

Testers

Verify fallback and override behavior across cascade tiers.

Each organization can choose which LLM providers and models to use (within tier and platform rules). Settings merge with global defaults and apply when orchestrators run workloads. Validate org-admin versus member access and tier limits on /api/orgs/{org_id}/llm-configs. Merged settings flow through SettingsService and OrganizationLLMConfigRepository at runtime.

Visual: where config flows

flowchart LR
  G[Global defaults] --> M[Merge]
  O[Org LLM configs] --> M
  M --> I[Orchestrator instance]

How to use it (operators)

Ensure the caller has cadence:org:llm-configs:read or :write (see router docstrings for BYOK-style changes).
Use GET/PATCH/... under /api/orgs/{org_id}/llm-configs as documented in OpenAPI for your deployment.
Create or update orchestrator instances so they pick up the merged configuration (see Orchestrator instances).

Key concepts

Term	Meaning
Cascade	Org-level entries override global defaults when the merge rules allow it.
BYOK	Bring-your-own API keys for providers, stored and validated per policy.
Tier	Subscription tier may cap which models or providers an org may use.

Technical details

Permissions: cadence:org:llm-configs:read and :write; the router docstrings describe who may change BYOK-style settings.

For request/tenant context, see Multi-tenancy. For orchestrator validation at create time, see Orchestrator instances.

Orchestrator instances Where LLM config meets framework and pool loading.

Multi-tenancy Organizations, X-ORG-ID, and quotas.

Configuration Environment variables and secrets for the API.