Models & embeddings

Kaizen does not host models. It routes to Kamiwaza deployments through its own gateway, which owns routing, policy, budgets, effort, retries, and response normalization.

Roles, not names

Models are assigned to roles rather than wired into features:

Role	Used by	Required
Chat	Every agent turn	Yes
Embedding	Query embedding and ingestion	Yes, for semantic retrieval
Reranker	Reordering ACL-visible document evidence	No

Several internal settings default to platform-selected, meaning Kaizen takes whatever the platform has assigned to that role rather than pinning a name that drifts out of date.

Assign roles from Admin area → Models, which writes the role assignment through the ops API.

The selected deployment and save status are visible there. Kaizen keeps the revision used to detect concurrent updates internal; it does not show a version number beside the model selector.

Interactive chat resolves that configured role against the member's current, authenticated platform inventory. If the configured chat deployment is absent but another authorized chat deployment is live, Kaizen uses the first live target and reports it as the effective default. An explicit model choice still fails closed when that exact deployment is unavailable. Credential-free background work also remains pinned to the complete server-owned target; it never selects a different model from a member inventory.

Context window for local chat models

Deploy a chat model served on the platform itself, for example through llama.cpp, with a context window of at least 32,768 tokens. Kaizen does not enforce this floor; it is the size at which agent turns proved reliable. Kaizen's system prompt and tool definitions take a large share of a smaller window, and a validation install at 16,384 tokens answered with a single token. Set the window when you deploy the model: on the platform's Models page, choose Deploy, open the configuration, and set ctx_size to 32768 or more for a llama.cpp deployment.

Kaizen sizes each prompt from the window the deployment declares. A local deployment that declares none is assumed to have KAMIWAZA_MODEL_CONTEXT_DEFAULT_WINDOW_TOKENS (128,000 by default), so Kaizen can send far more than a small ctx_size holds. Either declare the real window or add a KAIZEN_MODEL_CONTEXT_WINDOW_OVERRIDES entry; see the input budget under Provider capability boundaries. Hosted providers, such as Bedrock-hosted models, declare their own large windows and need nothing here.

After a model is redeployed

Kaizen saves the chat selection against the platform deployment it was made for. Stopping and deploying the model again on the platform creates a new deployment, and the saved selection is not moved to it:

• Admin area → Models keeps reporting the chat model as configured, but its selector cannot match the old deployment and usually shows Not configured;
• interactive chat falls back to the first live chat deployment the member can use, as described above, except for a member who explicitly chose the old deployment: that choice fails closed until they pick a live model;
• background work, such as Tasks, stays pinned to the old deployment and fails until the selection is made again.

After any redeploy, open Admin area → Models, choose the redeployed model as the Default chat model, and save. The embedding selection is saved the same way, so check it too after redeploying the embedding model.

Embedding dimensions

Kaizen's semantic index stores 768-dimensional vectors. An embedding model that returns fewer than 768 dimensions works: its vectors are zero-padded to 768, which leaves cosine, inner-product, and Euclidean distances between them unchanged. The platform-provisioned llama.cpp all-MiniLM-L6-v2 (384 dimensions) is therefore a valid embedding model, including on an air-gapped installation. (Kaizen's own OpenRouter-hosted MiniLM registration needs the internet and is not the one to choose offline.)

A model that returns more than 768 dimensions can still be selected and saved, but every embedding call then fails: the gateway records an embedding_shape error in the LLM audit, documents are not semantically indexed, and semantic search degrades to keyword search. Choose a different model rather than retrying.

Vectors from different models are not comparable, so changing the embedding model means re-embedding. Connector content re-embeds on its next sync, because the embedding fingerprint includes the deployment. The same happens after re-selecting a redeployed embedding model, even though the model itself is unchanged. Uploaded documents are different: an upload keeps the vectors of the model that indexed it, and Index does not re-embed a document that is already indexed. To move an upload to the new model, its owner removes the file and uploads it again; see When to reindex. Until then, semantic retrieval compares new query vectors with the old model's document vectors and quietly gets worse. Expect a bulk embedding load on the new model either way.

The embedding trap

This is the most common misconfiguration in a new deployment, so it is worth stating directly.

Pointing the embedding role at a deployment that does not serve an embeddings endpoint produces:

• ingestion failures — documents reach extraction and then fail to index;
• 404 responses on the embeddings path in worker logs;
• search degradation reported as "embedding failed; hybrid unavailable".

Kaizen fails closed rather than substituting a different deployment, so the symptom is honest but the cause is one level removed. Choose a deployment that actually serves embeddings, not a general chat deployment that happens to be healthy.

Diagnose by correlating an ingest failure event with a 404 on the embeddings path, and confirm the selected target's declared capabilities.

After upgrading, the embedding address is refreshed automatically. Kaizen stores the selected embedding target's address when an administrator picks it. An earlier release could store an address built from the platform's public domain (for example http://llamacpp-<id>.<public-domain>:8080/v1), which does not resolve inside the cluster and produces the same "embedding failed; hybrid unavailable" symptom. The first time Kaizen resolves the configured embedding deployment for a signed-in member after the upgrade (a search or a connector sync), it checks the stored address.

Only one shape of stored address is eligible for the refresh: the earlier release's form, http://<instance>.<domain>:<port>/v1. The instance name and port must match the instance the platform advertises now, and <domain> must not be a Service namespace (<namespace>.svc...). The domain does not have to match the current platform domain, so the refresh also works after the platform was moved to a new domain. Kaizen replaces the address only with that instance's in-cluster Service address, http://<instance>.<core-namespace>.svc.<cluster-domain>:<port>/v1. It saves the new address once, records the change as a configuration change by system:embedding-address-refresh, and logs embedding target address refreshed.

An address that was set by hand in exactly that form, using the instance name and port, plain http and the path /v1 under a non-Service domain, cannot be told apart from the earlier release's form. It is treated as stale and rewritten. Each Kaizen process re-checks the stored address at most every five minutes, so if an older pod or a restored setting writes the earlier form back, it is repaired again within that time.

Every other stored address is kept as it is. That includes:

• an address that is already a Service address;
• a bare instance name with no domain;
• an address set by hand through the settings API with a different host name, port, scheme or path;
• an external embedding provider;
• a target set in the environment.

Kaizen also keeps the stored address in these cases:

• the deployment is restarting and advertises no instance;
• the deployment is gone;
• the platform cannot be reached.

When a stored address has the earlier release's form but the in-cluster Service namespace cannot be determined (for example KAMIWAZA_INTERNAL_API_URL is unset), Kaizen keeps the address and logs a warning. The warning names the stored address and the manual repair below, and is repeated at most every five minutes. If saving the new address fails, Kaizen logs a warning and that process does not try again for five minutes. A document upload never triggers the refresh, so on a site where members only upload documents, use the manual repair below.

If the symptom persists after members have searched, repair the target by hand: open Admin area → Models, set the embedding model to Not configured and save, then select the embedding model again and save. Embedding is unavailable between the two saves.

How Kaizen decides what a deployment is

Kaizen classifies a deployment from the platform's structured deployment metadata — its declared type, pipeline tag, serving engine, and container — not from the capability strings it advertises. Kamiwaza labels many embedding deployments chat.completions, so a picker that believed the labels would hide working embedding models and offer chat models for the embedding role.

The practical consequence: a deployment that Kaizen has classified as an embedding model is used as one even when its advertised capabilities say otherwise. The capability strings remain the fallback for a target Kaizen could not classify, and that fallback still fails closed.

This is a classification of what the deployment is, not a claim that it works. An embedding role pointed at something that does not serve embeddings still fails at the first call, with the symptoms above.

Provider capability boundaries

Provider profiles carry declared capabilities, and Kaizen refuses to use a model for something its profile does not declare. A capability mismatch is refused before the call rather than discovered in a malformed response.

Pi uses its generated model catalog for provider compatibility. Output capacity comes from the concrete deployment's authenticated metadata when it declares max_output_tokens, max_completion_tokens, or max_tokens. Kaizen persists that authority with the accepted turn and intersects it with the live PI_MODEL_MAX_OUTPUT_TOKENS operator ceiling (65,536 by default) and the PI_MODEL_MAX_OUTPUT_CONTEXT_SHARE policy (0.25 by default) of the resolved context window. The same effective ceiling controls prompt admission, artifact append sizing, reasoning bounds, and the Pi provider request, which carries it in the max_tokens field the platform forwards to every provider; a request that named it only as max_completion_tokens would run under the provider's own default ceiling instead. Missing metadata is never guessed from a model name. Edit both output controls under Admin → Configuration → Models & roles.

A turn's input budget is the resolved context window less the reserved output and CHAT_CONTEXT_SAFETY_MARGIN_TOKENS. The window is resolved per request, in this order: the selected deployment's declared context window, then KAIZEN_MODEL_CONTEXT_WINDOW_OVERRIDES, then KAMIWAZA_MODEL_CONTEXT_DEFAULT_WINDOW_TOKENS. A prompt measured above that budget is refused before it is sent — the internal model route answers 413 context_length_exceeded naming the prompt size and the budget — rather than being sent, rejected upstream and retried with the same bytes. A deployment whose serving layer declares no window falls back to the default and logs that it did; a run of refusals on one model usually means the declaration is missing, not that members are writing long prompts.

For deployments whose authenticated metadata declares an exact reviewed reasoning-budget parameter, Kaizen maps effort to a bounded token budget and keeps PI_MODEL_ANSWER_RESERVE_TOKENS (4,096 by default) available for prose or a tool call. Structured llama.cpp metadata selects thinking_budget_tokens; structured vLLM metadata selects thinking_token_budget. An authenticated exact feature declaration can expose the same fields. If the platform cannot yet publish an explicit parameter, an operator may bind the exact canonical provider model ID with KAIZEN_MODEL_REASONING_BUDGET_OVERRIDES. An explicit platform parameter or feature still wins; the exact operator declaration overrides only generic serving-engine inference. Name-only, family-level, missing, or conflicting declarations leave both fields unset. The selected field and token count are manifest-bound before the sidecar can make a request.

Reasoning depth and reasoning budget are separate contracts. Kaizen's platform proxy uses its universal OpenAI reasoning field by default while retaining Pi's cataloged model limits and behavior. A deployment whose upstream server requires a different reviewed wire shape may declare reasoning_transport or thinking_format in authenticated inventory metadata. When the platform cannot publish that field yet, an operator can bind the exact canonical model ID with KAIZEN_MODEL_REASONING_TRANSPORT_OVERRIDES. A reasoning_transport.* token already present in the deployment's inventory capability list counts as authenticated metadata too. Authenticated sources that name different transports leave the deployment undeclared rather than falling back to the override, so Pi's catalog and conservative defaults apply. Kaizen binds the resolved declaration in the immutable run manifest. This lets a llama.cpp chat template receive its selected effort through template arguments while ordinary OpenAI-compatible models retain the flat effort field. A display name alone never selects a wire format; the one name-based rule is the Qwen3.8 catalog match below, which selects qwen_vllm.

Effort values must also be ones the deployment accepts. When Pi's catalog has no entry under a deployment's exact name, Kaizen tries the name's last path segment without a trailing serving quantisation marker (-NVFP4, -FP8, -AWQ, -GGUF, -INT4, -INT8), so a Kamiwaza route such as kamiwaza/<host>/Qwen3.8-Flash-Next-NVFP4 is recognised as a reasoning model. Only full-name equality counts, and only a Qwen3.8 match is used: it lends the deployment the fact that it reasons and the Qwen effort vocabulary below — never another vendor's limits, modalities or prices. Any other name match changes nothing, so such a deployment behaves exactly as its declaration says. Deployments declaring a Qwen transport receive Qwen's effort vocabulary: minimal and low send low, medium sends medium, and high and xhigh send xhigh. The transports differ in where that value goes and how none turns thinking off:

• qwen_vllm (Qwen on an OpenAI-compatible vLLM route) sends the flat reasoning_effort field, and none sends reasoning_effort: "none". A Qwen3.8 deployment with no declared transport, matched exactly or by its route name, uses qwen_vllm. A declared transport, openai included, always wins. qwen_vllm sends the system prompt as system unless the deployment declares developer-role support.
• qwen (DashScope's shape) sends a top-level enable_thinking with the effort, and none sends enable_thinking: false with no effort. vLLM ignores a top-level enable_thinking, so use qwen_vllm there.
• qwen_chat_template sends both through chat_template_kwargs, and none sends enable_thinking: false.

A deployment Kaizen does not recognise as reasoning still receives no reasoning field at all.

If a deployment rejects an effort and names the values it supports — for example "Supported types are xhigh (default), medium, and low" — the model proxy relays only those effort names, and the turn retries once with the nearest supported effort, preferring a deeper one. Later steps of the same turn send the substitute directly. Any other rejection fails the turn as before.

Whether an endpoint accepts an OpenAI developer-role message is a separate, independently declarable capability, because a serving engine can reject that role without also needing a Qwen chat template's kwargs. Authenticated inventory metadata may declare supports_developer_role; when it cannot yet, an operator may bind the exact canonical provider model ID with KAIZEN_MODEL_DEVELOPER_ROLE_OVERRIDES (values supported or unsupported). A deployment declaring the reviewed qwen_chat_template or qwen_vllm transport (or a Qwen3.8 deployment that uses qwen_vllm) keeps the developer role rejected by default — this is what keeps every Qwen chat template variant, not just one exact model ID, from failing generation with "Unexpected message role" — but an explicit declaration can override that default in either direction. A developer_role.* token already present in the inventory capability list counts as authenticated metadata and also wins over the override. Authenticated sources that contradict each other resolve to unsupported: a system message is accepted everywhere, so that is the safe reading of an ambiguous signal.

The declaration takes effect only when the model reasons, because that is the only case in which the developer role would be sent at all. Kaizen treats a model as reasoning when its catalog profile says so, or else when the platform declares it or its name matches a Qwen3.8 catalog entry. On a non-reasoning deployment an entry is accepted and has no effect, so declaring it can never break a turn. This differs from KAIZEN_MODEL_REASONING_TRANSPORT_OVERRIDES, whose entry declares reasoning by itself.

The same precedence applies to context capacity. Authenticated inventory wins; KAIZEN_MODEL_CONTEXT_WINDOW_OVERRIDES supplies an exact canonical-ID fallback before Kaizen uses its conservative platform default. This resolved capacity is passed into Pi so prompt admission, compaction, generation capacity, and the member's context meter use one bound.

The Pi adapter accepts a clean proxy stream that ends without a non-null finish_reason; it infers a normal answer or tool-use stop from the completed content. Explicit provider error reasons still fail the turn. Arbitrary OpenAI-compatible sampling fields and deferred provider responses are not accepted from member input or guessed from model names. A response that reaches its effective output ceiling without text or a tool call receives one semantic continuation at the same reasoning effort. If that also ends empty, output exhaustion is reported honestly; reducing reasoning effort or shortening the request leaves more answer capacity. A length-truncated tool batch is never executed, because any of its arguments may be incomplete; the model sees ordered error results and may reissue complete calls on its next bounded step.

Profile verification is cached for MODEL_PROFILE_VERIFICATION_TTL_HOURS (168) so capability checks do not cost a round trip per turn. After changing a deployment's capabilities, expect up to that long before Kaizen re-verifies — or restart to force it.

Bedrock-hosted Claude deployments

Two behaviors of Claude models served through AWS Bedrock reach members unchanged, because the platform proxy and Kaizen pass provider timing and reasoning through rather than reshaping them. Neither is a fault to troubleshoot, but both change what a member can observe.

Bedrock delivers a tool call's arguments as one burst when the call is complete. Measured directly against the Bedrock streaming API, Sonnet 5 returned 191 argument deltas with one at 2.0 s and the remaining 190 at 6.0 s, and Haiku 4.5 returned 188 with one at 0.5 s and 187 at 7.5 s, while ordinary text from the same models streamed normally. An artifact the assistant creates or updates therefore opens and completes within about a second of each other: the live source stream in the pane is short, Stop pressed while it is building lands after the tool has already finished, so the version completes rather than being retained as a partial one to resume, and no partially generated version is ever visible in the history. Nothing is lost — the completed content is what the model wrote — but the interrupted-and-resume path described under Artifacts does not arise on these deployments. The same withholding means no tool content is checkpointed while a long tool call is generated, which can take a minute or more for a large update. Kaizen keeps the live status moving during that window: the chat activity line continues to update, and a task step's Run trace row shows the time of its latest activity. The call's content then arrives all at once when it completes.

Bedrock-hosted Sonnet 5 uses adaptive thinking and returns a thinking block with no text and zero reasoning tokens at every effort, so members never see a Thinking panel for its turns, and the retained history records no reasoning for them. Bedrock-hosted Haiku 4.5 does return reasoning text at higher effort. Release acceptance probes each deployment for both behaviors before the browser lane and reports the affected checkpoints as not applicable rather than failed.

Vision declarations and compatibility

Kaizen resolves whether a selected chat deployment accepts images from the authenticated platform inventory; a member request can never turn it on. The sources are consulted in this order, and the first one that answers wins:

1. An explicit supports_vision, vision, or multimodal value on the deployment or in its metadata or model_info object. A JSON boolean, the integers 1 and 0, or the strings true, yes, 1, false, no, and 0 (trimmed, any case) count as a declaration; any other value is ignored. An explicit false is authoritative and overrides everything below. If explicit values disagree with each other, Kaizen treats the deployment as text-only rather than letting field order decide.
2. A vision entry in the deployment's features list — the Kamiwaza field that also carries tools, structured_output, and reasoning. It applies whatever the model's name or family. Current Kamiwaza releases define this field but leave it empty, so until the platform populates it the sources below decide for platform-registered deployments.
3. An image, vision, or multimodal marker in the deployment's listed capabilities.
4. For older inventories that expose none of the above, a bounded set of established Claude, GPT, Gemini, Qwen-VL, Pixtral, LLaVA, Llama Vision, and MiniCPM-V identifiers, matched against platform-supplied deployment identity only.

Sources 2 through 4 can only grant vision. A missing, empty, or malformed features value, or one that names only other features, declares nothing and falls through to the next source. To mark a deployment text-only despite its name, set an explicit false. When nothing grants vision the deployment is text-only, and the bytes of an attached image are never sent to it.

A turn that carries no selected platform deployment — only an older internal caller still does this — falls back to the supports_vision flag the gateway reports for the model assigned to the tool-iteration role, and does not apply the precedence above.

The attached-image release journey uploads a known-color PNG and requires the selected deployed chat model to answer from its actual pixels. If a newly selected model cannot pass that journey, correct its platform capability metadata or select a working vision deployment before release.

Live compatibility matrix

Before a customer release, run the deployed compatibility matrix against a fixed snapshot of every chat deployment visible to the authenticated UAT member. The matrix covers all member effort levels and Plain-speak both off and on. Models that do not expose reasoning or native streaming remain in the matrix; the shared gateway and runtime must preserve one durable answer, accurate usage, and bounded model-call behavior for them too.

Kaizen does not bootstrap providers or deploy models during this test. An operator must first make every intended target healthy and visible through the platform inventory. Keep deployment credentials and private endpoints in the platform's secret boundary, never in the UAT command or report. The runner requires at least two distinct deployments, restores the member's original Plain-speak preference, and archives all run-labeled conversations even when a case fails.

The matrix exercises generative chat behavior; it does not certify a model's willingness to faithfully recite server-authored facts, because Kaizen no longer depends on that for questions with a known answer. A member asking about their own compression settings always gets a server-rendered answer built from the same values Kaizen tracks internally, not the model's free-text recitation of them — so this class of question has no minimum model-capability bar and passing the matrix is sufficient.

Reranking

Selecting a reranker deployment lets Kaizen reorder ACL-visible document evidence after fusion.

With no reranker selected, Kaizen keeps the deterministic fused order and makes no reranking call at all. There is no silent default, and no hidden cost.

Reranking operates only on authorized survivors. It cannot introduce a document the fusion stage did not already authorize.

Effort

LLM_DEFAULT_EFFORT (default medium) sets the deployment-wide default. Members can set their own default and override per conversation.

Effort is applied by the gateway, so it behaves identically for interactive chat, delegated sub-agents, prompt chains, and scheduled task runs.

Budgets

Setting	Default	Bound
LLM_PER_USER_DAILY_TOKENS	0 (unlimited)	Optional per-member daily cap
LLM_LATENCY_BUDGET_SECONDS	5.0	Assist path
AGENT_LOOP_MAX_CYCLES	10	General tool-bearing model rounds per turn; staged creation adds a separate 52-round bounded protocol allowance
AGENT_LOOP_BUDGET_MS	120000	Maximum inactivity between healthy model or tool events

The default has no per-member daily ceiling. Set a positive LLM_PER_USER_DAILY_TOKENS before a pilot if model cost is a constraint. Per-turn and per-run bounds remain in effect.

Two embedding latency budgets

Deliberately separate:

Path	Setting	Default
Interactive query embedding	SEARCH_EMBED_LATENCY_BUDGET_SECONDS	5.0
Bulk ingestion	INGEST_EMBED_LATENCY_BUDGET_SECONDS	120.0

Keeping them apart is what stops a large reindex from making interactive search feel slow. If you raise one, do not assume the other should follow.

How embedding calls reach Core

When the embedding role is a platform deployment, Kaizen embeds through Core's batch embedding API with the signed-in member's own authorization. Core injects two addresses for itself: KAMIWAZA_API_URL, the platform's public origin, and KAMIWAZA_INTERNAL_API_URL, Core's in-cluster Service. Kaizen sends embedding calls to the in-cluster Service whenever KAMIWAZA_INTERNAL_API_URL names one (a host of the form <service>.<namespace>.svc...). Core checks the member's authorization the same way on either address. Otherwise Kaizen uses the public origin.

The in-cluster address is plain http at the application layer, and Kaizen sends only the member's bearer on it (never the session cookie). On a standard 1.3 install the hop is still encrypted: Kaizen's pods run the Istio sidecar, and Core's Ray head has a STRICT mTLS PeerAuthentication, so it accepts only mesh mTLS and refuses a plaintext connection. The extension's egress policy must allow Kaizen's pods to reach that Service and port (Core's default is core-raycluster-head-svc.<namespace>.svc.cluster.local:7777).

If the Service fails at the connection or TLS layer (it is unreachable, the connection times out, or an extension deployed without the mesh is refused by Core's mTLS), Kaizen retries that batch once on the public origin. If the public origin answers, Kaizen uses it for the next five minutes and logs one warning that names the cause; if both fail, the next call tries the Service first again. An HTTP answer from the Service, such as 401, is not retried elsewhere.

Every model call Kaizen makes from the server, embedding included, verifies the platform's certificate against the trust bundle the platform mounts in SSL_CERT_FILE, unless KAMIWAZA_VERIFY_SSL=false. That setting relaxes verification only for the platform's own origins, matched by scheme, host and port (KAMIWAZA_API_URL, KAMIWAZA_INTERNAL_API_URL, KAMIWAZA_ORIGIN and the platform gateway URLs). Any other origin, including an external embedding provider configured with its own API key, is always verified. Proxy routing (HTTPS_PROXY, NO_PROXY) is the same either way.

When embedding fails, search still answers, but lexically. Kaizen logs one warning per search surface and cause at most every five minutes:

embedding failed; hybrid unavailable: connector_search continued without hybrid retrieval (cause=tls_verify_failed, ...)

The surface is connector_search, uploaded_document_search or conversation_history_search. The cause is one of tls_verify_failed, tls_error, connect_error, timeout, unauthorized, forbidden, not_found or provider_unavailable. The warning never includes a URL, header or provider message. A document whose dense indexing fails stays searchable lexically, shows Semantic indexing unavailable, and logs document.ingest.semantic_degraded (during ingestion) or document.semantic_index.degraded (during Index) with the same cause field.

Capacity planning

Plan model capacity for:

• Peak concurrent turns × the ordinary general-round ceiling, with the additional staged-artifact protocol allowance included for creation-heavy traffic.
• Interactive embedding at query rate.
• Bulk embedding during ingestion and reindex — bursty and heavy.
• Reranking, if selected, at result-set size per query.

Embedding is usually the surprise: a full reindex of a large corpus generates far more embedding calls than steady-state chat generates chat calls.

Prompt caching

PROMPT_CACHE_ENABLED is off by default, and it is the master switch: speculative warming needs it too, so both are inert until you turn it on. Caching is a cost optimisation whose failure mode is a wrong answer rather than a slow one, so an installation opts in where somebody is watching the result. When it is on, caching is applied by the gateway, never crosses members and never caches authority.

Failure behavior

Condition	Behavior
Provider error	Normalized into a safe category, including errors inside HTTP-200 streams; raw errors never reach members, audit, or traces
Redirect or non-model response	Refused at the authenticated proxy as model unavailable; Kaizen never follows a sign-in redirect or passes its HTML into the agent runtime
Provider billing or credit rejection	Reported as an actionable billing/credit failure only when the authenticated model route confirms HTTP 402; raw account details remain hidden
Capability mismatch	Refused before the call
Budget exhausted	Policy outcome — one forced synthesis step, not a crash
Role unavailable	Fails closed. No silent substitution

info

A repeated sign-in redirect from server-side model traffic is a transport configuration problem, not a model retry condition. Current platforms inject KAMIWAZA_PLATFORM_GATEWAY_URL from immutable policy, and TASKS_PLATFORM_GATEWAY_URL overrides it when that standard setting is unavailable. The chain resolves in that order and then falls back to KAMIWAZA_ORIGIN, so leaving both empty does not mean "no gateway": it means the deployment's own public origin, which a pod normally cannot reach, and in a cluster that is the configuration that makes model calls hang until they fail as model_unavailable. Kaizen logs platform_model_route_is_public_origin at boot on every service when the chain resolves that way inside Kubernetes. Kaizen keeps authorization and delegated proofs bound to the registered HTTPS model authority even when bytes travel over the internal route. A direct Core service is not a gateway transport.

Unattended Tasks require Core's guarded delegated inference API. Kaizen derives that API from the authenticated deployment's public origin and deployment ID, then uses the same internal gateway transport. Keep Core and Kaizen builds compatible: a host model may serve interactive requests and draft tests while production Tasks fail if Core lacks the guarded route. Changing the model's ingress to accept delegated credentials directly would bypass effect accounting and is not a supported workaround.

Some OpenAI-compatible routes wrap a failure in an HTTP 200 event stream. If that happens before any model output, Kaizen retries the same authenticated model request once through the non-streaming transport. A successful response is bridged back into the normal stream; an HTTP 402 becomes the billing/credit message above. Kaizen does not retry after output starts, and it does not infer a billing problem from an unknown provider error.

Changing a model safely

1. Confirm the new deployment's declared capabilities match the role.
2. For embeddings, confirm it actually serves an embeddings endpoint.
3. Change the role assignment.
4. Verify: a chat turn completes with citations; a new upload reaches indexed state; search returns semantic matches, not only lexical ones.
5. Watch worker logs for embedding failures over the next ingest cycle.

Changing the embedding model invalidates existing embeddings. Vectors from a different model are not comparable. Plan a reindex, and expect the bulk embedding load that comes with it. Connector content re-embeds on its next sync, but uploads keep their old vectors until each owner re-uploads them; see Embedding dimensions.

Related

• Model gateway
• Ingestion & indexing
• Troubleshooting