Context management & compaction

Two different mechanisms shorten what a model sees. They are frequently confused, so this page separates them completely.

	System safety compaction	Member compression
Purpose	Keep long conversations inside the model window	Save tokens and shorten prose
Control	System-managed automatically; members can request it early with /compact	Member-owned, opt-in, off by default
Can be disabled	No	Yes, by the member
Touches	The model-bound context projection	Eligible tool results and visible prose
Touches the journal	Never	Never

System safety compaction

Compaction is mandatory. Without it, a long conversation would eventually exceed the model's window and fail.

The budget

Setting	Default	Role
CHAT_CONTEXT_BUDGET_ENABLED	on	Master
CHAT_CONTEXT_DEFAULT_WINDOW_TOKENS	8192	Assumed window when unknown
CHAT_CONTEXT_RESERVED_OUTPUT_TOKENS	1024	Held back for the answer
CHAT_CONTEXT_SAFETY_MARGIN_TOKENS	512	Headroom against tokenizer drift
CHAT_CONTEXT_RETRIEVAL_TOKEN_SHARE	0.5	Share retrieval may claim

The thresholds

Setting	Default	Meaning
CONTEXT_COMPACTION_MODE	eager	When to compact
CONTEXT_COMPACTION_SOFT_RATIO	0.60	Start considering
CONTEXT_COMPACTION_EAGER_RATIO	0.78	Compact now
CONTEXT_COMPACTION_TARGET_RATIO	0.50	Compact down to this

Eager mode compacts before a turn is under pressure rather than at the moment of failure, which keeps latency predictable instead of producing one very slow turn at the threshold.

Manual compaction

/compact enters the same owner-scoped compaction boundary before the next automatic threshold. The client accepts it locally, never records it as a chat message, and starts an exclusive durable task through POST /api/conversations/{id}/condense. Progress and the terminal result stream from the task events endpoint. Refresh recovery retains only the task pointer; the server remains authoritative for ownership, accepted journal head, model selection, and lifecycle state. The member surface projects this lifecycle as a compact timeline divider rather than a chat message. Automatic compaction is projected from the answer's durable context_info, so its completed indicator survives reload and is identical for every authorized room viewer.

The task summarizes an owner-authorized snapshot of the canonical conversation journal and appends an immutable checkpoint keyed to that exact journal head. Summary generation uses the gateway's streaming transport so long generations do not depend on a single buffered provider response. Provisional deltas remain inside the compaction worker; only a validated final summary can become a checkpoint or appear in the member timeline. Because no provisional text is published, the worker may replay a stream that ends before returning any content once through the platform's buffered transport. It never replays a stream after content arrives. Both transports use the same server-resolved platform gateway route, model authority, policy, budget, and audit funnel. Future turns receive the checkpoint summary followed by model-visible events after its fold position. Private and collaborative rooms use this same projection; a per-member Pi session is never treated as shared-room memory. Repeated compaction folds the prior checkpoint and only the newer journal messages, keeping work bounded as the room grows.

Manual compaction is rejected while an answer owns the conversation session, for a conversation the actor does not own, or when there is no earlier durable context to fold. A cross-replica room writer lease prevents compaction from interleaving with agent execution. It never edits the conversation journal.

When the model call itself fails, the member sees one generic notice — the chat could not be compacted — because a provider's own wording is never member-facing. The reason is not lost: the gateway's audit row for the failed call is committed before the failure unwinds, so an operator can tell a provider refusal from a timeout, a budget stop, or an unusable summary. Diagnosing one of these from the member's notice alone is not expected; read the audit.

Warm candidates

With CONTEXT_COMPACTION_WARM_CANDIDATES_ENABLED, compaction candidates are prepared ahead of need under a lease (CONTEXT_COMPACTION_WARM_CANDIDATE_LEASE_SECONDS, 900). The work happens off the critical path, so the turn that finally needs a compacted context does not pay for producing it.

What compaction never does

Compaction shortens the projection sent to the model. It never edits or shortens the durable conversation journal.

That separation is what allows aggressive compaction: the record a member, auditor, or replay depends on is untouched, so the only cost of compacting too hard is answer quality, never data loss.

Member compression

Three opt-in tools, all defaulting to off per member, implemented at exactly three shared seams so behavior stays uniform across every agent.

Tool-result compression

The seam is the authorized tool-result boundary in the callback handler. After sandbox execution and after all authorization and policy filtering complete, eligible public run_code stdout and stderr may be filtered before the model-bound copy is produced.

Properties:

• the worker keeps raw command output canonical for audit, presentation, and diagnostics;
• only the model-bound projection is smaller;
• artifact references, diagnostics, source identifiers, exit status, member-facing summaries, and rich presentation data are never rewritten;
• search, conversation-history search, and document retrieval bypass it byte-for-byte;
• unsupported tools, private collaboration output, and output that would not get smaller pass through and count as bypasses;
• failure or timeout returns the original already-authorized result and continues the turn.

Eligibility is deliberately limited to deterministic fields in non-search run_code results. Expanding that catalog to source-bearing evidence would require a separate fidelity specification and release gate.

The compressor runs inside the existing hardened worker at a fixed path and argv, with telemetry, tracking, and tee disabled, using isolated temporary state. It invokes no model and grants no new tool, sandbox, filesystem, network, identity, or approval authority.

Plain-speak responses

The seam is the shared system-prompt composition chain, joined after grounding and temporal-awareness instructions.

The member's effective state is captured once before the run starts, so a change activates only for newly started turns. The skill applies to user-visible thinking and final-answer prose; private reasoning stays complete.

Kaizen bundles a reviewed copy of the upstream skill. It does not run a developer-local skill, a third-party runtime extension, a runtime download, a second model call, or a rewrite after delivery. Composition failure falls back to the normal prompt and continues.

Full clarity overrides terseness for security warnings, destructive-action confirmations, approval prompts, ordered multi-step instructions, and anything where shorter wording would be materially ambiguous.

Prompt trim

The seam is the single shared point where the model-visible copy of the member's message is composed, immediately before document, artifact, and sub-agent context are folded in. The raw message continues to travel as the run's logical message, which is what the durable journal, the transcript, guardrails, audit, retries, and idempotency read; only the provider-bound copy is trimmed.

The pass is a fixed list of commented rules over the text — no model call, no network, no settings lookup inside it, and therefore no added latency. Each rule names what it removes: a request preamble ("I was just wondering if maybe you could possibly"), an apologetic hedge, a filler adverb, and closing courtesy.

Properties:

• fenced code, inline code, deliberately quoted spans, and URLs are never rewritten;
• a rule never fires inside a clause that already carries a negation, because removing a filler word there can invert the sentence;
• a result that is empty, or too small a share of the member's own message, is discarded in favor of the original text;
• an exception in the pass returns the member's message unchanged and the turn continues;
• each pass records one deterministic measurement (eligible and delivered bytes, and whether it applied, bypassed, or failed) through the same owner-bound compression stats recorder as its siblings.

The rule list is the known ceiling: it removes only the phrasing it lists, so unusual hedging survives. Widening it is not the upgrade path — a reviewed model-side rewrite behind this same member switch is, and it would need its own fidelity specification and release gate because it would put a model between the member's words and the prompt.

The settings projection

Turn-start state also produces a bounded, server-authored pipeline settings projection for every served agent. It lists only:

• the member's compression master preference,
• effective Tool-result, Plain-speak, and Prompt trim state, with a safe off-reason,
• always-on system safety compaction.

It is appended after member-authored resource context, so resource text cannot override it, and it contains no identity, credential, hidden prompt, or raw policy data.

Live verification found that a model's willingness to faithfully recite this block is not reliable, even on a model the compatibility matrix certifies — a model can produce a confident, plausible-sounding answer about compression state that does not match the projected values. Kaizen therefore does not trust the model's free-text answer for this question. When a turn's message matches the pipeline-settings vocabulary above, the delivered answer is replaced with one server-rendered directly from the same EffectiveCompressionState that composed the block, before the member ever sees the model's draft (normalize_self_inspection_delivery in kaizen.services.pipeline_self_inspection). The projection above remains useful context for the rest of the turn — routing decisions, follow-up reasoning — but a direct settings-state question always gets the server-rendered answer, not the model's recitation of it.

Availability versus opt-in

MEMBER_COMPRESSION_ENABLED controls whether the member settings surface exists. MEMBER_TOOL_RESULT_COMPRESSION_ENABLED, MEMBER_PLAIN_SPEAK_COMPRESSION_ENABLED, and MEMBER_PROMPT_FLUFF_COMPRESSION_ENABLED control tool availability after their respective release gates.

An availability flag being on does not opt anyone in. Each member remains opted out until they choose otherwise.

Related

• Chat compression — the member view
• Model gateway — where budgets are applied
• Tools & the tool boundary