Data model & storage
Three stores, with deliberately different guarantees. Knowing which is which tells you what survives a restart, what is safe to lose, and what has to be backed up.
PostgreSQL — system of record
Everything durable and authoritative.
| Domain | Holds |
|---|---|
| Conversations | Journals, positions, input queue, membership, audience versions, resources |
| Search | Passage and mini-chunk text, vector columns, projection outbox |
| Documents | Upload metadata, extraction status, chunk records |
| Library | Agents and versions, prompts and versions, prompt chains, collections, artifacts and versions |
| Projects | Grouping records, versioned pointers, membership, invitations |
| Tasks | Definitions, timezone-versioned schedule cursors, immutable occurrence ledger, runs, run events, opaque platform grant/run/claim/binding mappings, corpus manifests, partitions, records, provenance and exports |
| Members | Preferences, compression settings and statistics, circle choices and uploaded circle pictures, sub-agent selections |
| Audit | The append-only audit trail |
| Delegated authority | Opaque Core grant/run/claim/effect/resource/credential-binding IDs and revision/fence evidence; no capability material |
| Grants | task_capability_grant — authorization facts, never credentials. task_draft_test_model_grant — the one exception: a safe test's single-use model grant, holding the member's AES-GCM-sealed bearer bound to one test run until the run spends it, when the ciphertext is cleared |
| Capability broker | Source pointers plus one-time replay receipts; never schemas, arguments, results, or credentials |
PostgreSQL is also the schedule authority and the authorization boundary: access predicates live inside retrieval queries rather than in application code above them.
Valkey — ephemeral only
| Use | Lifetime |
|---|---|
| Core-issued opaque run references in the worker queue | Until claimed |
| Sandbox leases | SANDBOX_LEASE_TTL_SECONDS (900) |
| Task run leases | Fenced, heartbeated |
| Presence and typing leases | Seconds |
| "Room head changed" hints | Momentary |
| Embedding cache | EMBEDDING_CACHE_TTL_SECONDS (604800) |
Valkey runs without persistence on purpose. Losing it degrades liveness — presence disappears, in-flight leases expire, queued handoffs need repair — but loses no durable truth. Every consumer treats it as a hint and re-derives authority from PostgreSQL.
If you ever find yourself wanting to store a schedule, a definition, a capability, an assertion, a proof key, or a credential here, that is the signal that the design has drifted.
Object store — bytes
SeaweedFS, S3-compatible, holding:
- uploaded file bytes,
- exports,
- artifact contents,
- sandbox workspace snapshots (
SANDBOX_WORKSPACE_BACKEND = object_store).
Metadata about these objects lives in PostgreSQL. The object store holds bytes and nothing that decides who may read them — every fetch is brokered through an authenticated route that re-checks access.
Ownership and lifecycle
Every durable record has exactly one owner and one lifecycle:
| Record | Owner | Deleting it |
|---|---|---|
| Conversation | Creating member | Ends access; retained per policy |
| Document | Uploading member | Stops retrieval; derived records per policy |
| Artifact version | Authoring member | Immutable in history |
| Agent version | Author | Appends, never overwrites |
| Project | Creator | Removes pointers only — never the pointed-at items |
| Task | Owner | Stops scheduling; evidence retained |
| Journal event | The conversation | Never edited |
| External capability source | Creating member; optionally visible to its workroom | Owner removes the pointer; remote data is unaffected |
| Capability invocation receipt | Initiating member | Retained as non-secret idempotency and audit evidence |
| Identity-circle choice | The member, keyed on the platform subject | Restores the initial-letter default |
| Uploaded circle picture | The member, keyed on the platform subject | Restores the letter; the version keeps climbing |
The pattern is consistent: deletion changes reachability, not history.
Member circle records
member_identity_icon holds one row per member — owner_sub as the primary
key plus the chosen icon and color and an updated_at stamp. The row is
written only through the caller's own forwarded subject, and absence is the
documented default: no row means the member's circle renders the first letter
of their display name. A row may carry a color with no icon, which is how a
member's letter gets a color of its own.
The catalog itself stays in kaizen.domain.member_identity rather than in a
column constraint. Writes are validated against it and reads are re-validated
through it, so a retired symbol degrades to the initial letter instead of
needing a data migration, and no value outside the catalog can reach another
member's browser. The symbol catalog is closed to new choices — the picker
offers a picture or a letter — and it stays in the code precisely so the
members who chose a symbol while it was offered keep the circle they chose.
member_identity_avatar holds the uploaded picture, again one row per member
keyed on owner_sub: a monotonic version, the mime type, image_512 and
image_128 as bytea, and updated_at. The last four are null together when
a member removes their picture, enforced by a check constraint, and the row
itself survives so the version keeps climbing. That is deliberate: the picture
is addressed by owner plus version, so a cleared picture can never be served
out of a browser cache under an address that is still valid.
Two renditions rather than one because a chat roster draws many circles at the small size and only the settings page needs the large one, so the common read never ships four times the bytes it can use. The bytes live in Postgres rather than object storage because they are small, owner-scoped, and read through the same authorization the roster already crosses; a separate store would add a second place for a member's picture to outlive their account.
External capability records
capability_source stores an owner, workroom, display name, uncredentialed
HTTPS schema URL and API base, visibility, explicit cross-origin consent,
enabled state, and timestamps. It does not store the fetched OpenAPI document.
capability_invocation_receipt
claims a signed nonce before egress and records source, operation, tool, schema
digest, effect class, timestamps, and executing, succeeded, failed, or
ambiguous status. It deliberately excludes arguments, bodies, credentials,
and endpoint results.
Explicitly remembered credential bundles are not a fourth Kaizen store. They are written through the caller's forwarded Kamiwaza authorization to the platform DataHub/Catalog encrypted-secret service. Kaizen returns only the opaque URN to the browser, does not retain the value or URN in its capability tables, and exposes no value-read repository.
Corpus Task records
task_corpus_run is an owner-pinned child of one task_run and stores the exact
definition binding, effective limits, phase, control state, checkpoint sequence,
content-free counters, manifest digest, and outcome. task_corpus_input
snapshots deterministic source identity and version metadata.
task_corpus_partition owns fenced lease attempts and bounded source ranges.
task_corpus_record is canonical vendor-neutral output keyed by run, kind, and
stable ID. task_corpus_provenance joins every output to exact input revision,
partition, locator, and step. task_corpus_export records independent JSONL or
GraphML publication state, digest, byte count, object key, and Artifact binding.
No table stores unrestricted source content, credentials, or hidden reasoning.
Projections are disposable
Search passages, Project indexes, the turns cache, and library facets are all
projections. They are rebuilt from the record, they are never the record, and
they never become access capabilities.
The same transaction that stores an index-eligible journal event writes a projection outbox row; background workers then update or tombstone the indexes. An index row that outlives a grant cannot resurrect it, because opening its anchor re-authorizes.
Platform references are not authority
Kaizen persists only opaque Core identifiers needed to correlate its product ledger with the platform ledger: automation grant, run, claim, effect, protected-resource revision, credential binding, correlation identity, revision digest, authority deadline, and fencing token. These values support lookup, reconciliation, and audit but cannot execute anything by themselves.
Sender-constrained run and effect capabilities, projected workload assertions,
DPoP private keys/proofs, broker handles, one-use consumption tokens, and
provider credentials remain in trusted process memory and are explicitly
non-serializable. The historical task_capability_grant table may be retained
for upgrade evidence, but production execution never evaluates it as an
authorization source.
task_draft_test_model_grant is the only task table that ever holds a
credential, and only for a safe test on a deployment without a platform service
identity. Each row is addressed by run, owner, task and capability grant, holds
the member's platform bearer sealed with AES-GCM under KAIZEN_FIELD_KEY, and
expires with the test's capability grant. Consuming it clears the ciphertext in
the same row-locked statement, and a check constraint keeps consumed_at and
the ciphertext mutually exclusive, so a consumed row holds nothing usable. See
Task automation security.
Migrations
Schema changes are versioned migrations applied at deploy time. A database downgrade is not an operational rollback — feature flags are the rollback mechanism, and schema or data cleanup is a separate reviewed operation. See Install & upgrade.
What to back up
| Store | Back up | Why |
|---|---|---|
| PostgreSQL | Yes | Everything authoritative |
| Object store | Yes | Bytes cannot be regenerated |
| Valkey | No | Ephemeral by design |
| Search index | No | A projection; rebuildable by reindex |
See Backup & disaster recovery.