Policy Troubleshooting

Start with Kamiwaza's view of the OPA:

kapi GET /authz/policies/health | jq

Field	false means
enabled	Policy is off in the values file.
reachable	Kamiwaza can't reach the OPA, or the OPA returned an error. error says which.
bundle_loaded	The OPA answers, but hasn't loaded Kamiwaza's bundle.
enforcing	The OPA is behind: its active_revision is older than a change Kamiwaza has published.

kapi GET /authz/policies/status | jq shows what each OPA instance last reported, with any error it hit loading a bundle.

Results are withheld

When policy can't be applied, Kamiwaza returns an error instead of the result. Users see it in Kaizen as a failed tool call.

Message	Cause	Fix
"Content policy couldn't be evaluated, so the result is withheld."	The OPA can't be reached, timed out, or returned an error. This includes a template that fails at runtime, such as calling lower on a number.	Check health. If the OPA is up, check its log for the evaluation error, then fix the template (Write a Template) or retire the constraint that uses it.
"A policy update for this data source isn't active on the policy engine yet (it has revision N, needs M). Retry shortly."	A change to this data source's constraints is published, but the OPA hasn't loaded it. Normal for a few seconds after a change.	If it lasts, the OPA can't download or load the bundle: see The OPA isn't connected.
"The policy asked to deny, which connector results don't support yet.", or the same with mask or pseudonymize	A template on this data source returned deny, mask or pseudonymize, which connector results don't apply yet. Rarely, a constraint's template is missing from the bundle.	List the data source's constraints (below) and check each one's template with kapi GET /authz/policies/templates/<name>. Retire the constraint, or fix the template.
"Content policy is turned off, but these constraints are still attached to this data source: ..."	Policy was turned off while constraints were attached.	Turn policy back on, or detach the constraints: see Turn policy off.
"The policy redacted an item's id, which Core needs to return it."	A constraint on this data source redacts id on browse or search results. Core can't return an item without its id.	Redact a different field, or drop the record instead.
403 on a fetch: "Content policy doesn't allow this item to be returned."	Policy dropped the item that was asked for, or redacted one of its fields. A file is never returned in part.	This is policy working. Check the constraint if the item should be visible.
Fewer results than expected, or none, with no error	Policy dropped them. Users aren't told that records were dropped.	Check the data source's constraints (below). A constraint whose field is missing from the records drops every record, for content_filter marking rules and for templates written to fail closed.

The constraints that apply to a data source:

kapi GET "/authz/policies/constraints?resource=$(jq -rn --arg ds "$DATA_SOURCE" '$ds|@uri')" \
  | jq '.[] | {name, template, version, enforcement_action}'

Other data sources aren't affected by a problem with one data source's constraints, unless the OPA itself is down.

The OPA isn't connected

Check the OPA's own log first: OPA logs each bundle download and status report, with the HTTP status Kamiwaza returned.

Symptom	Fix
The OPA's log shows Bundle load failed: server replied with Unauthorized or status update failed, server replied with HTTP 401	The OPA's bundle token doesn't match the kamiwaza-policy-bundle-token Secret. Fix one to match the other. If you changed the Secret, restart Kamiwaza's API pods so they read it.
The OPA's log shows Conflict or 409 for the bundle	Kamiwaza has no bundle token set: core.authz.contentPolicy.bundle.tokenSecretName is empty.
The OPA's log shows a certificate error	Add Kamiwaza's CA under services.kamiwaza.tls.ca_cert in the OPA's configuration.
health says reachable: false with a certificate error	Check core.authz.contentPolicy.opa.caSecretName and the CA in that Secret.
health says reachable: false with 401 or 403	The OPA requires a token Kamiwaza isn't sending, or doesn't accept it. Check opa.tokenSecretName and the OPA's authorization policy.
health says reachable: false with ConnectError, and the OPA pod is running but not ready	The OPA can't load Kamiwaza's bundle, so its /health?bundles readiness probe fails and Kubernetes keeps it out of its Service. Read the OPA's log for why, using the rows above.
health says reachable: false, timed out or refused, and the OPA is ready	Open the route from the Kamiwaza cluster to the OPA's port, and check opa.url.
status lists no instances	The OPA's configuration has no status section, or it points at another service. See Connect Your OPA, step 2.

Drafts

Symptom	Fix
A draft stays pending	The OPA isn't downloading the kamiwaza_drafts bundle, or isn't reporting status. Check both are in its configuration.
A draft is failed	Read errors in GET /authz/policies/drafts/<id>: each has the line and column. Fix the Rego and start a new draft.
Starting a draft returns "The OPA hasn't loaded Kamiwaza's template check yet"	Kamiwaza checks templates in the OPA, and the OPA hasn't loaded the live bundle. Check health.
Starting a draft returns "Policy is off"	Turn policy on first. Drafts need an OPA to compile them.
A dry run returns "hasn't loaded this draft yet"	The OPA replica that answered hasn't downloaded the draft. Run it again.
Promote returns "changed since this draft was started"	Someone published a newer version in the meantime. Start a new draft from the current version.

Turn policy off

Kamiwaza won't stop filtering a data source just because policy was turned off. Detach every constraint first, then turn policy off.

List the constraints that apply to any data source:

kapi GET /authz/policies/constraints \
  | jq -r '.[] | select(.match.resources | length > 0) | .name as $n | .match.resources[] | "\($n) \(.)"'

Detach them all:

kapi GET /authz/policies/constraints \
  | jq -c '.[] | .name as $n | .match.resources[] | {name: $n, resource: .}' \
  | while read -r c; do
      n=$(jq -r .name <<<"$c")
      jq -c '{resource}' <<<"$c" | kapi POST "/authz/policies/constraints/$n/detach" -d @- \
        | jq -r '"detached \(.constraint.name) (version \(.constraint.version))"'
    done

Run the list command again: it should print nothing.

Or retire constraints you no longer need:

kapi POST /authz/policies/constraints/<name>/retire -d '{"change_note": "<why>"}'

When the list is empty, set core.authz.contentPolicy.enabled: false in your values file and upgrade. Templates and constraints stay stored, with their history, for when you turn policy back on.