Roll back a bad policy deployment. Policies are loaded at server startup from the configured policy directory.

Symptoms of a Bad Policy

Entities that should be masked are passing through unprotected.
Entities that should pass through are being blocked or masked.
Customer complaints about broken responses or missing data.
Unexpected spike in guardai_errors_total or guardai_output_guard_triggers_total.

Step 1: Confirm the Problem

# View active policy
kubectl exec deploy/guardai -- cat /app/policies/default/policy.yaml      # Kubernetes
cat /etc/guardai/policies/default/policy.yaml                              # systemd

# Test a known input (include X-API-Key for api_key auth mode)
curl -s -X POST http://guardai.internal:3000/v1/detect \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $GUARDAI_API_KEY" \
  -d '{"input": "My SSN is 123-45-6789 and email is test@example.com", "include_values": false}' | jq .

Compare output against expected behavior for the active policy.

Step 2: Identify What Changed

git log --oneline -10 -- policies/
git diff HEAD~1 -- policies/
diff previous-policy.yaml current-policy.yaml

Step 3: Revert the Policy

Kubernetes (Helm):

helm rollback guardai
helm status guardai

If policy is in a standalone ConfigMap:

kubectl create configmap guardai-policy --from-file=policy.yaml=known-good-policy.yaml \
  --dry-run=client -o yaml | kubectl apply -f -
kubectl rollout restart deployment/guardai
kubectl rollout status deployment/guardai --timeout=5m

Docker Compose (policies bind-mounted from policies/ directory):

cp policies/default/policy.yaml.bak policies/default/policy.yaml
docker compose -f deploy/docker/docker-compose.yml restart server

systemd:

cp /etc/guardai/policies/default/policy.yaml.bak /etc/guardai/policies/default/policy.yaml
sudo systemctl restart oguardai-server

Hot reload (no restart)

After restoring the known-good files on disk, swap the active policy in place instead of restarting:

curl -s -X POST http://guardai.internal:3000/v1/admin/policy/reload \
  -H "X-API-Key: $GUARDAI_ADMIN_KEY" | jq .

The reload requires the admin scope and runs the same fail-closed chain as startup (integrity, load, validate). On success it atomically swaps the engine and returns { "reloaded": true, "changes": [...] }. If the restored set is invalid or empty in non-dev it returns "reloaded": false and keeps the running policy unchanged, so a botched rollback never drops protection. With policy.watch: true the same reload fires automatically a few seconds after the files change.

Step 4: Verify the Rollback

curl -sf http://guardai.internal:3000/livez

curl -s -X POST http://guardai.internal:3000/v1/detect \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $GUARDAI_API_KEY" \
  -d '{"input": "My SSN is 123-45-6789 and email is test@example.com", "include_values": false}' | jq .

Confirm entity actions (mask, block, passthrough) match expected policy. Monitor guardai_errors_total for 10 min.

Step 5: Investigate Root Cause

After rollback is stable:

Review bad policy for: missing entity types, wrong actions (passthrough vs mask), incorrect rule ordering (first-match wins), typos in entity type names.

Dry-run the bad policy:

curl -s -X POST http://guardai.internal:3000/v1/evaluate-policy \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $GUARDAI_API_KEY" \
  -d '{"entities": [{"type": "ssn", "value": "123-45-6789"}], "policy": "strict-pii"}' | jq .

Prevention

Back up current policy before every change: cp policy.yaml policy.yaml.bak
Test new policies in staging first.
Use the evaluate-policy endpoint as a dry run before production deployment.
Version control all policy files. Tag releases for easy rollback targets.
Always review the diff before deploying: diff old-policy.yaml new-policy.yaml

Policy Rollback Runbook