Deployment Guide

This guide covers every deployment mode from local development to production Kubernetes clusters.

Deployment model: self-hosted. You run the entire OGuardAI runtime in your own infrastructure. There is no hosted or managed SaaS, so no third party holds your secrets: the operator generates, stores, and injects every secret, and holds all of them. Provision them with the Secrets and Key Ceremony runbook.

Note: The current runtime uses sealed sessions (client-carried encrypted blobs), so no shared session store is required, and each instance is stateless for session handling. A multi-replica deployment still needs a shared revocation backend (redis), because a value revoked on one replica must be refused by all.

Availability: The registry, image and package installs below are private today and provided under a written agreement with Oronts. Request access at /request.

Table of Contents

1. Prerequisites
2. Quick Start (Docker)
3. Pin and roll back by digest
4. Local evaluation
5. Production (Docker Compose)
6. Kubernetes (Helm)
7. Process memory posture
8. Configuration Reference
9. Session Backend Selection
10. Key Rotation Procedure
11. Monitoring and Alerting
12. Troubleshooting
13. Rate Limiting in Multi-Instance Deployments
14. Metrics in Multi-Instance Deployments
15. Detector Mode Selection

Prerequisites

Quick Start (Docker)

Run OGuardAI with a single command using the pre-built Docker image:

docker run -p 3000:3000 \
  -e GUARDAI_SESSION_SECRET=$(openssl rand -base64 32) \
  -e GUARDAI_HOST=0.0.0.0 \
  -e GUARDAI_ALLOW_DEV_NON_LOOPBACK=1 \
  ghcr.io/oronts/oguardai/server:latest

The shipped config binds loopback under dev auth, so GUARDAI_HOST=0.0.0.0 makes the published port reachable and GUARDAI_ALLOW_DEV_NON_LOOPBACK=1 accepts that dev auth is exposed. Both are for evaluation only; use a production profile or the Helm chart for a real deployment.

This starts OGuardAI with:

• Builtin regex detectors (30+ patterns)
• Sealed session backend (client-carried encrypted blobs)
• Dev auth mode (all requests accepted, not for production)
• Default policy
• Bundled Tesseract OCR (English, German, Arabic), so the /v1/transform/image and /v1/redact/image endpoints work out of the box

Verifying image provenance

Published images are Trivy-scanned before push, signed with cosign (keyless, via Sigstore Fulcio/Rekor), and carry an SPDX SBOM attestation. Verify the signature and provenance before running in production (cosign and jq required):

IMAGE=ghcr.io/oronts/oguardai/server:latest
IDENTITY='https://github.com/oronts/.*/.github/workflows/docker.yml@.*'
ISSUER=https://token.actions.githubusercontent.com

# Verify the keyless signature (identity = the release workflow's OIDC subject).
cosign verify "$IMAGE" --certificate-identity-regexp "$IDENTITY" --certificate-oidc-issuer "$ISSUER"

# Verify and extract the SPDX SBOM attestation.
cosign verify-attestation "$IMAGE" --type spdxjson \
  --certificate-identity-regexp "$IDENTITY" --certificate-oidc-issuer "$ISSUER" \
  | jq -r '.payload | @base64d | fromjson | .predicate' > sbom.spdx.json

A failed cosign verify means the image was not built by the official pipeline; do not run it.

Verify the server is running:

curl http://localhost:3000/v1/health
# {"status":"healthy","version":"0.1.0","uptime_seconds":1.2}

Test a transform:

curl -X POST http://localhost:3000/v1/transform \
  -H "Content-Type: application/json" \
  -d '{"input": "Contact julia@example.com for help"}'

Expected response:

{
  "safe_text": "Contact {{email:e_001:7b1068662ab5}} for help",
  "session_id": "...",
  "session_state": "...",
  "entities": [{"token": "{{email:e_001:7b1068662ab5}}", "type": "email"}]
}

The third token segment is a random per-token capability suffix; rehydrate resolves a token only on an exact {{type:id:cap}} match (fail closed), so LLM output must be passed back verbatim.

Image OCR: The official server image bundles Tesseract with English, German, and Arabic language data, so /v1/transform/image and /v1/redact/image work without extra setup. The native release binary needs Tesseract installed on the host, and the separate proxy image does not include it (see PDF Support).

Pin and roll back by digest

A version tag names a release, but a tag is a pointer: a registry can re-point it, and with imagePullPolicy: IfNotPresent two nodes can end up on different bytes for the same tag. Regulated deployments that must prove which bytes ran, and roll back to exactly those bytes, pin the image by its immutable digest (repository@sha256:...) instead. The recipes keep the semver tag as the readable default; pin the digest when determinism matters.

The signed release ledger is the source of truth for each release's digests. Every release attaches oguardai-<version>-release-ledger.json plus its .sig and .pem, recording each image by its digest under the ghcr-images destination.

set -euo pipefail

VERSION=0.1.0
BASE="https://github.com/oronts/OGuardAI/releases/download/v${VERSION}"
curl -LO "$BASE/oguardai-${VERSION}-release-ledger.json"
curl -LO "$BASE/oguardai-${VERSION}-release-ledger.json.sig"
curl -LO "$BASE/oguardai-${VERSION}-release-ledger.json.pem"

# Verify the ledger was signed by this version's OGuardAI release-tag workflow (keyless, via
# Sigstore), then read the digests. The identity regexp is anchored (^...$) and every regex
# metacharacter in VERSION is escaped, so it matches only this exact tagged workflow ref: a bare
# pattern substring-matches, and an unescaped metacharacter (a dot, or a plus in a 1.2.3+build.5
# version) lets a near miss through, so v0.1.0 would also match v0x1x0 and v1.2.3+build.5 would
# accept v1.2.33build.5.
VERSION_RE=$(printf '%s' "$VERSION" | sed -E 's/[].[^$*+?(){}|\\]/\\&/g')
LEDGER_IDENTITY="^https://github\.com/oronts/OGuardAI/\.github/workflows/release\.yml@refs/tags/v${VERSION_RE}$"
ISSUER=https://token.actions.githubusercontent.com
cosign verify-blob "oguardai-${VERSION}-release-ledger.json" \
  --signature "oguardai-${VERSION}-release-ledger.json.sig" \
  --certificate "oguardai-${VERSION}-release-ledger.json.pem" \
  --certificate-identity-regexp "$LEDGER_IDENTITY" --certificate-oidc-issuer "$ISSUER" \
  || { echo "signature verification failed" >&2; exit 1; }

SERVER_DIGEST=$(jq -r '.artifacts[] | select(.destination=="ghcr-images" and .name=="server")      | .id' \
  "oguardai-${VERSION}-release-ledger.json")
DETECTOR_DIGEST=$(jq -r '.artifacts[] | select(.destination=="ghcr-images" and .name=="detector-py") | .id' \
  "oguardai-${VERSION}-release-ledger.json")

# Fail closed: an empty or malformed ledger digest must never fall back to the mutable tag.
echo "$SERVER_DIGEST"   | grep -Eq '^sha256:[0-9a-f]{64}$' ||
  { echo "server digest missing or malformed in the ledger: '$SERVER_DIGEST'" >&2; exit 1; }
echo "$DETECTOR_DIGEST" | grep -Eq '^sha256:[0-9a-f]{64}$' ||
  { echo "detector digest missing or malformed in the ledger: '$DETECTOR_DIGEST'" >&2; exit 1; }

Pin those digests on each deployment surface:

• Helm: set server.image.digest (and detector.image.digest) to the digest, or fill and apply the deploy/helm/oguardai/values-digest-pinned.yaml overlay. A set digest wins over the tag, and the empty default keeps the tag render unchanged.

  helm upgrade oguardai oci://ghcr.io/oronts/charts/oguardai --version 0.1.0 -f my-values.yaml \
    --set server.image.digest="$SERVER_DIGEST" --set detector.image.digest="$DETECTOR_DIGEST" --wait

• Docker Compose, ECS, Cloud Run, Azure Container Apps: reference the image as ghcr.io/oronts/oguardai/server:${VERSION}@${SERVER_DIGEST}. The tag stays readable and the digest fixes the bytes; when both are present the digest is authoritative.

To roll back deterministically, repeat the lookup against the previous version's ledger and re-deploy with that version's digests. The rollout then lands on the exact bytes that version published, not on whatever the tag currently resolves to.

Local evaluation

Runtime source contributions are governed by the contributor agreement documented in CONTRIBUTING.md. For a local evaluation config, run the container image against the file below instead of a local build.

Local evaluation config

Create oguardai.yaml for local development:

server:
  host: "127.0.0.1"
  port: 3000

auth:
  mode: dev

session:
  backend: sealed
  secret: "dev-only-secret-change-in-prod!!"
  ttl_seconds: 3600

detector:
  mode: builtin

policy:
  default: default
  directory: policies
  # Builtin-only detection, so drop the shipped policies that require NER instead of refusing to start.
  skip_incompatible_policies: true

transform:
  context_strategy: full
  max_context_tokens: 4096

Run it against the image instead of a local build. The config binds loopback, so override the bind to 0.0.0.0 and opt into dev auth on a non-loopback bind, otherwise the server only listens on the container's own loopback and the published port cannot reach it:

docker run -p 3000:3000 -v ./oguardai.yaml:/app/oguardai.yaml \
  -e GUARDAI_HOST=0.0.0.0 -e GUARDAI_ALLOW_DEV_NON_LOOPBACK=1 \
  ghcr.io/oronts/oguardai/server:latest --config /app/oguardai.yaml

For the NER detector alongside it, see Production (Docker Compose) below, which runs both services from the published images.

Production (Docker Compose)

Docker Compose deploys three services: the OGuardAI server, the Python NER detector, and a Redis instance for durable revocation state. Sessions use the sealed backend (client-carried encrypted blobs), so no external session store is required; Redis here holds only revocation digests, not session state.

Environment Setup

Create a .env file (never commit this to version control):

# Required: 32-byte session encryption secret
GUARDAI_SESSION_SECRET=$(openssl rand -base64 32)

# Optional: log filter. The compose file substitutes it into RUST_LOG, which replaces
# the server's built-in filter outright. Keep the bare warn floor and the oguardai_server
# bin target: they carry the startup report and the output guard's block reasons.
GUARDAI_LOG_LEVEL=warn,oguardai_server=info,guardai_server=info,tower_http=info

Start the Full Stack

# Required environment variables:
export GUARDAI_SESSION_SECRET="$(openssl rand -base64 32)"
export GUARDAI_API_KEY_1="$(openssl rand -hex 32)"

docker compose -f deploy/docker/docker-compose.yml up

# Test with authenticated request:
# curl -H "X-API-Key: $GUARDAI_API_KEY_1" http://localhost:3000/v1/health

This starts:

Production Docker Compose Configuration

The production docker-compose.yml at deploy/docker/docker-compose.yml includes:

• Health checks on all three services (HTTP readiness for server and detector, redis-cli ping for Redis).
• Restart policy: unless-stopped for automatic recovery.
• Volume mounts: Policies directory mounted read-only; a named redis-data volume for revocation persistence.
• Dependency ordering: The server waits for both the detector and Redis to be healthy before starting.
• Auth: The server runs in api_key mode (GUARDAI_AUTH_MODE=api_key) and requires GUARDAI_API_KEY_1 to be set before the stack starts.
• Durable revocation: The server sets GUARDAI_REVOCATION_BACKEND=redis and GUARDAI_REDIS_URL=redis://redis:6379. Redis persists only HMAC-SHA-256 digests of revoked (type, value) pairs (append-only), never raw PII, so a value revoked once survives a restart.
• Durable, fail-closed audit trail: The server sets GUARDAI_AUDIT_BACKEND=file, GUARDAI_AUDIT_FILE_PATH=/var/lib/guardai/audit/audit.log (on the named guardai-audit volume), and GUARDAI_AUDIT_STRICT=true. A non-dev start requires this durable, strict trail; set GUARDAI_ALLOW_EPHEMERAL_AUDIT=1 only to explicitly accept an ephemeral one.
• Detector auth: The detector is not published on a host port and runs with GUARDAI_DETECTOR_ALLOW_INSECURE=true because it is reachable only on the internal compose network. If you expose it, set GUARDAI_DETECTOR_API_KEY (the X-Detector-API-Key shared secret) on both the detector and the server instead.

Minimal Docker Compose (Server Only)

For deployments that do not need advanced NER or shared sessions:

docker compose -f deploy/docker/docker-compose.minimal.yml up

This starts only the OGuardAI server with builtin detectors and sealed sessions.

Two-instance Docker Compose (one host)

deploy/docker/docker-compose.ha.yml runs two server processes behind nginx with one Redis and one detector on a single host. It selects the ha-production profile and exercises the runtime's multi-replica contract (shared revocation, replay, and idempotency), which is what tests/ha.py checks. It is not a high-availability deployment: the server process is the only redundant component, and Redis, nginx, the detector, and the host are each a single point of failure. Host-level availability needs the Helm chart across several nodes with an external Redis that has its own replication and failover; see Distributed Deployment. The installation guide has the recipe.

Kubernetes (Helm)

Install the Helm Chart

# Install from the OCI registry
helm install oguardai oci://ghcr.io/oronts/charts/oguardai --version 0.1.0 \
  --namespace oguardai \
  --create-namespace \
  --set session.secret="$(openssl rand -base64 32)" \
  --set auth.apiKeys[0].name=default \
  --set auth.apiKeys[0].key="$(openssl rand -hex 32)" \
  --set audit.backend=file \
  --set audit.strict=true \
  --set audit.allowEphemeral=true \
  --set revocation.allowEphemeral=true

The chart defaults to auth.mode=api_key, and a non-dev server start requires a durable, strict audit trail (audit.backend=file plus audit.strict=true), so the install command above sets both.

That command is a first install, not a production one. Both allowEphemeral flags waive durability the chart otherwise insists on: the audit trail and the revocation table each sit on a per-pod volume that is lost when the pod is rescheduled, which breaks the audit hash chain and makes a value revoked before the replacement pod started restorable again. A single-replica production install answers the trail with audit.persistence.enabled=true (a PersistentVolumeClaim); every production install answers the revocation table with revocation.backend=redis, as Production Helm Values below does.

The audit claim should use a durable storage class; a block-backed class presenting ext4, XFS, or Btrfs is recommended for the strongest fsync and locking guarantees. NFS and EFS, SMB and CIFS including Azure Files, FUSE, and unknown filesystems qualify with a startup durability warning under the default, and fail qualification when audit.require_certified_filesystem: true restores the strict allowlist; memory-backed volumes are always refused. Set audit.persistence.storageClass explicitly when the cluster default is not durable. A PVC proves persistence, not the strongest custody guarantees.

Architecture support

The container images are published multi-arch: each image tag is a manifest list carrying a linux/amd64 and a linux/arm64 variant, so docker pull and the Kubernetes kubelet select the matching architecture automatically. This runs on x86-64 clusters and on ARM enterprise Kubernetes (AWS Graviton, Azure Cobalt, Ampere) with no image change.

The Helm chart therefore imposes no node-architecture constraint by default: pods schedule on amd64 or arm64 nodes alike. Set platform.arch to amd64 or arm64 to pin the server and detector pod to one architecture; the chart then merges it into the pod's nodeSelector as kubernetes.io/arch: <arch>, alongside any constraints already set in server.nodeSelector. An operator may instead pin the architecture directly through server.nodeSelector (kubernetes.io/arch: arm64). Setting platform.arch to a value other than amd64, arm64, or "", or setting a kubernetes.io/arch in server.nodeSelector that conflicts with a non-empty platform.arch, fails the render closed.

The Docker Compose stacks under deploy/docker/ pin platform: linux/amd64 for reproducible local development on a single architecture. Remove that pin to run the native image on an arm64 host now that the arm64 variant is published.

The detector image (detector-py) resolves its Python, PyTorch, GLiNER, and spaCy dependencies from aarch64 (manylinux) wheels, so the arm64 variant is functionally complete. Building it for arm64 under emulation is heavy (the PyTorch stack); a native arm64 build runner is recommended in CI.

Production Helm Values

Create values-production.yaml:

server:
  replicaCount: 3
  resources:
    requests:
      cpu: 500m
      memory: 512Mi
    limits:
      cpu: "2"
      memory: 1Gi

# replicaCount > 1 requires the shared redis backends (revocation + idempotency),
# or the chart refuses to render.
revocation:
  backend: redis
idempotency:
  backend: redis
redis:
  external:
    url: redis://redis:6379

session:
  backend: sealed
  ttlSeconds: 3600
  # Reference an existing Kubernetes secret
  existingSecret: oguardai-session-secret
  existingSecretKey: session-secret

auth:
  mode: api_key
  # API keys must use the name/key format matching the Helm chart template.
  # The template generates env vars GUARDAI_API_KEY_{NAME} from each entry.
  apiKeys:
    - name: my-service
      key: "GENERATE_WITH_openssl_rand_hex_32"  # Replace before deploying

policy:
  default: default
  directory: /app/policies

# Durable, fail-closed audit trail. REQUIRED for a non-dev (api_key/jwt/oidc) start:
# the server refuses to boot with the ephemeral log backend unless
# GUARDAI_ALLOW_EPHEMERAL_AUDIT=1 is set.
# This multi-replica values file accepts a per-pod trail; durable custody is your log shipper
# (chart README, Multi-replica audit custody). At replicaCount 1, drop allowEphemeral and set
# audit.persistence.enabled=true for a PersistentVolumeClaim instead.
audit:
  backend: file
  strict: true
  allowEphemeral: true
  file:
    path: /audit/audit.log

transform:
  contextStrategy: full
  maxContextTokens: 4096

# Enable Python NER detector
detector:
  enabled: true
  # The detector handles raw PII and fails closed (503) without the shared
  # X-Detector-API-Key secret. When detector.enabled, the chart requires
  # auth.apiKey or auth.existingSecret (or the explicit dev-only opt-out
  # auth.allowInsecure=true).
  auth:
    existingSecret: oguardai-detector-auth
    existingSecretKey: detector-api-key
  resources:
    requests:
      cpu: 500m
      memory: 1Gi
    limits:
      cpu: "2"
      memory: 2Gi

# Enable horizontal pod autoscaler
autoscaling:
  enabled: true
  minReplicas: 3
  maxReplicas: 10
  targetCPUUtilizationPercentage: 70

# Shared revocation is REQUIRED for more than one replica. The chart refuses to
# install when replicaCount > 1 or autoscaling.enabled unless revocation.backend
# is redis with an external URL: the default memory backend is pod-local, so a
# value revoked on one pod would not be refused by the others. Redis stores only
# HMAC-SHA-256 digests of revoked (type, value) pairs, never raw PII.
revocation:
  backend: redis

redis:
  external:
    url: "redis://redis-master.oguardai.svc.cluster.local:6379"

# Pod security
podSecurityContext:
  runAsNonRoot: true
  runAsUser: 1000
  fsGroup: 1000

securityContext:
  readOnlyRootFilesystem: true
  allowPrivilegeEscalation: false
  capabilities:
    drop:
      - ALL

Install with production values:

helm install oguardai oci://ghcr.io/oronts/charts/oguardai --version 0.1.0 \
  --namespace oguardai \
  --create-namespace \
  -f values-production.yaml

Above one replica the chart keeps the hash-chained trail on a per-pod volume and requires audit.allowEphemeral=true. Every request the server acknowledged was first appended and fsynced to that pod's chain, and each pod's chain verifies on its own; there is no cluster-wide chain. A graceful stop finalizes the chain before the pod exits, so a shipper that outlives the server delivers all of it, but records not yet delivered when a pod is deleted without grace (node loss, eviction, OOM kill) are lost with the volume. Durable custody is your log shipper and append-only destination. See the chart README, "Multi-replica audit custody", for the shipper rules and the verify procedure.

Using External Secrets

To use an external secret manager (Vault, AWS Secrets Manager) with the Helm chart:

# Create the Kubernetes secret first
kubectl create secret generic oguardai-session-secret \
  --namespace oguardai \
  --from-literal=session-secret="$(openssl rand -base64 32)"

# Reference it in Helm values
helm install oguardai oci://ghcr.io/oronts/charts/oguardai --version 0.1.0 \
  --set session.existingSecret=oguardai-session-secret \
  --set session.existingSecretKey=session-secret \
  --set auth.apiKeys[0].name=default \
  --set auth.apiKeys[0].key="$(openssl rand -hex 32)" \
  --set audit.allowEphemeral=true \
  --set revocation.allowEphemeral=true

The two allowEphemeral flags keep this example about the external Secret: they accept a per-pod audit trail and a pod-local revocation table, both lost when the pod is rescheduled. Carry the durable settings from Production Helm Values instead when this goes to a real cluster.

Helm Chart Components

The chart deploys the following Kubernetes resources:

Process memory posture

The server and proxy deny core dumps in process in the first statement of main(), before any request is served: they zero RLIMIT_CORE and, on Linux, clear PR_SET_DUMPABLE. Clearing the dumpable flag is what actually stops a dump of those processes on modern distributions, because a piped core_pattern (systemd-coredump) ignores the size limit at the kernel level. It also makes /proc/<pid>/environ unreadable and denies same-uid ptrace, so a co-located unprivileged process cannot read the environment (where a session secret may live) or attach to the running service. The PDF extraction sandbox child re-applies the same denial as its own first statement: an execve resets the dumpable flag to 1, so the child re-denies rather than inheriting it. Startup fails closed if a control the platform offers refuses to apply.

The systemd unit also sets LimitCORE=0 as defense in depth. No Helm or Docker Compose setting is needed: the in-process control travels with the binary, and Kubernetes exposes no per-container core-dump limit to configure.

Child process confinement

The OCR (Tesseract) and PDF extraction children parse attacker-supplied input, so each confines itself in its own fork-side setup, before it runs, rather than trusting what it inherits. Every such child clears the runtime's environment down to a short allow-list, runs in a private scratch directory in its own process group, and applies a zero core limit that survives execve (so a crash cannot spill the content it holds). On Linux each child also sets no-new-privileges, so no helper it execs can gain rights, and a Landlock filesystem allow-list that reaches only its own binary, the loader and library trees, its language data, and its scratch directory, with all outbound TCP denied. On Linux each child also installs a seccomp syscall filter before it runs: it cannot create a process, cannot open any socket, cannot ptrace, share memory with, or send a process-directed signal to any process outside itself, and cannot create a namespace or reach the io_uring, bpf, keyring, module, or mount surfaces. The OCR child may write to its scratch directory only; the PDF child may write nowhere. The whole child process group is reaped on every failure path, so a helper it forked cannot outlive the request.

Landlock and the syscall filter are the layers that depend on the host rather than the binary. Where seccomp filter mode is present the syscall deny-list applies; on a kernel that lacks it, the child falls back to the other layers and its posture is recorded rather than the spawn being failed. Landlock confines the filesystem on Linux 5.13 and newer, denies outbound TCP on 6.7 and newer, and scopes signals and abstract-socket connects to the child's own domain on 6.12 and newer, degrading to a no-op for the parts an older kernel lacks, where the process-group, environment, and core limits still apply. Two residuals follow from this: below Landlock 6.12 a thread-directed signal by a guessed thread id can still reach another process, since the syscall filter keeps tgkill open for glibc's own raise and abort; and where seccomp filter mode is absent the syscall deny-list does not apply at all. Check /sys/kernel/security/lsm for landlock to confirm the host offers it, and note that a container runtime's seccomp profile must permit the seccomp and Landlock syscalls (Docker's default profile does).

What this does not cover:

• The pre-main window. Between process start and that first statement the process is still dumpable and env-provided secrets are already in memory, so a crash in that brief interval can still dump.
• A piped host core_pattern. execve resets a child's dumpable flag to 1, and a piped core_pattern (for example systemd-coredump) ignores RLIMIT_CORE at the kernel level. The in-process zero core limit stops the ordinary path, but on a host configured that way the Tesseract or PDF child could still be dumped while it holds raw content. Disable the piped handler, or route it to a non-piped target, on hosts that run these children.
• Swap and hibernation. Pages holding secrets can still reach disk through swap or a hibernation image. Disable swap on hosts that run the runtime, or encrypt it, and disable hibernation.
• Key material. Only dropped copies of the redacting secret wrapper are wiped. The derived 32-byte session, audit, and revocation roots are fixed-size values the process keeps for its lifetime and are not zeroized, and stack or allocator residue is likewise outside the guarantee.

To capture a crash for debugging, run the process under a debugger as root rather than relying on a core file; a same-uid unprivileged attach is denied by design.

Deploy-layer process confinement

The confinements above travel with the binary. The deploy artifacts add an OS-level boundary around the whole service process, stacking on top of the runtime's own child confinement rather than replacing it.

• systemd. The unit runs a system-call allow-list (@system-service plus the sandbox calls the runtime uses to confine its OCR and PDF children, with the privileged, module, mount, swap, reboot, raw-io, clock, debug, and cpu-emulation classes subtracted), restricts sockets to AF_INET, AF_INET6, and AF_UNIX, and caps the service cgroup with MemoryMax and TasksMax.
• Docker Compose. The server and detector services drop all Linux capabilities (the single-node server keeps only CHOWN, SETUID, and SETGID, which its entrypoint needs to create the audit directory and step down to the runtime user), set no-new-privileges, and cap memory and process count with mem_limit and pids_limit.
• Kubernetes. The Helm chart already provides the equivalent: a RuntimeDefault seccomp profile, runAsNonRoot, allowPrivilegeEscalation: false, all capabilities dropped, and resources.limits. The cluster's podPidsLimit is the pod-tree process ceiling.

All three back the OCR and PDF scratch path with memory: the Helm chart mounts /tmp as a memory emptyDir (scratch.memoryBacked, default on, with a required strictly-positive scratch.sizeLimit), the Compose stacks mount /tmp as a tmpfs, and the systemd unit points TMPDIR at a RuntimeDirectory under /run (a tmpfs removed at unit stop). A scratch file then lives in memory for its whole lifetime, including a retained one: when the runtime cannot unlink an OCR scratch file it empties it in place (the guardai_ocr_scratch_retention_faults_total counter increments only when that emptying also fails), and that file never touches the container writable layer, the node disk under the kubelet pod dir, or a persistent volume, though swap or hibernation can still page a tmpfs page to disk, so keep swap off on OCR nodes (the Kubernetes default) or accept that residual. This is defense in depth over the runtime's own cleanup, bounded in what it promises. It does not make a full mount safe: Tesseract can report success after its final transcript write fails, and the runtime checks the transcript's size and row shape rather than its completeness, so a full mount can yield a shortened transcript instead of an error (tracked in the gap register). It also does not by itself keep the container under its memory limit.

Size the mount so it cannot fill, and size the memory limit for everything sharing it. Per-run scratch is about 50Mi (file_upload.max_size_bytes) plus 64Mi (the transcript cap); concurrent OCR runs default to limits.max_blocking_ops, which honors the cgroup CPU quota, so the Helm default cpu: "1" allows one run at a time while the Compose and systemd stacks set no CPU limit and default to the host core count (capped at 256). Keep scratch.sizeLimit at or above that overlap times the per-run footprint, and keep server.resources.limits.memory sized for the process, uploads, the decoded image, and scratch together (the default pairs 256Mi with a 512Mi limit for one run at a time). Set limits.max_blocking_ops or the CPU limit explicitly on many-core hosts.

Configuration Reference

The oguardai.yaml file controls all server behavior. Every field has a sensible default.

Server Section

server:
  host: "0.0.0.0"      # Bind address (binary default 127.0.0.1; a non-dev deployment like this binds 0.0.0.0)
  port: 3000            # Bind port (default: 3000)
  allowed_origins:      # CORS allowlist enforced in non-dev auth modes; env GUARDAI_CORS_ORIGINS (comma-separated)
    - "https://app.example.com"
  tls:                  # Optional in-process TLS; TLS is usually terminated at the ingress instead
    enabled: false
    cert_path: "/etc/oguardai/tls.crt"   # env GUARDAI_TLS_CERT
    key_path: "/etc/oguardai/tls.key"    # env GUARDAI_TLS_KEY

Auth Section

auth:
  mode: api_key         # dev | api_key | jwt | oidc (default: dev)
  api_keys:             # Only used when mode=api_key
    - key: sk-...
      identity: my-service
      tenant_id: tenant-1  # Optional: multi-tenancy scope
      scopes:           # Optional: restrict to specific scopes
        - transform
        - rehydrate
  # JWT config (only used when mode=jwt)
  jwt:
    secret: "your-hmac-secret"          # HMAC secret; dedicate it to this deployment (see note)
    issuer: "https://auth.example.com"  # REQUIRED: expected `iss` claim; startup fails if unset or blank
    audience: "guardai-api"             # REQUIRED: expected `aud` claim; a token whose audience differs is rejected
    max_token_age_secs: 900             # Optional replay-window bound; a token must carry `iat` and be no older, and a token without `iat` is rejected when this is set
  # OIDC config (only used when mode=oidc)
  oidc:
    issuer: "https://issuer.example.com"
    jwks_url: "https://issuer.example.com/.well-known/jwks.json"
    audience: "guardai-api"
    tenant_claim: "tenant_id"   # Claim that carries the tenant identifier (required)
    algorithms: [RS256]         # Asymmetric allowlist: RS256/384/512, ES256/384, PS256/384/512

Note: The api_keys entries use key, identity, tenant_id, and scopes fields.

JWT binding: jwt.issuer and jwt.audience are required for mode: jwt; startup fails with a named error until both are set. They stop a token minted for another issuer or service from authenticating on the shared HMAC secret. Dedicate the secret to this deployment: a secret shared across services still collapses the trust boundary if that other service leaks it. max_token_age_secs is optional and bounds the replay window of a leaked token.

Session Section

session:
  backend: sealed       # sealed (default: sealed)
  secret: "..."         # 32-byte encryption secret (REQUIRED for production)
  ttl_seconds: 3600     # Session TTL in seconds (default: 3600)

The GUARDAI_SESSION_SECRET environment variable overrides session.secret.

Supported backends: sealed (default, stateless, cross-replica), memory (in-process, dev/test, single-instance), and redis (server-side sessions encrypted at rest in a shared Redis, cross-replica HA; set session.redis_url, replay_backend: redis, and revocation_backend: redis). managed is planned.

Revocation Backend

Revocation lists which entity values must be refused at rehydrate (only HMAC digests are stored, never raw PII).

revocation_backend: memory   # memory (file-backed, per-instance) | redis (shared)
replay_backend: memory       # memory (per-instance) | redis (shared); rejects replayed continuations
session:
  redis_url: "redis://redis:6379"   # required when revocation_backend or replay_backend is redis

GUARDAI_REVOCATION_BACKEND, GUARDAI_REPLAY_BACKEND, and GUARDAI_REDIS_URL override these. Use redis whenever you run more than one instance, so a value revoked on one is refused by all. The provided deploy/docker/docker-compose.ha.yml wires this for two instances on one host (see Two-instance Docker Compose).

Detector Section

detector:
  mode: builtin         # builtin | advanced | both (default: builtin)
  advanced_url: "http://detector:9090"  # URL of Python NER sidecar
  default_language: null  # ISO 639-1 fallback when a request omits its own language; null falls back to "en" (no auto-detect)
  timeout_secs: 5  # NER request timeout (max 120); env GUARDAI_DETECTOR_TIMEOUT_SECS

Policy Section

policy:
  default: default      # Default policy name (default: "default")
  directory: policies   # Directory containing policy YAML files (default: "policies")

On a builtin-only server keep policy.default: default so it stays loadable. The shipped NER deployment profiles (ner, single-node-production, ha-production, and the Helm chart when detector.enabled is set) instead select default-ner-required: it masks and restores identically to default but declares person, company, and location as a hard NER floor, so a request fails closed with GUARDAI_DETECTION_UNAVAILABLE (503, retriable) on a sidecar outage rather than returning a name raw. Pin policy.default: default explicitly to keep the best-effort behaviour. GET /v1/capabilities reports the resulting detection_floor.

Transform Section

transform:
  context_strategy: full   # full | type_summary | referenced_only | none (default: full)
  max_context_tokens: 4096 # Max tokens for entity_context (default: 4096)

Output Protection Section

output_protection:
  enabled: true         # Output guard is enabled by default
  mode: strict          # strict | permissive (default: strict)
  default_action: mask  # allow | warn | mask | block (default: mask)
  exempt_types:         # Only 'url' is exemptible; a URL is component-scanned before forwarding.
    - url               # Any other entry is rejected at startup. To pass a type through use
                        # entity_actions: {<type>: allow}; for a trusted sink use GUARDAI_TRUSTED_OUTPUT_CHANNELS.

Prompt Security Section

prompt_security:
  enabled: true         # Enable prompt injection scanning (default: true)
  action: strip         # warn | strip | block (default: strip)

File Upload Section

file_upload:
  max_size_bytes: 52428800  # 50MB default

Rate Limit Section

rate_limit:
  enabled: false        # Enable rate limiting (default: false)
  requests_per_second: 100  # Global rate limit (default: 100)
  burst_size: 200       # Burst allowance (default: 200)

Audit Section

audit:
  backend: file         # log (ephemeral, fail-open; the default) | file (durable, hash-chained)
  strict: true          # fail a request closed (503) when its audit event cannot be recorded (default: false)
  file:
    path: /var/lib/guardai/audit/audit.log   # required when backend=file
  allow_ephemeral: false  # explicit waiver of the non-dev durable, strict requirement

A non-dev (api_key/jwt/oidc) start requires the durable, strict trail (backend: file plus strict: true); it refuses to boot otherwise. Set audit.allow_ephemeral: true (or GUARDAI_ALLOW_EPHEMERAL_AUDIT=1) to explicitly accept an ephemeral or non-strict trail. The --compliance preset always forces file + strict and ignores the waiver.

Place the live file in a stable, trusted parent directory that does not traverse symlinks. The leaf must be regular, owned by the runtime user, single-link, and exact mode 0600; the runtime can safely narrow only a strict permission superset. Unsafe existing leaves are rejected. See Compliance Controls for archive source and create-new export requirements.

Idempotency Section

idempotency:
  backend: memory              # memory (per-replica) | redis (shared; uses session.redis_url)
  ttl_seconds: 86400           # how long a completed Idempotency-Key result is replayed (default: 86400)
  in_progress_ttl_seconds: 60  # how long an in-flight claim blocks a duplicate before it is reclaimable (default: 60)

Mutating, non-streaming routes (transform, batch transform, RAG ingest/delete, revoke, session delete) accept an Idempotency-Key header so a client can retry without executing the mutation twice.

Tenants Section

tenants:
  acme-corp:
    default_policy: gdpr-strict
    rate_limit:
      requests_per_second: 50
      burst_size: 100
  startup-inc:
    default_policy: default
    rate_limit:
      requests_per_second: 200
      burst_size: 400

Environment Variable Overrides

Setting RUST_LOG replaces the server's built-in filter entirely. Keep the bare warn floor and the oguardai_server bin target (distinct from the guardai_server library) in whatever you set: they carry the startup report and the output guard's own block reasons, so a filter naming only guardai_server leaves a refused request with no server-side explanation.

The provided compose stack relies on GUARDAI_AUTH_MODE, GUARDAI_API_KEY_1, GUARDAI_REVOCATION_BACKEND, GUARDAI_REDIS_URL, and the GUARDAI_AUDIT_* trio above. deploy/systemd/oguardai.env.example is the env-file starting point for systemd deployments; add the GUARDAI_AUDIT_* variables to it, since a non-dev start requires the durable, strict audit trail.

Session Backend Selection

Sealed Backend

The sealed backend is the default session backend. Session state is encrypted (AES-GCM) and returned to the client as an opaque blob. The client must carry this blob between transform and rehydrate calls.

Advantages:

• No server-side state; horizontal scaling on the session path with no session store or session affinity. A multi-replica deployment still shares one dependency, the revocation store (revocation_backend=redis), so an erased value stays erased across replicas.
• You want the client to control session lifecycle.

Considerations:

• Blob size grows with entity count.
• Client must carry blob between requests.
• Your entity counts per request should be moderate (< 100 entities).

The memory backend is available for dev/test (in-process, lost on restart, single-instance). The redis session backend ships for multi-instance shared session state: each session is encrypted at rest (AES-256-GCM) so Redis holds only ciphertext. The Redis revocation and replay backends also ship for multi-instance shared state.

Key Rotation Procedure

The procedure below rotates session-sealing keys. Audit-root rotation is separate because existing audit records must remain verifiable under their writing root. If audit.secret is unset, changing the base session.secret changes both roots, so archive the live audit file and configure the old root in audit.retired_secrets first. See the Key Rotation Runbook.

OGuardAI seals through a key ring indexed by key ID (kid). New blobs seal under current_kid; blobs sealed under any other key still in the ring keep decrypting until that key is removed. This supports both a simple single-key change and a zero-downtime multi-key rotation.

Single-key change (brief disruption)

With no session.keyring block the runtime holds one key derived from session.secret. Replacing that secret invalidates every existing blob, so do it in a low-traffic window.

# Generate a cryptographically random 32-byte secret
openssl rand -base64 32

session:
  secret: "new-production-secret-32-bytes!!"

After deploy, blobs sealed under the old secret fail authentication and return GUARDAI_SESSION_EXPIRED; clients re-transform the original text to get a new blob.

Zero-downtime rotation (multi-key keyring)

Configure a session.keyring and roll the key in stages so in-flight sessions never break:

1. Add the new key (kid 1) but keep current_kid on the old key, then deploy to every replica so all of them know kid 1 before any pod seals under it:

session:
  keyring:
    current_kid: 0
    keys:
      - kid: 0
        secret: "${OLD_SECRET}"
      - kid: 1
        secret: "${NEW_SECRET}"

2. Switch current_kid to 1 (keep both keys): new sessions seal under kid 1 while kid 0 blobs still decrypt.
3. After one session TTL, remove kid 0 from the ring. Blobs sealed under it then fail to decrypt.

See the key rotation runbook for the full session and audit procedures and their ordering constraints.

Key Rotation Timeline

Single-key change:
  Old key active   |  Deploy new secret  |  New key only
  (all sessions)   |  (old blobs become  |  (clients re-transform
                   |   undecryptable)    |   to get new blobs)

Zero-downtime keyring:
  Seal under kid 0 | Add kid 1, seal kid 0 | Switch current_kid 1 | Remove kid 0
                   |                       | (kid 0 decrypts      | (kid 0 blobs
                   |                       |  until TTL)          |  now invalid)

Monitoring and Alerting

Health Check

The /v1/health endpoint returns the overall system status (requires auth when enabled; use /livez for unauthenticated probes):

curl http://localhost:3000/livez                    # Unauthenticated liveness probe
curl -H "X-API-Key: $GUARDAI_API_KEY" http://localhost:3000/v1/health  # Full health check

Response:

{
  "status": "healthy",
  "version": "0.1.0",
  "uptime_seconds": 3600.5,
  "components": {
    "detector": {"status": "healthy", "message": "builtin_and_ner (full entity detection)"},
    "session": {"status": "healthy", "message": "sealed"},
    "policies": {"status": "healthy", "message": "3 policies loaded"}
  },
  "license": {"status": "absent", "edition": "developer", "release_date": "2026-07-29"}
}

/v1/health reports healthy by design as long as the handler runs: it must not flip a pod on an optional sidecar. Do not build a readiness gate on it. Use /readyz, which fails when no policy is loaded, when a required NER sidecar does not answer, or when the audit sink is unhealthy.

The license block reports the deployment's licence and gates nothing; see Observability.

What to Monitor

Recommended Alert Rules

# Example Prometheus alerting rules
groups:
  - name: guardai
    rules:
      - alert: OGuardAIUnhealthy
        expr: probe_success{job="guardai-health"} == 0
        for: 2m
        labels:
          severity: critical
        annotations:
          summary: "OGuardAI health check failing"

      # Replace 0.5 with a threshold derived from your own measured baseline.
      - alert: OGuardAIHighLatency
        expr: histogram_quantile(0.95, rate(guardai_transform_duration_seconds_bucket{job="guardai"}[5m])) > 0.5
        for: 5m
        labels:
          severity: warning
        annotations:
          summary: "OGuardAI transform p95 above the configured threshold"

      - alert: OGuardAIHighErrorRate
        expr: rate(guardai_errors_total{job="guardai"}[5m]) / (rate(guardai_transforms_total{job="guardai"}[5m]) + rate(guardai_rehydrates_total{job="guardai"}[5m]) + 1) > 0.01
        for: 5m
        labels:
          severity: critical
        annotations:
          summary: "OGuardAI 5xx error rate above 1%"

Log Aggregation

OGuardAI emits structured logs compatible with all major SIEM and log aggregation systems. Ordinary application log events carry whatever fields the emitting span attached, so the fields are not uniform across every line. The consistent, machine-parseable per-operation record is the audit event on the guardai::audit target, whose fields are fixed:

{
  "target": "guardai::audit",
  "message": "audit_event",
  "event_type": "Transform",
  "tenant_id": "822b33ad87c148a0",
  "session_id": "9f2c4a1d7b3e0058",
  "policy": "4d1d5f676d7930f1",
  "policy_content_hash": "<sha256-of-resolved-policy>",
  "entity_types": ["email", "phone"],
  "entity_count": 3,
  "entities_blocked": 1,
  "entities_tokenized": 2,
  "duration_ms": 12.5,
  "detector_mode": "both",
  "output_guard": false,
  "prompt_security": false,
  "failure_code": "none"
}

tenant_id, session_id, and trace_id are keyed fingerprints. Free-form policy and GDPR labels use separate keyed field domains, so none reaches the log verbatim. Correlate with those fingerprints and use the exact policy_content_hash to identify resolved policy content. The full field list is in the Security Whitepaper.

A failed request additionally emits one request failed event carrying error_code, status, and request_id. That is where you read a specific error code from, because guardai_errors_total is a single unlabelled counter with no per-code breakdown.

Forward logs to your SIEM using standard log collectors (Fluent Bit, Fluentd, Vector, Filebeat).

Troubleshooting

Server does not start

Symptom: oguardai-server exits immediately.

Causes and fixes:

"session expired" errors on rehydrate

Symptom: Rehydrate returns {"error": "...", "code": "GUARDAI_SESSION_EXPIRED"}.

Causes and fixes:

Detector sidecar not connecting

Symptom: /readyz returns 503, or guardai_ner_degraded_total / GUARDAI_DETECTION_UNAVAILABLE increase. Note that /v1/health always reports the detector healthy even when a configured NER sidecar is unreachable, so use /readyz (which probes the sidecar) and ner_active on /v1/capabilities to detect a downed sidecar.

Causes and fixes:

High latency on first request

Symptom: First request takes significantly longer than subsequent requests.

Cause: Not the built-in regex set. Every built-in pattern is compiled when the detector is constructed at startup, so pattern compilation is not on the request path. Look at what is still cold instead: the NER sidecar (/readyz reports not-ready until a model is loaded), the upstream provider connection, and your own caches.

Fix: Gate traffic on /readyz rather than /livez so a pod is not sent work before its detector is usable, and send a warmup request after deployment.

Entity not detected

Symptom: Known PII is not being replaced.

Causes and fixes:

Session seal/unseal failures

Symptom: GUARDAI_SESSION_EXPIRED or seal/unseal errors on rehydrate. The /v1/health session component always reports healthy and only echoes the configured backend name; it does not surface per-request seal failures, so watch the error codes and logs rather than the health status.

Causes and fixes:

Rate Limiting in Multi-Instance Deployments

OGuardAI's rate limiter operates per-instance. Each server process maintains independent rate limit buckets.

Implications for Kubernetes / multi-instance:

Recommended configurations:

For strict per-client limits: Use sticky sessions (session affinity) in your load balancer so each client always hits the same instance.

For global limits: Divide the desired global limit by the number of instances:

# 3 instances, want 300 req/s global -> 100/s per instance
rate_limit:
  enabled: true
  requests_per_second: 100
  burst_size: 200

For shared rate limiting (advanced): Deploy a Redis-backed rate limiter in front of OGuardAI (e.g., Kong, Envoy, or API gateway rate limiting). OGuardAI's built-in rate limiter then serves as a secondary defense.

Metrics in Multi-Instance Deployments

Each OGuardAI instance exposes its own /metrics endpoint. Prometheus scrapes all instances independently.

Prometheus scrape config:

# prometheus.yml
scrape_configs:
  - job_name: 'guardai'
    # /metrics requires the `admin` scope whenever auth is enabled. Present an admin
    # credential: an Authorization: Bearer token in jwt/oidc mode, or route the scrape
    # through a proxy that injects the X-API-Key header in api_key mode.
    authorization:
      type: Bearer
      credentials: '<admin-scoped-jwt>'
    kubernetes_sd_configs:
      - role: pod
        selectors:
          - role: pod
            label: app=oguardai-server
    relabel_configs:
      - source_labels: [__meta_kubernetes_pod_ip]
        target_label: __address__
        replacement: '$1:3000'

Aggregation in PromQL:

# Total transforms across all instances
sum(guardai_transforms_total)

# Per-instance error rate
rate(guardai_errors_total[5m])

# p95 latency across all instances
histogram_quantile(0.95, sum(rate(guardai_transform_duration_seconds_bucket[5m])) by (le))

Key point:

Per-instance metrics is the standard pattern for Prometheus-based observability. No shared metrics backend needed.

Detector Mode Selection

Quick Decision Guide

How to Check Current Mode

# Health endpoint shows detector status (requires auth when enabled)
curl -H "X-API-Key: $GUARDAI_API_KEY" http://localhost:3000/v1/health | jq '.components.detector'

# Capabilities shows what entity types are available
curl -H "X-API-Key: $GUARDAI_API_KEY" http://localhost:3000/v1/capabilities | jq '.entity_types[].name'

# Diagnostics shows full config (requires admin scope)
curl -H "X-API-Key: $GUARDAI_API_KEY" http://localhost:3000/v1/diagnostics | jq '.detector_mode'

NER Sidecar Availability

When detector.mode: both is configured:

Latency Troubleshooting

If transform latency is above the baseline you measured for this deployment:

1. Check detector mode: curl -H "X-API-Key: $GUARDAI_API_KEY" http://localhost:3000/v1/diagnostics | jq .detector_mode
2. If "Both": Check if NER sidecar is running at the configured URL
3. If NER is down: Either start the sidecar or switch to detector.mode: builtin
4. Verify: curl -H "X-API-Key: $GUARDAI_API_KEY" http://localhost:3000/metrics | grep transform_duration_seconds

This is the canonical deployment guide. A stub exists at docs/deployment-guide.md for repository navigation.