Architecture Overview

                    Load Balancer (nginx/ALB/Envoy)
                           |
              +------------+------------+
              v            v            v
         OGuardAI-1    OGuardAI-2    OGuardAI-3
              |            |            |
              +------------+------------+
                           |
                Sealed sessions (stateless)
                  Shared revocation via Redis
        (or shared server-side sessions via encrypted Redis)
                           |
                     Python NER (optional)
                    +------+------+
                    |  GLiNER     |
                    |  sidecar    |
                    +-------------+

Stateless Design

OGuardAI is stateless by default using sealed sessions:

Session state travels in the encrypted blob (client-held)
No server-side session storage needed
Any instance can handle any request
Horizontal scaling = add more instances

Components by Statefulness

Component	Stateless?	Multi-Instance Safe?	Notes
Transform pipeline	Yes	Yes	Pure function
Sealed sessions	Yes	Yes	Client-held encrypted blob
Policy engine	Yes	Yes	Loaded from files at startup
Detector (builtin)	Yes	Yes	Compiled regex, no state
Rate limiter	Per-instance	Acceptable	See rate limiting section
Metrics	Per-instance	Correct	Prometheus scrapes each instance
Revocation (file)	Per-instance	Not recommended	Use Redis for multi-instance
Revocation (Redis)	Shared	Yes	Set `revocation_backend: redis` + `GUARDAI_REDIS_URL`
Replay store (memory)	Per-instance	Cold-start window	Per-replica; replay state lost on restart
Replay store (Redis)	Shared	Yes	Set `replay_backend: redis` + `GUARDAI_REDIS_URL`; closes the cross-replica/restart window. The proxy uses the same store via `GUARDAI_REPLAY_BACKEND=redis` (env-configured, no YAML)
Session backend (sealed)	Stateless	Yes	Client-held encrypted blob, no shared state needed
Session backend (Redis)	Shared	Yes	Set `session.backend: redis` + `GUARDAI_REDIS_URL` for server-side sessions the client references by `session_id`; each is encrypted at rest (AES-256-GCM), so Redis holds only ciphertext. Requires `replay_backend: redis` (a shared replay store serializes cross-replica continuations); startup fails closed otherwise. Use `maxmemory-policy noeviction`; the bundled Docker Compose stacks ship this plus an authenticated scoped ACL and network isolation by default
Session backend (memory)	Per-instance	No	Pod-local; a session on one replica is invisible to others. Single-instance only
NER sidecar	Shared	Yes	Single sidecar serves all instances

Revocation must be the single shared authority on every replica, or a value revoked on one stays restorable through another. Startup fails closed unless revocation_backend: redis (GUARDAI_REDIS_URL set) whenever a multi-replica topology is advertised: a redis session or replay backend implies it, and the Helm chart derives it from replicaCount. A direct (non-Helm) deployment running sealed (stateless) sessions across replicas carries no such signal, so it MUST advertise HA explicitly with high_availability: true (or GUARDAI_HIGH_AVAILABILITY=true); the proxy uses the same env flag. An invalid flag value aborts startup rather than silently disabling the check.

Recommended Production Setup

Kubernetes (3 instances, sealed sessions, NER)

The server selects the shipped ha-production profile bundled in the image at /app/profiles and overrides only the per-deployment secrets and endpoints. It does real NER through a GLiNER detector that runs as its own Deployment behind a Service named detector, the endpoint the profile resolves at http://detector:9090. Because the detector provides NER, the bundled policies at /app/policies all load: no capability is silently dropped.

The API runs as a StatefulSet so every ordinal owns one ReadWriteOnce audit PVC. The cluster's default StorageClass should present durable storage; ext4, XFS, or Btrfs block storage is recommended for the strongest guarantees, and it is required when audit.require_certified_filesystem: true restores the strict allowlist. Set storageClassName in the claim template when the default is not durable. A multi-replica Deployment with emptyDir is not a durable audit topology because emptyDir is memory-backed or node-local and replacing a pod deletes that replica's chain.

# guardai.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: oguardai-server
  labels:
    app: oguardai-server
spec:
  serviceName: oguardai-server-headless
  replicas: 3
  selector:
    matchLabels:
      app: oguardai-server
  template:
    metadata:
      labels:
        app: oguardai-server
    spec:
      terminationGracePeriodSeconds: 180
      # fsGroup opens a fresh volume root to the non-root initializer once. The server receives only
      # the private 0700 subdirectory, never the group-writable root.
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
        fsGroup: 1000
        fsGroupChangePolicy: OnRootMismatch
      initContainers:
        - name: audit-volume-init
          image: ghcr.io/oronts/oguardai/server:0.1.0
          command: ["/usr/bin/install"]
          args: ["-d", "-m", "0700", "/audit-volume/runtime"]
          securityContext:
            readOnlyRootFilesystem: true
            allowPrivilegeEscalation: false
            capabilities:
              drop: ["ALL"]
          volumeMounts:
            - name: audit
              mountPath: /audit-volume
      containers:
        - name: guardai
          # Pin an immutable digest in production. Replace the tag below with the digest you pulled
          # and verified, for example:
          #   image: ghcr.io/oronts/oguardai/server:0.1.0@sha256:REPLACE_WITH_PULLED_DIGEST
          image: ghcr.io/oronts/oguardai/server:0.1.0
          # Select the shipped ha-production profile; override only the secrets and endpoints below.
          # Never hand-assemble a config: the profile already boots and honors the HA contract.
          args: ["--config", "/app/profiles/ha-production.yaml"]
          ports:
            - containerPort: 3000
          livenessProbe:
            httpGet:
              path: /livez
              port: 3000
            initialDelaySeconds: 5
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /readyz
              port: 3000
            initialDelaySeconds: 3
            periodSeconds: 5
          env:
            - name: GUARDAI_SESSION_SECRET
              valueFrom:
                secretKeyRef:
                  name: guardai-secrets
                  key: session-secret
            - name: GUARDAI_API_KEY_1
              valueFrom:
                secretKeyRef:
                  name: guardai-secrets
                  key: api-key
            - name: GUARDAI_DETECTOR_API_KEY
              valueFrom:
                secretKeyRef:
                  name: guardai-secrets
                  key: detector-api-key
            - name: GUARDAI_REDIS_URL
              valueFrom:
                secretKeyRef:
                  name: guardai-secrets
                  key: redis-url
            # Least privilege for the primary key: the policy, destination, and restore channel it
            # may use. Deny-by-default otherwise, so these make the key usable without widening it.
            - name: GUARDAI_API_KEY_1_ALLOWED_POLICIES
              value: "default"
            - name: GUARDAI_API_KEY_1_ALLOWED_DESTINATIONS
              value: "external_llm"
            - name: GUARDAI_API_KEY_1_ALLOWED_OUTPUT_CHANNELS
              value: "customer_email"
          volumeMounts:
            - name: audit
              mountPath: /var/lib/guardai/audit
              subPath: runtime
  volumeClaimTemplates:
    - metadata:
        name: audit
      spec:
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 10Gi
---
apiVersion: v1
kind: Service
metadata:
  name: oguardai-server-headless
spec:
  clusterIP: None
  selector:
    app: oguardai-server
  ports:
    - port: 3000
      targetPort: 3000
---
apiVersion: v1
kind: Service
metadata:
  name: oguardai-server
spec:
  selector:
    app: oguardai-server
  ports:
    - port: 3000
      targetPort: 3000
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: detector
  labels:
    app: detector
spec:
  replicas: 1
  selector:
    matchLabels:
      app: detector
  template:
    metadata:
      labels:
        app: detector
    spec:
      containers:
        - name: detector
          image: ghcr.io/oronts/oguardai/detector-py:0.1.0
          ports:
            - containerPort: 9090
          env:
            - name: NER_BACKEND_STRICT
              value: "true"
            - name: GUARDAI_DETECTOR_API_KEY
              valueFrom:
                secretKeyRef:
                  name: guardai-secrets
                  key: detector-api-key
          readinessProbe:
            httpGet:
              path: /readyz
              port: 9090
---
apiVersion: v1
kind: Service
metadata:
  name: detector
spec:
  selector:
    app: detector
  ports:
    - port: 9090
      targetPort: 9090

Each StatefulSet ordinal writes an independent audit chain. Verify, export, retain, and externally anchor each chain separately; never concatenate replica files. StatefulSet PVCs survive pod replacement, but deleting a claim deletes its chain, so protect the claims with the cluster's retention, snapshot, and backup controls.

Create the secrets

The server reads its per-deployment secrets and endpoints from a Secret; the profile provides everything else. The detector key authenticates the server to detector hop:

kubectl create secret generic guardai-secrets \
  --from-literal=session-secret="$(openssl rand -base64 32)" \
  --from-literal=api-key="$(openssl rand -hex 32)" \
  --from-literal=detector-api-key="$(openssl rand -hex 32)" \
  --from-literal=redis-url="redis://your-redis-host:6379"

Scaling Guidelines

No instance-count-per-request-rate table is published here, because no load run supports one: see Benchmarks for the rule. Size from your own measurement, on two independent axes:

API instances carry the builtin detection path and scale horizontally, because sealed sessions travel with the request and instances do not coordinate on the transform path. Measure guardai_transform_duration_seconds and your per-instance CPU headroom, then add instances.
NER sidecars carry model inference. Adding API instances adds no NER throughput. A NER-heavy workload needs sidecar capacity sized separately, on the sidecar's own hardware.

Whatever the count, more than one replica means shared revocation. See the fail-closed rule above.

Health Checks

livenessProbe:
  httpGet:
    path: /livez
    port: 3000
  initialDelaySeconds: 5
  periodSeconds: 10

readinessProbe:
  httpGet:
    path: /readyz
    port: 3000
  initialDelaySeconds: 3
  periodSeconds: 5

/livez and /readyz are unauthenticated, safe for K8s probes even with api_key auth enabled. Use /v1/health with X-API-Key for detailed health status.

Distributed Deployment