Architecture Overview

                    Load Balancer (nginx/ALB/Envoy)
                           |
              +------------+------------+
              v            v            v
         OGuardAI-1    OGuardAI-2    OGuardAI-3
              |            |            |
              +------------+------------+
                           |
                Sealed sessions (stateless)
                  Shared revocation via Redis
        (or shared server-side sessions via encrypted Redis)
                           |
                     Python NER (optional)
                    +------+------+
                    |  GLiNER     |
                    |  sidecar    |
                    +-------------+

Stateless Design

OGuardAI is stateless by default using sealed sessions:

Session state travels in the encrypted blob (client-held)
No server-side session storage needed
Any instance can handle any request
Horizontal scaling = add more instances

Components by Statefulness

Component	Stateless?	Multi-Instance Safe?	Notes
Transform pipeline	Yes	Yes	Pure function
Sealed sessions	Yes	Yes	Client-held encrypted blob
Policy engine	Yes	Yes	Loaded from files at startup
Detector (builtin)	Yes	Yes	Compiled regex, no state
Rate limiter	Per-instance	Acceptable	See rate limiting section
Metrics	Per-instance	Correct	Prometheus scrapes each instance
Revocation (file)	Per-instance	Not recommended	Use Redis for multi-instance
Revocation (Redis)	Shared	Yes	Set `revocation_backend: redis` + `GUARDAI_REDIS_URL`
Replay store (memory)	Per-instance	Cold-start window	Per-replica; replay state lost on restart
Replay store (Redis)	Shared	Yes	Set `replay_backend: redis` + `GUARDAI_REDIS_URL`; closes the cross-replica/restart window. The proxy uses the same store via `GUARDAI_REPLAY_BACKEND=redis` (env-configured, no YAML)
Session backend (sealed)	Stateless	Yes	Client-held encrypted blob — no shared state needed
Session backend (Redis)	Shared	Yes	Set `session.backend: redis` + `GUARDAI_REDIS_URL` for server-side sessions the client references by `session_id`; each is encrypted at rest (AES-256-GCM), so Redis holds only ciphertext. Requires `replay_backend: redis` (a shared replay store serializes cross-replica continuations); startup fails closed otherwise. Use `maxmemory-policy noeviction`
Session backend (memory)	Per-instance	No	Pod-local; a session on one replica is invisible to others. Single-instance only
NER sidecar	Shared	Yes	Single sidecar serves all instances

Revocation must be the single shared authority on every replica, or a value revoked on one stays restorable through another. Startup fails closed unless revocation_backend: redis (GUARDAI_REDIS_URL set) whenever a multi-replica topology is advertised: a redis session or replay backend implies it, and the Helm chart derives it from replicaCount. A direct (non-Helm) deployment running sealed (stateless) sessions across replicas carries no such signal, so it MUST advertise HA explicitly with high_availability: true (or GUARDAI_HIGH_AVAILABILITY=true); the proxy uses the same env flag. An invalid flag value aborts startup rather than silently disabling the check.

Recommended Production Setup

Kubernetes (3 instances, sealed sessions, NER)

# guardai-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: oguardai-server
spec:
  replicas: 3
  template:
    spec:
      containers:
        - name: guardai
          image: ghcr.io/oronts/oronts-guardai/oguardai-server:latest
          ports:
            - containerPort: 3000
          env:
            - name: GUARDAI_SESSION_SECRET
              valueFrom:
                secretKeyRef:
                  name: guardai-secrets
                  key: session-secret
          args: ["--config", "/app/oguardai.yaml"]
          volumeMounts:
            - name: config
              mountPath: /app/oguardai.yaml
              subPath: oguardai.yaml
              readOnly: true
            - name: policies
              mountPath: /policies
              readOnly: true
      volumes:
        - name: config
          configMap:
            name: guardai-config
        - name: policies
          configMap:
            name: guardai-policies

oguardai.yaml for distributed deployment

server:
  host: 0.0.0.0
  port: 3000

auth:
  mode: api_key
  api_keys:
    - key: "${GUARDAI_API_KEY}"
      identity: "production-service"
      name: "production-key"
      scopes: ["transform", "rehydrate", "detect"]

session:
  backend: sealed  # Stateless -- recommended for K8s
  secret: "${GUARDAI_SESSION_SECRET}"
  ttl_seconds: 3600
  redis_url: "redis://redis:6379"  # required for shared revocation below

# Shared so a value revoked on one replica is refused by all (only HMAC digests
# are stored). Required for multi-instance; the file backend is per-instance.
revocation_backend: redis

# Shared so a replayed transform continuation is rejected across replicas and
# survives a restart. Memory (default) is per-replica with a cold-start window.
replay_backend: redis

detector:
  mode: builtin  # Or 'both' if NER sidecar is deployed

output_protection:
  enabled: true
  mode: strict

rate_limit:
  enabled: true
  requests_per_second: 100  # Per instance
  burst_size: 200

prompt_security:
  enabled: true
  action: strip

Scaling Guidelines

Workload	Recommended Instances	NER	Notes
Under 100 req/s	1-2	Optional	Single instance sufficient
100-500 req/s	3-5	Separate pod	Horizontal scaling
Over 500 req/s	5+ with HPA	Dedicated NER pool	Auto-scaling

Health Checks

livenessProbe:
  httpGet:
    path: /livez
    port: 3000
  initialDelaySeconds: 5
  periodSeconds: 10

readinessProbe:
  httpGet:
    path: /readyz
    port: 3000
  initialDelaySeconds: 3
  periodSeconds: 5

/livez and /readyz are unauthenticated, safe for K8s probes even with api_key auth enabled. Use /v1/health with X-API-Key for detailed health status.

Distributed Deployment