Operations
Distributed Deployment
Architecture and configuration guide for multi-instance OGuardAI deployments
Architecture Overview
Load Balancer (nginx/ALB/Envoy)
|
+------------+------------+
v v v
OGuardAI-1 OGuardAI-2 OGuardAI-3
| | |
+------------+------------+
|
Redis (optional)
+------+------+
| Sessions |
| Revocations|
+-------------+
|
Python NER (optional)
+------+------+
| GLiNER |
| sidecar |
+-------------+Stateless Design
OGuardAI is stateless by default using sealed sessions:
- Session state travels in the encrypted blob (client-held)
- No server-side session storage needed
- Any instance can handle any request
- Horizontal scaling = add more instances
Components by Statefulness
| Component | Stateless? | Multi-Instance Safe? | Notes |
|---|---|---|---|
| Transform pipeline | Yes | Yes | Pure function |
| Sealed sessions | Yes | Yes | Client-held encrypted blob |
| Policy engine | Yes | Yes | Loaded from files at startup |
| Detector (builtin) | Yes | Yes | Compiled regex, no state |
| Rate limiter | Per-instance | Acceptable | See rate limiting section |
| Metrics | Per-instance | Correct | Prometheus scrapes each instance |
| Revocation (file) | Per-instance | Not recommended | Use Redis for multi-instance |
| Revocation (Redis) | Shared | Yes | All instances share revocation set |
| Session backend (Redis) | Shared | Yes | For server-side sessions |
| NER sidecar | Shared | Yes | Single sidecar serves all instances |
Recommended Production Setup
Kubernetes (3 instances, Redis, NER)
# guardai-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: oguardai-server
spec:
replicas: 3
template:
spec:
containers:
- name: guardai
image: ghcr.io/oronts/oronts-guardai/oguardai-server:latest
ports:
- containerPort: 3000
env:
- name: GUARDAI_SESSION_SECRET
valueFrom:
secretKeyRef:
name: guardai-secrets
key: session-secret
volumeMounts:
- name: config
mountPath: /etc/guardai
readOnly: true
- name: policies
mountPath: /policies
readOnly: true
volumes:
- name: config
configMap:
name: guardai-config
- name: policies
configMap:
name: guardai-policiesoguardai.yaml for distributed deployment
server:
host: 0.0.0.0
port: 3000
auth:
mode: api_key
api_keys:
- name: "production-key"
key: "${GUARDAI_API_KEY}"
scopes: ["transform", "rehydrate", "detect"]
session:
backend: sealed # Stateless -- recommended for K8s
secret: "${GUARDAI_SESSION_SECRET}"
ttl_seconds: 3600
detector:
mode: builtin # Or 'both' if NER sidecar is deployed
output_protection:
enabled: true
mode: strict
rate_limit:
enabled: true
requests_per_second: 100 # Per instance
burst_size: 200
prompt_security:
enabled: true
action: stripScaling Guidelines
| Workload | Recommended Instances | NER | Notes |
|---|---|---|---|
| Under 100 req/s | 1-2 | Optional | Single instance sufficient |
| 100-500 req/s | 3-5 | Separate pod | Horizontal scaling |
| Over 500 req/s | 5+ with HPA | Dedicated NER pool | Auto-scaling |
Health Checks
livenessProbe:
httpGet:
path: /v1/health
port: 3000
initialDelaySeconds: 5
periodSeconds: 10
readinessProbe:
httpGet:
path: /v1/health
port: 3000
initialDelaySeconds: 3
periodSeconds: 5