OGuardAI
Guides

OpenAI Drop-In Proxy

Point your provider SDK at the proxy, set the base URL and the proxy auth header, and get PII protection without changing your prompts or application logic

The Killer Use Case

Point your provider SDK at the proxy: set the base URL and add the proxy auth header. Your prompts, models, and application logic stay the same.

Before (unprotected)

client = OpenAI(api_key="sk-...")
response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Customer Erika Mustermann (erika.mustermann@example.com) needs help"}]
)
# Erika's name and email sent directly to OpenAI

After (protected by OGuardAI proxy)

The proxy runs on port 8081 by default (separate from the OGuardAI server on port 3000 or 8080). Point your SDK's base_url at the proxy:

client = OpenAI(
    api_key="sk-...",
    base_url="http://localhost:8081/v1",  # Point to OGuardAI proxy (port 8081)
    default_headers={"X-GuardAI-API-Key": os.environ["GUARDAI_PROXY_API_KEY"]},
)
response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Contact erika.mustermann@example.com or call +49 30 12345678"}]
)
# OpenAI sees: "Contact {{email:e_001:1bcaef1a4aff}} or call {{phone:ph_001:71453119be53}}"
# Response automatically restored with original values

How It Works

  1. Your app sends request to OGuardAI proxy (port 8081)
  2. Proxy masks PII in all user/assistant messages
  3. Proxy forwards to real provider API
  4. Provider responds with tokens (e.g., "{{email:e_001:1bcaef1a4aff}}")
  5. Proxy restores tokens to real values
  6. Your app receives the response with original values

Detection scope: By default (GUARDAI_DETECTOR_MODE unset, or builtin) the proxy uses builtin regex detectors and covers emails, phones, IPs, IBANs, SSNs, VAT IDs, URLs, credit cards, and other structured patterns, but NOT person, company, or location names. To also mask names, run the Python NER sidecar and start the proxy with GUARDAI_DETECTOR_MODE=both (builtin plus NER, regex fallback if the sidecar is down) or GUARDAI_DETECTOR_MODE=advanced (NER required, fail closed on a sidecar outage) and GUARDAI_DETECTOR_ADVANCED_URL=http://localhost:9090. In the default builtin mode a raw name can reach the model, so enable NER for name-bearing traffic.

Inline images: The proxy OCR-scans inline image bytes (OpenAI image_url data: URIs and Anthropic base64 image blocks). If the image renders detectable PII or cannot be scanned, the request is rejected, so PII inside an image never reaches the LLM. Clean images are forwarded unchanged. OCR requires Tesseract on the proxy host (apt install tesseract-ocr); without it, inline images are rejected fail-closed rather than forwarded unscanned. The maximum decoded image size is 5 MiB, which GUARDAI_PROXY_INLINE_IMAGE_MAX_BYTES can only lower. Remote image URLs are not fetched: the proxy cannot OCR-scan the pixels behind a URL, so by default a remote image URL is rejected fail-closed. An operator can opt in with GUARDAI_PROXY_ALLOW_REMOTE_IMAGE_URLS to forward remote URLs after a URL-string PII check only (the image pixels are still not scanned).

Generated media (model output): The proxy protects text, so it fails closed on any model output it cannot scan. If the model returns audio, images, documents, or an inline data: base64 payload that sits outside a text field, the response is refused with 422 GUARDAI_OUTPUT_MEDIA_REJECTED; an output shape the proxy has no scanner for, such as a structured content array or an unrecognized block, is refused with 422 GUARDAI_OUTPUT_UNSUPPORTED. The proxy cannot verify the PII inside generated pixels or audio, so it does not forward them unscanned. A request that asks the upstream for a non-text output modality (modalities, audio) is refused before the upstream is called. Text output is unaffected and byte-identical.

Other content blocks: Any non-text content block the proxy cannot scan (an Anthropic document block, an OpenAI file or input_audio block, or any unrecognized block type) is rejected fail-closed instead of being forwarded to the LLM unscanned. To send a document through AI safely, extract its text first and route it through the server /v1/transform API, which has a sandboxed document and PDF ingestion pipeline.

Provider credentials: The proxy forwards the upstream provider auth and routing headers (Authorization, x-api-key, OpenAI-Organization, OpenAI-Project, anthropic-version, anthropic-beta) to the provider unchanged, because they are required to authenticate and route the call. These are credential and routing values, not request data, so they are intentionally never scanned and never logged. Your OGuardAI proxy key (X-GuardAI-API-Key) authenticates you to the proxy and is never forwarded upstream.

Setup

Availability: The proxy image used in the docker run below is private and provided under a written agreement with Oronts. It is not a public pull; request access before setup at /request.

# Generate one proxy key; the proxy and every client must present the same value.
export GUARDAI_PROXY_API_KEY=$(openssl rand -hex 32)

# Start the OGuardAI proxy (with auth) from the published image, inheriting the exported key with a
# valueless -e. --network host serves it on the host's localhost:8081.
docker run --rm --network host \
  -e GUARDAI_SESSION_SECRET=$(openssl rand -base64 32) \
  -e GUARDAI_PROXY_API_KEY \
  ghcr.io/oronts/oguardai/proxy:0.1.0 \
  --target https://api.openai.com --policy default --port 8081 \
  --single-tenant --rate-limit-enabled

Clients must include the X-GuardAI-API-Key: <your-proxy-key> header. The upstream provider key (sk-...) is passed via the standard Authorization header by the OpenAI SDK.

Client Setup

Run the client in the same shell where you exported GUARDAI_PROXY_API_KEY above, so the proxy and the client present the same key:

from openai import OpenAI
import os

client = OpenAI(
    base_url="http://localhost:8081/v1",
    default_headers={"X-GuardAI-API-Key": os.environ["GUARDAI_PROXY_API_KEY"]},
)

Streaming Support

Streaming works transparently:

stream = client.chat.completions.create(
    model="gpt-4",
    messages=[...],
    stream=True  # Streaming works through proxy
)
for chunk in stream:
    print(chunk.choices[0].delta.content, end="")

Streaming budgets

On top of the per-chunk idle timeout, the proxy enforces three per-stream lifetime budgets on both the OpenAI-compatible and the Anthropic streaming paths. They bound a stream that keeps sending timely chunks the idle timeout never catches, or that never ends.

Env varFlagDefaultPurpose
GUARDAI_PROXY_STREAM_MAX_DURATION_SECS--stream-max-duration3600Total wall-clock deadline, in seconds, for one streaming response
GUARDAI_PROXY_STREAM_MAX_TOTAL_BYTES--stream-max-total-bytes67108864 (64 MiB)Total received-byte cap for one streaming response
GUARDAI_PROXY_STREAM_MAX_CONCURRENT--stream-max-concurrent100Maximum concurrent in-flight streams

Set an axis to 0 to disable it (unlimited). When a stream breaches the duration or byte cap mid-flight, the proxy flushes any already-scanned held-back content, then ends the stream with a terminal SSE error event of type GUARDAI_STREAM_BUDGET_EXCEEDED. No partial or raw content is forwarded past the cap. When the concurrency cap is saturated, a new streaming request is rejected after auth with 503 and a retry-after: 1 header. A non-dev proxy that disables all three axes logs a startup warning.

A stream that starts emitting generated media the proxy cannot scan (an audio or image chunk, or an unscannable content block) is ended the same way: the proxy flushes any already-scanned held-back content, then emits one terminal SSE error event of type GUARDAI_OUTPUT_MEDIA_REJECTED or GUARDAI_OUTPUT_UNSUPPORTED with no data: [DONE], so no media bytes reach the client.

Streaming terminal events

A streamed response is a 200 the moment its first byte leaves, so the proxy cannot withhold it the way the buffered path does. Its only truthful channel after the status line is the terminal event. When an upstream event cannot be protected, the proxy flushes any already-scanned held-back content and then ends the stream with a terminal SSE error event instead of a clean data: [DONE] (OpenAI) or message_stop (Anthropic). No unscanned or partial content is forwarded.

Terminal event typeReason labelWhen
GUARDAI_STREAM_UNPROCESSABLEmalformed_eventA data line, terminal, or passthrough payload was not valid JSON
GUARDAI_STREAM_UNPROCESSABLEambiguous_eventOne event carried multiple content choices, content beside a refusal, content beside a tool call, or multiple tool-argument sources
GUARDAI_STREAM_UNPROCESSABLEunsupported_deltaAn Anthropic content-block delta of a type the pipeline cannot protect
GUARDAI_STREAM_UNPROCESSABLEunknown_eventAn Anthropic event name outside the documented allowlist
GUARDAI_STREAM_UNPROCESSABLEincomplete_tool_blockA tool-argument block was still open when the terminal arrived
GUARDAI_STREAM_UNPROCESSABLEbuffer_limitA tool-argument accumulation or SSE line buffer grew past its configured bound
GUARDAI_STREAM_TRUNCATED(none)The upstream ended before its completion terminal (disconnect or transport error)
GUARDAI_OUTPUT_BLOCKED(none)The output guard judged streamed model output and blocked it: newly generated PII in tool arguments, or a policy denial
GUARDAI_OUTPUT_GUARD_UNAVAILABLE(none)The output guard could not complete a scan or restore of streamed model output, so the content was never judged. Covers the text channel, an extended-thinking block, tool arguments, and passthrough events (a detector or NER outage, a sibling scan that failed closed, or a token the session could not rehydrate)

The OpenAI-shaped payload is {"error":{"message":"<reason>","type":"GUARDAI_STREAM_UNPROCESSABLE","code":"stream_unprocessable"}}; the Anthropic-shaped payload is {"type":"error","error":{"type":"GUARDAI_STREAM_UNPROCESSABLE","message":"<reason>"}}. The two output guard terminals reuse the buffered path's codes and the same shape: OpenAI carries "code":"output_blocked" or "code":"output_guard_unavailable". A guard block and a guard outage are distinct terminals: a block means the model output was judged and refused, an outage means the scan could not run and the same request may succeed once the deployment is fixed. The OpenAI and Anthropic SDKs raise on a terminal error event, so an SDK caller sees an exception rather than a truncated-but-clean response.

Scope of v1 streaming. The proxy protects a stream shaped like the first-party OpenAI and Anthropic dialects: n=1 (streaming refuses any other n before the upstream call), one content choice per event, and one tool call per event. A third-party OpenAI-compatible backend that batches multiple choices or several tool calls into a single event, or an event that mixes content and a tool call, cannot be guarded per item by the single per-stream pipeline, so it ends the stream with an ambiguous_event error rather than forwarding a lossy result. Route those workloads through the buffered (non-streaming) path. Because an undocumented Anthropic event name faults, a genuinely new provider event type requires an allowlist update in the proxy.

Raw-SSE consumers. A client that reads the raw SSE stream itself, rather than through an SDK, must treat event: error as terminal in addition to data: [DONE] / message_stop. A stream that ends on event: error was not completed; the fault is recorded on the audit trail either way, and the specific reason is carried in the error event, not the audit record.

What Gets Protected

Message RoleScanned?
systemYes
userYes
assistantYes
tool callsYes (function arguments)
tool resultsYes

Readiness and coverage

The proxy answers two operational endpoints locally instead of forwarding them upstream:

  • GET /readyz (unauthenticated) reports dependency health for a load balancer. It returns 503 when the proxy has no policy loaded, when a store it depends on is unhealthy, or, in both/advanced detector mode, when the NER sidecar is unreachable, so a deployment that would silently fall back to regex-only detection stops receiving traffic. In builtin mode it never probes a sidecar.
  • GET /v1/capabilities (authenticated) reports the detector's EFFECTIVE coverage: {covered_entity_types, ner_active, detector_mode}, plus protection_overrides (a list of the enabled override ids, omitted when none is enabled). Built-in format types (email, phone, IBAN, and so on) are always covered; person, company, and location appear only when NER is active, so a consumer can tell whether names are masked before sending traffic. The NER triple assumes a standard backend (the shipped GLiNER and spaCy backends both emit it); do not pair a subset backend (for example the none backend, which emits nothing) with both/advanced mode, since capabilities would then advertise names it does not detect.

Anthropic Support

Same pattern for Anthropic:

import os
client = Anthropic(
    api_key="sk-ant-...",
    base_url="http://localhost:8081",
    default_headers={"X-GuardAI-API-Key": os.environ["GUARDAI_PROXY_API_KEY"]},
)

OpenAI-compatible providers (Mistral and others)

Any provider that speaks the OpenAI /v1/chat/completions wire format is protected by the same handler. Point the proxy target at the provider and use it as usual. For Mistral:

docker run --rm --network host \
  -e GUARDAI_SESSION_SECRET=$(openssl rand -base64 32) -e GUARDAI_PROXY_API_KEY \
  ghcr.io/oronts/oguardai/proxy:0.1.0 \
  --target https://api.mistral.ai --policy default --port 8081 --single-tenant --rate-limit-enabled
# or pass the target as an env var: -e GUARDAI_PROXY_TARGET=https://api.mistral.ai
# GUARDAI_PROXY_API_KEY is inherited from the value exported in Setup above.

For a gateway or aggregator, point the target at the origin plus any prefix that sits before the route the proxy appends. The proxy forwards the client's full path, which begins with /v1, so the target is https://openrouter.ai/api and https://api.groq.com/openai, not a URL that already ends in /v1. A target that ends in the mounted route's first segment, or that carries a percent sign, a query string, or a fragment, is refused at startup with the reason to fix.

The client sends its normal Mistral request to http://localhost:8081/v1/chat/completions with its Mistral API key in the Authorization: Bearer header (forwarded unchanged to the upstream). Message content, tool-call arguments, tool/function definitions, top-level user, and metadata are tokenized before the request leaves the runtime; buffered and streaming responses are rehydrated.

Restored tool-call arguments are redacted by default. The proxy cannot verify that the client's tool runs inside the trust boundary, so it does not return raw values into tool_calls arguments. A deployment whose tools are trusted and in-boundary opts into raw restoration with GUARDAI_PROXY_TOOL_PAYLOAD_RAW=true (or --tool-payload-raw); this flag is the only lever, a policy rule cannot widen it back to raw. It stays a production option (warned at startup, never gated), and when enabled it is surfaced on the /v1/capabilities protection_overrides list and the startup log.

Mistral-specific request fields are handled correctly: prefix completion uses a boolean prefix flag on the last assistant message and the prefix text lives in content, which is scanned like any other message content; safe_prompt (boolean) and random_seed (number) carry no PII and pass through unchanged.

Scope: only /v1/chat/completions (and Anthropic /v1/messages) are intercepted and protected. /v1/embeddings is protected through the embedding path. Other Mistral endpoints such as /v1/fim/completions are not supported and are rejected at the proxy boundary (fail-closed, not forwarded), so raw PII never reaches an unprotected endpoint. Use the server /v1/transform API for those payloads.

Azure OpenAI

Azure uses a different URL scheme (/openai/deployments/{deployment}/chat/completions?api-version=...) and an api-key header instead of a Bearer token, so it needs an explicit provider mode. Set --provider azure (or GUARDAI_PROXY_PROVIDER=azure) and point the target at your Azure resource:

docker run --rm --network host \
  -e GUARDAI_SESSION_SECRET=$(openssl rand -base64 32) -e GUARDAI_PROXY_API_KEY \
  ghcr.io/oronts/oguardai/proxy:0.1.0 \
  --provider azure \
  --target https://my-resource.openai.azure.com \
  --policy default --port 8081 --single-tenant --rate-limit-enabled

Clients send their normal Azure request to http://localhost:8081/openai/deployments/{deployment}/chat/completions?api-version=YYYY-MM-DD with their api-key header. The proxy tokenizes the request through the same OpenAI-compatible pipeline, preserves the deployment path and api-version query when forwarding, and forwards the api-key (and Authorization, for Entra ID) header unchanged. Azure embeddings (/openai/deployments/{deployment}/embeddings) are protected through the embedding path. The provider mode is never auto-detected from the target URL: you must set it explicitly.

Vertex AI (OpenAI-compatible)

Vertex AI exposes an OpenAI-compatible chat endpoint at /v1beta1/projects/{project}/locations/{location}/endpoints/{endpoint}/chat/completions (the endpoint segment is openapi for managed Gemini, or your endpoint id for a self-deployed model) with an OAuth2 Bearer token. Set --provider vertex-openai and target your regional aiplatform host:

docker run --rm --network host \
  -e GUARDAI_SESSION_SECRET=$(openssl rand -base64 32) -e GUARDAI_PROXY_API_KEY \
  ghcr.io/oronts/oguardai/proxy:0.1.0 \
  --provider vertex-openai \
  --target https://us-central1-aiplatform.googleapis.com \
  --policy default --port 8081 --single-tenant --rate-limit-enabled

The client sends its normal Vertex request to the same path on the proxy with its Authorization: Bearer <token> header. The proxy tokenizes the request through the shared OpenAI-compatible pipeline, preserves the full project/location/endpoint path when forwarding, and forwards the Bearer token unchanged. Only this OpenAI-compatible endpoint is protected; the native :generateContent API uses a different body schema and is not yet supported (use the server /v1/transform API for those payloads).

Bedrock

Amazon Bedrock is not supported by the transparent proxy. Bedrock signs each request with AWS SigV4 over a hash of the request body, so tokenizing the body would invalidate the signature; protecting Bedrock would require the proxy to hold AWS credentials and re-sign every request. --provider bedrock therefore refuses to start with an explanation. Use the server /v1/transform API for Bedrock payloads.

Configuration

The proxy respects all OGuardAI policy settings. Select a policy by adding --policy <name> to the Setup command above; a builtin-only policy such as default needs nothing else.

strict-pii (blocks SSN/IBAN/passport and masks person and company) and german-support (German formal salutations) declare person detection as required, so the proxy refuses to start them in the default builtin detector mode. Run the NER sidecar and start the proxy in advanced mode:

# NER sidecar, bound to loopback (it runs unauthenticated, so keep it on 127.0.0.1)
docker run -d --rm --network host -e GUARDAI_DETECTOR_ALLOW_INSECURE=true \
  --name oguardai-detector ghcr.io/oronts/oguardai/detector-py:0.1.0 \
  --host 127.0.0.1 --port 9090

# Proxy in advanced (NER-required) mode with the strict-pii policy. GUARDAI_PROXY_API_KEY is inherited
# from the value exported in Setup above.
docker run --rm --network host \
  -e GUARDAI_SESSION_SECRET=$(openssl rand -base64 32) -e GUARDAI_PROXY_API_KEY \
  -e GUARDAI_DETECTOR_MODE=advanced -e GUARDAI_DETECTOR_ADVANCED_URL=http://localhost:9090 \
  ghcr.io/oronts/oguardai/proxy:0.1.0 \
  --target https://api.openai.com --policy strict-pii --port 8081 --single-tenant --rate-limit-enabled

For german-support, run the same two commands with --policy german-support.

Detection and tenancy are configured through environment variables:

Env varDefaultPurpose
GUARDAI_DETECTOR_MODEbuiltinbuiltin (regex only), both (regex + NER best-effort), or advanced (NER required, fails closed)
GUARDAI_DETECTOR_ADVANCED_URLhttp://localhost:9090URL of the Python NER detector service
GUARDAI_DETECTOR_API_KEY-Shared secret for the proxy-to-detector hop (X-Detector-API-Key). When set it must be at least 16 characters and not a placeholder value, refused at startup outside dev
GUARDAI_DETECTOR_LANGUAGEautoLanguage hint forwarded to detection (ISO 639-1) when requests are known to be single-language
GUARDAI_DETECTOR_CUSTOM-JSON array of custom detector patterns (same shape as a policy's custom_patterns), so the proxy detects deployment-specific identifiers
GUARDAI_PROXY_TENANT_ID-Tenant id bound to every session the proxy creates, isolating this proxy's sessions in multi-tenant deployments. When set: non-empty, at most 128 bytes, and no whitespace or control characters. A malformed value fails the proxy startup
GUARDAI_PROXY_METRICS_API_KEY-Optional dedicated credential for the GET /metrics scrape (the Observability role). When set it must differ from every ingress key and requires the ingress key to be set; either rule fails startup. With it set, the ingress key serves only the data plane (chat, /v1/embeddings, /v1/capabilities) and this key serves only /metrics, carrying no tenant. Unset, the ingress key scrapes /metrics as before, so a single-key deployment is unchanged. Least-privilege for the scrape, not an exfiltration control: a leaked ingress key cannot call the upstream or read restored output. When set it must be at least 16 characters and not a placeholder value, refused at startup outside dev
GUARDAI_PROXY_API_KEY_NEXT-Optional second accepted ingress key for a zero-downtime rotation overlap. It shares the tenant, rate bucket, and audit attribution of GUARDAI_PROXY_API_KEY; only the key differs. When set it must differ from the primary ingress key and from any metrics key, and requires the primary to be set; either rule fails startup. While set, GET /metrics gains a proxy-local guardai_proxy_ingress_auth_total{key_slot="current"|"next"} counter so an operator can watch the retiring key drain to zero. Unset, one ingress key is accepted and /metrics is unchanged. See the key rotation runbook. When set it must be at least 16 characters and not a placeholder value, refused at startup outside dev
GUARDAI_PROXY_SYSTEM_PREAMBLEfalseOpt-in: inject the prompt-security system preamble into forwarded chats (off by default so token usage is unchanged)

For a multi-replica (HA) proxy, share the request-replay store across replicas with GUARDAI_REPLAY_BACKEND=redis + GUARDAI_REDIS_URL (default memory is per-replica, so a replayed continuation could otherwise slip through on another replica). An unknown backend or an unreachable Redis fails the proxy startup closed.

A multi-replica proxy MUST also share the revocation store, or a value revoked on the server or another replica stays restorable through this ingress. Set GUARDAI_REVOCATION_BACKEND=redis + GUARDAI_REDIS_URL (the same Redis the server uses) and the same GUARDAI_SESSION_SECRET as the server: revocation digests are keyed to that secret, and the shared store binds to it on first use and refuses to start a process presenting a different secret, so a mismatch fails closed rather than silently splitting the two into separate revocation sets. Advertising HA with GUARDAI_REPLAY_BACKEND=redis or GUARDAI_HIGH_AVAILABILITY=true while leaving revocation on the per-replica memory default fails the proxy startup closed.

Rate Limiting

A non-dev proxy must run a bounded request limiter: startup is refused when it is off unless GUARDAI_PROXY_DEV=1 is set, which is a local dev opt-out. Enable the per-client token-bucket limiter with GUARDAI_PROXY_RATE_LIMIT_ENABLED=true (or --rate-limit-enabled; the env var wins over the flag). Tune GUARDAI_PROXY_RATE_LIMIT_RPS (default 100) and GUARDAI_PROXY_RATE_LIMIT_BURST (default 200). Enabling it with an rps or burst below 1, or a non-numeric value, aborts startup. When the limit is exceeded the proxy returns 429 with Retry-After: 1 in the caller's provider error shape.

The limiter runs in two layers. A per-client-IP layer runs ahead of authentication and charges only failed authentication, so a flood of invalid X-GuardAI-API-Key values is answered with 429 before it costs key comparison or log volume, while a valid caller and a relayed upstream 401 never spend its budget. Tune it with GUARDAI_PROXY_AUTH_RATE_LIMIT_RPS (default 5) and GUARDAI_PROXY_AUTH_RATE_LIMIT_BURST (default 10); set the rps to 0 to disable this layer, or with GUARDAI_PROXY_RATE_LIMIT_ENABLED on, a positive rps with a zero burst aborts startup; with the switch off the limiter does not run, but a malformed rps or burst value is still rejected at startup. The per-tenant layer runs after authentication and caps the overall request rate with the GUARDAI_PROXY_RATE_LIMIT_RPS/_BURST knobs above. Both resolve the client IP with the same trusted-proxy rules described below, and every 429 increments guardai_rate_limit_rejections_total on GET /metrics.

Requests are bucketed by the authenticated tenant when auth is enabled. Without a tenant the bucket key is the client IP. By default (GUARDAI_PROXY_RATE_LIMIT_TRUSTED_PROXY_DEPTH=0) that IP is the TCP peer address, which a client cannot forge. If the proxy runs behind one or more reverse proxies that set X-Forwarded-For, set GUARDAI_PROXY_RATE_LIMIT_TRUSTED_PROXY_CIDRS (or --rate-limit-trusted-proxy-cidrs) to the proxy address range; the client IP is then resolved by a rightmost-trusted walk over the header. The depth flag alone is refused outside a dev proxy (GUARDAI_PROXY_DEV=1), because a direct peer could forge the header and dodge the rate limit. Leaving both unset behind an L7 proxy shares one bucket across all clients, so set the trusted CIDRs. Behind a shared NAT or corporate egress the per-client-IP layer groups distinct clients into one bucket, but it counts only their failed authentications, so legitimate traffic from that address is not throttled while the address still has failed-auth budget. Once a shared address has exhausted that budget, every request from it is shed with 429 before authentication, valid keys included, until the budget refills at the configured rate. Under a concurrent flood a handful of requests beyond the burst can still reach the key compare; the per-second rate bounds the long-run cost. Set the trusted CIDRs to separate the failed-auth budgets if several clients share the egress.

How Token Restoration Works

When the LLM generates output containing builtin-detected tokens like {{email:e_001:1bcaef1a4aff}}, the proxy automatically restores them using the session state that was created during the request transformation. The restore mode (full, partial, masked, formatted, abstract) is controlled by the policy configuration (none removes the value entirely).

For example, with builtin detection:

  • Input: "Contact erika.mustermann@example.com or IP 10.0.0.1 for help"
  • To provider: "Contact {{email:e_001:1bcaef1a4aff}} or IP {{ip:ip_001:cd9903b926e3}} for help"
  • From provider: "I'll reach out to {{email:e_001:1bcaef1a4aff}}..."
  • To your app: "I'll reach out to erika.mustermann@example.com..."

In the default builtin mode the proxy covers structured patterns only (emails, phones, IPs, IBANs, SSNs, etc.). To also mask person/company/location names, run the proxy with the NER sidecar (GUARDAI_DETECTOR_MODE=both or advanced, see Configuration), or use the full server API.

Architecture

Your App
  |
  | (standard OpenAI SDK calls)
  v
OGuardAI Proxy (port 8081)
  |
  | 1. Transform: mask PII in request
  | 2. Forward: send safe request to OpenAI
  | 3. Rehydrate: restore tokens in response
  |
  v
OpenAI API (never sees real PII)

Limitations

  • The proxy adds a small latency overhead for PII detection and token restoration
  • All message roles (including system) are scanned for PII
  • The proxy requires network access to both your app and the target API
  • Session state is per-request; multi-turn conversations need session continuity (the proxy handles this automatically via sealed session blobs)
  • The proxy scans text output only. Generated media (audio, images, documents), an inline data: payload outside a text field, and structured output shapes such as a content array are refused rather than forwarded, because their bytes cannot be scanned for PII
  • A bare base64 blob carried in a vendor extension field with no media declaration is governed by the text scanning posture, not by the generated-media refusal, so it is treated as opaque text rather than refused

Troubleshooting

Tokens not being restored: Check that the LLM is faithfully reproducing the token format {{type:id:cap}}. OGuardAI includes a 3-stage token repair pipeline (strict, repair, fuzzy) that handles common LLM token mangling.

Performance: In builtin (regex) mode the proxy adds only a small detection and restoration overhead on short inputs; enabling the NER sidecar (both or advanced) adds more. Measure it against your own inputs and hardware rather than assuming a fixed figure. For large inputs, consider using the chunking API directly.

Runnable examples

Three end-to-end examples live in the repo examples/ directory. Each one points a stock provider SDK at the proxy and changes nothing else about the calling code. Every example ships two run modes:

  • Explain (the default): calls the same runtime contract the proxy uses internally (/v1/transform and /v1/rehydrate) directly against the OGuardAI server, with no LLM call and no provider key. It uses only the language standard library, so it runs with nothing installed beyond the server. It prints what leaves your network, a leak check, the safe metadata the model receives, and the full restore lattice.
  • Live (--live): the stock provider SDK pointed at the proxy. The only change from a normal integration is the base URL and the X-GuardAI-API-Key header.

Jump to:

OpenAI Python, customer support desk

A support desk for Musterfirma (a fictional US online outdoor retailer) asks an OpenAI model to draft the reply to a customer's order email, while the customer's name, email, phone, address, customer number, order number, and bank IBAN never leave the network in the clear. Directory: examples/openai-proxy-python/. Policy: customer-support-en.

Prerequisite (needs-NER): a running OGuardAI server. This example masks the person name and postal address, which needs the NER detector sidecar; the bundled Docker stack starts in both mode (builtin regex plus NER). customer-support-en declares person and company detection as required, so a server without the NER sidecar rejects the transform (fail closed) rather than let an un-tokenized name reach the model.

Start the runtime, then run the offline explain path (no OpenAI key, no proxy, no pip install):

# Start the OGuardAI runtime (server + NER detector). NER is needed for person/address. The bundled
# stack requires these four secrets, and its client key ships least-privilege, so widen it to the
# customer-support-en policy and the channels this walkthrough restores on (user_output,
# internal_summary).
export GUARDAI_SESSION_SECRET=$(openssl rand -base64 32)
export GUARDAI_API_KEY_1=$(openssl rand -hex 32)
export GUARDAI_DETECTOR_API_KEY=$(openssl rand -hex 32)
export GUARDAI_REDIS_PASSWORD=$(openssl rand -hex 32)
export GUARDAI_API_KEY_1_ALLOWED_POLICIES=default,default-ner-required,german-support,customer-support-en
export GUARDAI_API_KEY_1_ALLOWED_OUTPUT_CHANNELS=user_output,internal_summary
docker compose -f deploy/docker/docker-compose.yml up -d

# Run the walkthrough. No OpenAI key, no proxy, no pip install required.
export GUARDAI_URL=http://localhost:3000
export GUARDAI_API_KEY="$GUARDAI_API_KEY_1"   # the bundled stack runs api_key auth
python main.py

The email the example reads (examples/openai-proxy-python/order_issue.txt):

From: erika.mustermann@example-mail.com
To: support@example.com
Subject: Wrong size delivered, order MO-2026-88214

Hello Musterfirma Support,

my name is Ms. Erika Mustermann (customer number CUST-583920). I ordered the
Ridgeline down jacket in medium, but order MO-2026-88214 arrived in extra small
and I would like to exchange it for the correct size.

You can reach me by email at erika.mustermann@example-mail.com or by phone at
+1 (415) 555-0182. My delivery address is 1420 Example Street, Portland, OR 97205.

If a refund turns out to be easier than an exchange, please send it to my bank
account, IBAN GB29 NWBK 6016 1331 9268 19.

Thanks for your help,
Erika Mustermann

Scenario constants, the system prompt (which names no customer data), and the custom order-id pattern that declares this tenant's order scheme to the detector (examples/openai-proxy-python/main.py):

POLICY = "customer-support-en"
LANGUAGE = "en"
HERE = Path(__file__).resolve().parent
ORDER_FILE = HERE / "order_issue.txt"

# System instruction the agent gives the model. It names no customer data: the model works entirely
# from tokens plus the metadata OGuardAI attaches to each one.
SYSTEM_PROMPT = (
    "You are a customer-support assistant for Musterfirma. Write a short, courteous reply "
    "in English that confirms the order, acknowledges the wrong-size delivery, and explains the "
    "exchange or refund next steps. The message contains {{type:id:cap}} placeholders that stand for "
    "real data you must not see. Copy every placeholder verbatim; never decode, translate, "
    "reformat, or invent one, and never write a real name, email, phone number, or IBAN yourself. "
    "Do not write a salutation word before a person placeholder: the runtime inserts the correct "
    "Mr./Ms. honorific itself. Open with 'Hello {{person:...}},'."
)

STRUCTURED_PII = {
    "customer email": "erika.mustermann@example-mail.com",
    "support email": "support@example.com",
    "phone": "+1 (415) 555-0182",
    "customer id": "CUST-583920",
    "order": "MO-2026-88214",
    "iban": "GB29 NWBK 6016 1331 9268 19",
}
NER_PII = {
    "name": "Erika Mustermann",
    "address": "1420 Example Street",
}

# The built-in order detector knows common prefixes, not this tenant's scheme, so the order id
# would reach the model raw. A custom pattern is how a deployment declares its own identifiers.
CUSTOM_PATTERNS = [
    {"entity_type": "order_ref", "pattern": r"\bMO-\d{4}-\d{5}\b", "confidence": 0.99}
]

The explain path drives the runtime through a thin client that posts to /v1/transform and /v1/rehydrate (examples/openai-proxy-python/og_runtime.py):

def transform(
    text: str,
    policy: str,
    language: str | None = None,
    context: dict[str, Any] | None = None,
    custom_patterns: list[dict[str, Any]] | None = None,
) -> dict[str, Any]:
    """Detect + tokenize `text` under `policy`. Returns safe_text, session_state, entity_context."""
    payload: dict[str, Any] = {"input": text, "policy": policy}
    if language:
        payload["language"] = language
    if context:
        payload["context"] = context
    if custom_patterns:
        payload["custom_patterns"] = custom_patterns
    return _post("/v1/transform", payload)


def rehydrate(
    output: str,
    session_state: str,
    *,
    output_channel: str | None = None,
    restore_mode: str | None = None,
    restore_overrides: dict[str, str] | None = None,
) -> dict[str, Any]:
    """Restore tokens in `output` under the session, governed by channel / restore mode."""
    payload: dict[str, Any] = {"output": output, "session_state": session_state}
    if output_channel:
        payload["output_channel"] = output_channel
    if restore_mode:
        payload["restore_mode"] = restore_mode
    if restore_overrides:
        payload["restore_overrides"] = restore_overrides
    return _post("/v1/rehydrate", payload)

Run it live against the real proxy and OpenAI. The live proxy pins the SAME policy the explain path uses (customer-support-en), which requires NER for the person and company, so the proxy runs in advanced mode with the NER sidecar up. The proxy also takes the tenant order pattern through the GUARDAI_DETECTOR_CUSTOM env var, because the SDK request cannot carry the per-request custom patterns the explain path passes to /v1/transform. With both in place, live and explain protect the same values:

pip install -r requirements.txt   # OpenAI SDK, only needed for --live

# This example masks person names (and exposes gender/formality metadata), which needs NER, so run
# the detector in the host network namespace and put the proxy in `advanced` mode (NER required, fail
# closed on a sidecar outage). `customer-support-en` declares person and company detection as
# required, so the proxy refuses to start in `builtin` or `both` mode. `--host 127.0.0.1` keeps the
# unauthenticated detector bound to loopback, reachable by the proxy on the same host but not exposed
# on any other interface.
docker run -d --rm --network host -e GUARDAI_DETECTOR_ALLOW_INSECURE=true \
  --name oguardai-detector ghcr.io/oronts/oguardai/detector-py:0.1.0 \
  --host 127.0.0.1 --port 9090   # NER sidecar bound to loopback :9090

# Start the proxy in front of OpenAI, from the published image (the release archive ships the CLI and
# server only, not a proxy binary). --network host lets the proxy reach the detector on
# localhost:9090 and serve on the host's localhost:8081. GUARDAI_PROXY_API_KEY is the value the client
# sends as X-GuardAI-API-Key; the SDK's OPENAI_API_KEY bearer token is forwarded upstream unchanged.
# --policy pins the same policy the explain path uses, and GUARDAI_DETECTOR_CUSTOM declares the tenant
# order id pattern the explain path passes per request, so the proxy masks it too.
export GUARDAI_SESSION_SECRET=$(openssl rand -base64 32)
export GUARDAI_PROXY_API_KEY=$(openssl rand -hex 32)
export GUARDAI_DETECTOR_MODE=advanced
export GUARDAI_DETECTOR_ADVANCED_URL=http://localhost:9090
export GUARDAI_DETECTOR_CUSTOM='[{"entity_type":"order_ref","pattern":"\\bMO-\\d{4}-\\d{5}\\b","confidence":0.99}]'
docker run --rm --network host \
  -e GUARDAI_SESSION_SECRET -e GUARDAI_PROXY_API_KEY \
  -e GUARDAI_DETECTOR_MODE -e GUARDAI_DETECTOR_ADVANCED_URL -e GUARDAI_DETECTOR_CUSTOM \
  ghcr.io/oronts/oguardai/proxy:0.1.0 \
  --port 8081 --target https://api.openai.com --policy customer-support-en \
  --single-tenant --rate-limit-enabled

# In another shell, point the SDK at the proxy and run for real.
export OPENAI_API_KEY="sk-..."             # your real OpenAI key, read from the environment
export GUARDAI_PROXY_API_KEY="<same-as-above>"
export GUARDAI_MODEL="gpt-4o-mini"          # optional, via OPENAI_MODEL
python main.py --live

The only change from a normal OpenAI integration (examples/openai-proxy-python/main.py, run_live):

def run_live(order: str) -> int:
    # Imported lazily so the explain path needs no third-party packages.
    try:
        from openai import OpenAI
    except ImportError:
        print("Live mode needs the OpenAI SDK: pip install openai", file=sys.stderr)
        return 2

    proxy_url = os.environ.get("GUARDAI_PROXY_URL", "http://localhost:8081") + "/v1"
    proxy_key = os.environ.get("GUARDAI_PROXY_API_KEY", "")
    if not os.environ.get("OPENAI_API_KEY"):
        print("Live mode needs OPENAI_API_KEY (forwarded by the proxy to OpenAI).", file=sys.stderr)
        return 2

    # The only change from a normal OpenAI integration: base_url points at the proxy, and the
    # proxy's own auth key rides in X-GuardAI-API-Key. The SDK still sends OPENAI_API_KEY as its
    # bearer token, which the proxy forwards upstream. Everything else is stock OpenAI.
    client = OpenAI(
        base_url=proxy_url,
        default_headers={"X-GuardAI-API-Key": proxy_key},
    )
    model = os.environ.get("OPENAI_MODEL", "gpt-4o-mini")

    print(RULE)
    print(f"LIVE MODE: OpenAI SDK -> OGuardAI proxy ({proxy_url}) -> OpenAI ({model})")
    print(RULE)
    completion = client.chat.completions.create(
        model=model,
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": order},
        ],
    )
    # The model saw only tokens; the proxy restored real values on the user_output channel before
    # this response reached the SDK.
    print("\nRestored draft returned by the proxy:\n")
    print(completion.choices[0].message.content)
    return 0

Dependencies (examples/openai-proxy-python/requirements.txt), only needed for --live:

openai>=1.0.0

Expected output (explain path). Token ids and fingerprints vary per run; the shape is what matters:

[1] What leaves your network (sent to OpenAI):
    ... my name is {{person:p_001:...}} (customer number {{customer_id:cid_001:...}}) ...
    ... order {{order:o_001:...}} ... email at {{email:e_001:...}} ... phone at {{phone:ph_001:...}} ...
    ... IBAN {{iban:ib_001:...}}.
[1b] Leak check on the outbound text:
    PASS, none of the 8 raw values are present. detector: builtin_and_ner

[2] Safe metadata the model receives with the tokens:
    type          gender    formality   role        lang
    email         -         -           customer    en
    person        female    formal      customer    en
    ...

[4] What your OpenAI client receives back (user_output restore):
    Hello Ms. Erika Mustermann, ...  (IBAN stays masked)

[5] Restore-mode ladder on the salutation (same session, same draft):
    full       -> Hello Erika Mustermann,
    formatted  -> Hello Ms. Erika Mustermann,
    partial    -> Hello E. Mustermann,
    masked     -> Hello E**** *********n,
    none       -> Hello [REDACTED],

[6] Role-gated tightening (caller_role=support_agent, internal_summary channel):
    agent console sees -> Hello E. Mustermann,

[7] Fail-closed guards:
    tampered session -> rejected with GUARDAI_INVALID_INPUT (HTTP 400); no raw value returned
    forged token     -> resolved=0 unresolved=['{{iban:ib_999:deadbeef0000}}'] (left verbatim, never guessed)

OpenAI Node, sales copilot

A sales copilot for Musterfirma (a fictional B2B SaaS vendor) drafts a follow-up to a prospect's inbound email, while the prospect's name, company, email, phone, the CRM opportunity and account ids, and a lead score never leave the network in the clear. Directory: examples/openai-proxy-node/. Policy: crm-sales. The explain path uses the global fetch built into modern Node.js.

Prerequisite (needs-NER): a running OGuardAI server with the NER detector active. crm-sales lists person and company under detection.required_for, so a runtime without NER refuses the transform (fail closed) rather than let a prospect or account name reach the model. Run the bundled Docker stack (both mode) so the full walkthrough executes.

Start the runtime, then run the offline explain path (no OpenAI key, no proxy, no npm install of openai):

# Start the OGuardAI runtime (the product under test). NER is needed for person/company/address. The
# bundled stack requires these four secrets, and its client key ships least-privilege, so widen it to
# the crm-sales policy and the channels this walkthrough restores on (user_output, crm_sync,
# sales_export, log_safe).
export GUARDAI_SESSION_SECRET=$(openssl rand -base64 32)
export GUARDAI_API_KEY_1=$(openssl rand -hex 32)
export GUARDAI_DETECTOR_API_KEY=$(openssl rand -hex 32)
export GUARDAI_REDIS_PASSWORD=$(openssl rand -hex 32)
export GUARDAI_API_KEY_1_ALLOWED_POLICIES=default,default-ner-required,german-support,crm-sales
export GUARDAI_API_KEY_1_ALLOWED_OUTPUT_CHANNELS=user_output,crm_sync,sales_export,log_safe
docker compose -f deploy/docker/docker-compose.yml up -d

# Run the walkthrough. No OpenAI key, no proxy, no npm install of openai required.
export GUARDAI_URL=http://localhost:3000
export GUARDAI_API_KEY="$GUARDAI_API_KEY_1"   # the bundled stack runs api_key auth
npx tsx index.ts

The prospect email the example reads (examples/openai-proxy-node/prospect_inbound.txt):

From: erika.mustermann@example.com
To: max.mustermann@example.com
Subject: Re: Musterfirma proposal, opportunity OPP-2026-4471

Hi Max,

thanks for the walkthrough yesterday. This is Erika Mustermann, VP Operations at
Beispielfirma and the economic buyer on our side.

We are moving forward internally on opportunity OPP-2026-4471 against account
ACCT-88213. Our lead score in your system is lead score: 82, which sounds about
right given our timeline.

Could you send a follow-up summarizing the enterprise tier and next steps? You can
reach me at erika.mustermann@example.com or on +49 89 4455123.

Best,
Erika

The fixture that proves the hard financial floor. crm-sales uses on_redact: reject, so a pasted card blocks the whole request and the model is never called (examples/openai-proxy-node/card_paste.txt):

Quick internal note before the call: the prospect accidentally pasted a corporate
card, 4111 1111 1111 1111, into the email thread for opportunity OPP-2026-4471.
Please scrub it from the record.

Scenario constants, the system prompt, and the tracked raw values (examples/openai-proxy-node/index.ts):

const POLICY = "crm-sales";
const LANGUAGE = "en";
const HERE = dirname(fileURLToPath(import.meta.url));

// System instruction the rep gives the model. It names no prospect data: the model works entirely
// from tokens plus the metadata OGuardAI attaches to each one.
const SYSTEM_PROMPT = [
  "You are a B2B sales copilot for Musterfirma. Draft a concise, friendly follow-up email to a",
  "prospect that summarizes the enterprise tier and proposes next steps. The message contains",
  "{{type:id:cap}} placeholders that stand for real data you must not see. Copy every placeholder",
  "verbatim; never decode, translate, reformat, or invent one, and never write a real name, company,",
  "email, or deal id yourself. Do not write a salutation word before a person placeholder: the",
  "runtime inserts the correct honorific. Open with 'Hi {{person:...}},'.",
].join(" ");

// Structured identifiers the runtime detects deterministically: builtin regex for the emails and
// phone, and the crm-sales custom_patterns for the opportunity and account ids. They must never
// reach the model on any deployment, so their presence in the outbound text is a hard failure.
const STRUCTURED_PII = [
  "erika.mustermann@example.com",
  "max.mustermann@example.com",
  "+49 89 4455123",
  "OPP-2026-4471",
  "ACCT-88213",
];

// Prospect and account names. These are NER-only: the regex floor detects no names, so a runtime
// without NER cannot tokenize them. crm-sales lists person and company under detection.required_for,
// so such a runtime refuses the request instead of leaking them.
const NER_PII = ["Erika Mustermann", "Beispielfirma"];

The explain path drives the runtime through a thin client that posts to /v1/transform and /v1/rehydrate (examples/openai-proxy-node/og_runtime.ts):

export function transform(
  input: string,
  opts: { policy: string; language?: string; context?: TransformContext },
): Promise<TransformResult> {
  return post<TransformResult>("/v1/transform", {
    input,
    input_type: "text",
    policy: opts.policy,
    language: opts.language,
    context: opts.context,
  });
}

export function rehydrate(opts: {
  output: string;
  sessionState: string;
  channel?: string;
  restoreMode?: string;
}): Promise<RehydrateResult> {
  return post<RehydrateResult>("/v1/rehydrate", {
    output: opts.output,
    session_state: opts.sessionState,
    output_channel: opts.channel,
    restore_mode: opts.restoreMode,
  });
}

Run it live against the real proxy and OpenAI. Start the NER sidecar and the proxy as in the OpenAI Python example above (target https://api.openai.com, port 8081, advanced mode), but pass --policy crm-sales, the policy the explain path uses. crm-sales requires NER for the prospect and account names, and it declares its deal, account, and lead-score patterns in the policy file, so no GUARDAI_DETECTOR_CUSTOM is needed. Then run the Node client:

npm install   # installs the openai SDK (declared in package.json), only needed for --live

# In another shell, point the SDK at the proxy and run for real.
export OPENAI_API_KEY="sk-..."             # your real OpenAI key, read from the environment
export GUARDAI_PROXY_API_KEY="<same-as-above>"
export OPENAI_MODEL="gpt-4o-mini"           # optional
npx tsx index.ts --live

The only change from a normal OpenAI integration (examples/openai-proxy-node/index.ts, runLive):

async function runLive(): Promise<number> {
  const proxyUrl = `${process.env.GUARDAI_PROXY_URL ?? "http://localhost:8081"}/v1`;
  const proxyKey = process.env.GUARDAI_PROXY_API_KEY ?? "";
  if (!process.env.OPENAI_API_KEY) {
    console.error("Live mode needs OPENAI_API_KEY (forwarded by the proxy to OpenAI).");
    return 2;
  }

  // Imported lazily so the explain path needs no third-party packages.
  const { default: OpenAI } = await import("openai");

  // The only change from a normal OpenAI integration: baseURL points at the proxy, and the proxy's
  // own auth key rides in X-GuardAI-API-Key. The SDK still sends OPENAI_API_KEY as its bearer token,
  // which the proxy forwards upstream. Everything else is stock OpenAI.
  const client = new OpenAI({
    baseURL: proxyUrl,
    defaultHeaders: { "X-GuardAI-API-Key": proxyKey },
  });
  const model = process.env.OPENAI_MODEL ?? "gpt-4o-mini";

  console.log(RULE);
  console.log(`LIVE MODE: OpenAI SDK -> OGuardAI proxy (${proxyUrl}) -> OpenAI (${model})`);
  console.log(RULE);
  const completion = await client.chat.completions.create({
    model,
    messages: [
      { role: "system", content: SYSTEM_PROMPT },
      { role: "user", content: fixture("prospect_inbound.txt") },
    ],
  });
  // The model saw only tokens; the proxy restored real values on the user_output channel before
  // this response reached the SDK.
  console.log("\nRestored draft returned by the proxy:\n");
  console.log(completion.choices[0]?.message.content ?? "");
  return 0;
}

Dependencies and scripts (examples/openai-proxy-node/package.json):

{
  "name": "guardai-openai-proxy-node",
  "version": "1.0.0",
  "description": "OpenAI Node SDK with OGuardAI transparent PII proxy",
  "type": "module",
  "scripts": {
    "typecheck": "tsc --noEmit",
    "test": "npx tsx --test og_runtime.test.ts",
    "start": "npx tsx index.ts",
    "explain": "npx tsx index.ts --explain",
    "live": "npx tsx index.ts --live"
  },
  "dependencies": {
    "openai": "^4.0.0"
  },
  "devDependencies": {
    "@types/node": "^20",
    "tsx": "^4.0",
    "typescript": "^5.5"
  }
}

Expected output (explain path). Token ids and fingerprints vary per run; the shape is what matters:

[1] What leaves your network (sent to OpenAI):
    ... This is {{person:p_001:...}}, VP Operations at {{company:c_001:...}} ...
    ... opportunity {{custom:deal_id:x_001:...}} against account {{custom:account_id:x_002:...}} ...
    ... lead score: {{custom:lead_score:x_003:...}} ... reach me at {{email:e_001:...}} or on {{phone:ph_001:...}}.
[1b] Leak check on the outbound text:
    PASS, none of the 7 raw values are present. detector: builtin_and_ner

[4] One session, four readers (same draft, different restore):
    --- channel=user_output  (...) ---     opportunity OPP-2026-4471 for Beispielfirma, real email
    --- channel=crm_sync     (...) ---     ids and contacts masked
    --- channel=sales_export (...) ---     names abstracted, contacts dropped
    --- channel=log_safe     (...) ---     everything redacted

[5] Role-gated deal id (user_output channel, same inbound):
    caller_role=sales_rep -> ... opportunity OPP-2026-4471 ...
    no role               -> ... opportunity ******** ...

[6] Fail-closed guards:
    a) tampered session -> rejected with GUARDAI_INVALID_INPUT (HTTP 400); no raw value returned
    b) forged token     -> resolved=0 unresolved=["{{custom:deal_id:x_999:deadbeef0000}}"] (left verbatim, never guessed)
    c) pasted card      -> whole request blocked: GUARDAI_POLICY_DENIED (HTTP 403); the model was never called

Anthropic Python, insurance claims desk

An AI claims desk for Musterversicherung AG (a fictional German motor insurer) asks Claude to draft the formal German acknowledgement of a claim email, while the policyholder name, two emails, phone, postal address, German tax ID, bank IBAN, policy and claim numbers, vehicle VIN and licence plate, and a reserve amount never leave the network in the clear. The same proxy binary that fronts OpenAI also fronts Anthropic on /v1/messages. Directory: examples/anthropic-proxy-python/. Policy: insurance-de. All data in the fixture is fictional.

Prerequisite (needs-NER): a running OGuardAI server with the NER detector active (person and address detection need it). The insurance-de policy requires NER for the policyholder name, so a runtime without NER fails closed rather than let an un-tokenized name cross. The live path runs the proxy in advanced mode (NER required, fail closed on a sidecar outage).

Start the runtime, then run the offline explain path (Python standard library only, nothing to install):

# The bundled stack requires these four secrets, and its client key ships least-privilege, so widen it
# to the insurance-de policy and the channels this walkthrough restores on (user_output,
# customer_email, reinsurer_export, log_safe).
export GUARDAI_SESSION_SECRET=$(openssl rand -base64 32)
export GUARDAI_API_KEY_1=$(openssl rand -hex 32)
export GUARDAI_DETECTOR_API_KEY=$(openssl rand -hex 32)
export GUARDAI_REDIS_PASSWORD=$(openssl rand -hex 32)
export GUARDAI_API_KEY_1_ALLOWED_POLICIES=default,default-ner-required,german-support,insurance-de
export GUARDAI_API_KEY_1_ALLOWED_OUTPUT_CHANNELS=user_output,customer_email,reinsurer_export,log_safe
docker compose -f deploy/docker/docker-compose.yml up -d

export GUARDAI_BASE_URL=http://localhost:3000
export GUARDAI_API_KEY="$GUARDAI_API_KEY_1"   # the bundled stack runs api_key auth
python main.py

The claim email the example reads (examples/anthropic-proxy-python/claim_email.txt):

Von: erika.mustermann@example-mail.de
An: schaden@example.de
Betreff: Schadenmeldung Kaskoversicherung, Auffahrunfall vom 04.07.2026

Sehr geehrte Damen und Herren,

ich, Frau Dr. Erika Mustermann, Versicherungsnehmerin, moechte einen Schaden
melden. Am 04.07.2026 ist mir auf der A9 ein Fahrzeug aufgefahren.

Meine Kontaktdaten fuer Rueckfragen: E-Mail erika.mustermann@example-mail.de,
Telefon +49 89 1122334. Anschrift: Musterstrasse 47, 80331 Muenchen.

Zum Vertrag: Versicherungsschein-Nr. VS-4820091, Schadennummer SCH-2026-55831.
Meine Steuer-Identifikationsnummer lautet 86 095 742 719.
Betroffen ist mein Fahrzeug, Fahrgestellnummer WVWZZZ1KZAW008421,
amtliches Kennzeichen M-KV 2847.

Bitte ueberweisen Sie eine etwaige Erstattung auf meine IBAN
DE97 9900 0000 0123 4567 89. Nach Einschaetzung meiner Werkstatt liegt der
Schaden bei rund 18.500 EUR.

Bitte bestaetigen Sie mir kurz den Eingang und die weiteren Schritte.

Mit freundlichen Gruessen
Dr. Erika Mustermann

Scenario constants, the German system prompt, and the tracked raw values (examples/anthropic-proxy-python/main.py):

POLICY = "insurance-de"
LANGUAGE = "de"
HERE = Path(__file__).resolve().parent
CLAIM_FILE = HERE / "claim_email.txt"

# System instruction the agent gives Claude. It intentionally names no customer data: the model
# works entirely from tokens plus the metadata OGuardAI attaches to each one.
SYSTEM_PROMPT = (
    "Du bist die Schadenassistenz der Musterversicherung AG. Formuliere eine kurze, formelle "
    "Eingangsbestaetigung auf Deutsch (Sie-Form). Sprich die Person mit korrekter Anrede an, "
    "nenne Versicherungsschein- und Schadennummer, bestaetige die hinterlegte Bankverbindung fuer "
    "eine etwaige Erstattung und kuendige die Pruefung an. Behandle jeden {{typ:id:pruefsumme}}-Platzhalter als "
    "unveraenderlichen Baustein: niemals dekodieren, umschreiben oder erfinden."
)

STRUCTURED_PII = [
    "erika.mustermann@example-mail.de",
    "schaden@example.de",
    "+49 89 1122334",
    "86 095 742 719",
    "DE97 9900 0000 0123 4567 89",
    "18.500",
    "VS-4820091",
    "SCH-2026-55831",
    "WVWZZZ1KZAW008421",
    "M-KV 2847",
]
NER_PII = [
    "Erika Mustermann",
]

The explain path drives the runtime through a thin client that posts to /v1/transform and /v1/rehydrate (examples/anthropic-proxy-python/og_runtime.py):

def transform(text: str, policy: str, language: str | None = None) -> dict[str, Any]:
    """Detect + tokenize `text` under `policy`. Returns safe_text, session_state, entity_context."""
    payload: dict[str, Any] = {"input": text, "policy": policy}
    if language:
        payload["language"] = language
    return _post("/v1/transform", payload)


def rehydrate(
    output: str,
    session_state: str,
    *,
    output_channel: str | None = None,
    restore_mode: str | None = None,
    restore_overrides: dict[str, str] | None = None,
) -> str:
    """Restore tokens in `output` under the session, governed by channel / restore mode."""
    payload: dict[str, Any] = {"output": output, "session_state": session_state}
    if output_channel:
        payload["output_channel"] = output_channel
    if restore_mode:
        payload["restore_mode"] = restore_mode
    if restore_overrides:
        payload["restore_overrides"] = restore_overrides
    return _post("/v1/rehydrate", payload)["restored_text"]

Run it live against the real proxy and Claude. Start the proxy pointed at Anthropic, with the insurance-de policy and the NER detector in advanced mode:

export GUARDAI_SESSION_SECRET=$(openssl rand -base64 32)
export GUARDAI_PROXY_API_KEY=$(openssl rand -hex 32)
export GUARDAI_DETECTOR_MODE=advanced
export GUARDAI_DETECTOR_ADVANCED_URL=http://localhost:9090

# NER detector in the host network namespace, from the published image. `--host 127.0.0.1` keeps the
# unauthenticated detector bound to loopback, reachable by the proxy on the same host (localhost:9090)
# but not exposed on any other interface.
docker run -d --rm --network host -e GUARDAI_DETECTOR_ALLOW_INSECURE=true \
  --name oguardai-detector ghcr.io/oronts/oguardai/detector-py:0.1.0 \
  --host 127.0.0.1 --port 9090

# The proxy runs from the published image (the release archive ships the CLI and server only, not a
# proxy binary). --network host lets it reach the detector on localhost:9090 and serve on
# localhost:8081.
docker run --rm --network host \
  -e GUARDAI_SESSION_SECRET -e GUARDAI_PROXY_API_KEY \
  -e GUARDAI_DETECTOR_MODE -e GUARDAI_DETECTOR_ADVANCED_URL \
  ghcr.io/oronts/oguardai/proxy:0.1.0 \
  --provider anthropic --target https://api.anthropic.com \
  --port 8081 --policy insurance-de --single-tenant --rate-limit-enabled

Then run the client. The Anthropic SDK reads ANTHROPIC_API_KEY from the environment and the proxy forwards it upstream. No key is hardcoded:

pip install -r requirements.txt
export ANTHROPIC_API_KEY=sk-ant-...
export GUARDAI_PROXY_API_KEY=<same value the proxy was started with>
export GUARDAI_PROXY_URL=http://localhost:8081
python main.py --live

The only change from a normal Anthropic integration (examples/anthropic-proxy-python/main.py, run_live):

def run_live(claim: str) -> int:
    # Imported lazily so the explain path needs no third-party packages.
    try:
        import anthropic
    except ImportError:
        print("Live mode needs the Anthropic SDK: pip install anthropic", file=sys.stderr)
        return 2

    proxy_url = os.environ.get("GUARDAI_PROXY_URL", "http://localhost:8081")
    proxy_key = os.environ.get("GUARDAI_PROXY_API_KEY", "")
    if not os.environ.get("ANTHROPIC_API_KEY"):
        print("Live mode needs ANTHROPIC_API_KEY (forwarded by the proxy to Claude).", file=sys.stderr)
        return 2

    # The only change from a normal Anthropic integration: base_url points at the proxy, and the
    # proxy's own auth key rides in X-GuardAI-API-Key. The SDK still sends ANTHROPIC_API_KEY, which
    # the proxy forwards upstream. Everything else is stock Anthropic.
    client = anthropic.Anthropic(
        base_url=proxy_url,
        default_headers={"X-GuardAI-API-Key": proxy_key},
    )
    model = os.environ.get("ANTHROPIC_MODEL", "claude-sonnet-4-5")

    print(RULE)
    print(f"LIVE MODE: Anthropic SDK -> OGuardAI proxy ({proxy_url}) -> Claude ({model})")
    print(RULE)
    message = client.messages.create(
        model=model,
        max_tokens=512,
        system=SYSTEM_PROMPT,
        messages=[{"role": "user", "content": claim}],
    )
    text = "".join(block.text for block in message.content if block.type == "text")
    # The model saw only tokens; the proxy restored real values on the user_output channel before
    # this response reached the SDK.
    print("\nRestored draft returned by the proxy:\n")
    print(text)
    return 0

Dependencies (examples/anthropic-proxy-python/requirements.txt), only needed for --live:

anthropic>=0.40

How the policy governs restoration. The insurance-de policy (policies/insurance-de/policy.yaml) decides each value's base restore mode, then each channel may only tighten it, which is why the same draft renders differently for each reader:

Valuecustomer_emailreinsurer_exportlog_safe
policyholder namegender aware anredeabstractnone
policy / claim no.fullmaskedmasked
IBANmaskednonenone
VIN / licence platemaskednonenone
reserve amountband (partial)band (partial)none

Expected output (explain mode):

[1] What leaves your network (sent to Claude):
... ich, Frau Dr. {{person:p_001:...}}, Versicherungsnehmerin, ...
Versicherungsschein-Nr. {{custom:policy_number:x_001:...}}, Schadennummer {{custom:claim_id:x_002:...}}.
Meine Steuer-Identifikationsnummer lautet {{custom:de_tax_id:x_003:...}}.

[1b] Leak check on the outbound text:
    PASS, none of the 11 raw values are present. detector: builtin_and_ner

[2] Safe metadata Claude receives with the tokens:
    type                  gender    formality   role          lang
    person                female    formal      policyholder  de
    ...

[4] Restoration is governed per destination (same session, different reader):
    --- channel=customer_email ---
    ... Versicherungsschein-Nr. VS-4820091 ... Bankverbindung D*** **** **** **** **** *0 ...
        Rueckstellung liegt bei moderate (5k-50k).
    --- channel=reinsurer_export ---
    ... Versicherungsschein-Nr. V********1 ... Bankverbindung [REDACTED] ...
    --- channel=log_safe ---
    Sehr geehrte [REDACTED], ... Versicherungsschein-Nr. [REDACTED] ...

[5] Fail-closed: a tampered session must not restore anything:
    PASS, rejected with GUARDAI_INVALID_INPUT (HTTP 400); no raw value returned