OpenAI Drop-In Proxy
Point your provider SDK at the proxy, set the base URL and the proxy auth header, and get PII protection without changing your prompts or application logic
The Killer Use Case
Point your provider SDK at the proxy: set the base URL and add the proxy auth header. Your prompts, models, and application logic stay the same.
Before (unprotected)
client = OpenAI(api_key="sk-...")
response = client.chat.completions.create(
model="gpt-4",
messages=[{"role": "user", "content": "Customer Erika Mustermann (erika.mustermann@example.com) needs help"}]
)
# Erika's name and email sent directly to OpenAIAfter (protected by OGuardAI proxy)
The proxy runs on port 8081 by default (separate from the OGuardAI server on port 3000 or 8080). Point your SDK's base_url at the proxy:
client = OpenAI(
api_key="sk-...",
base_url="http://localhost:8081/v1", # Point to OGuardAI proxy (port 8081)
default_headers={"X-GuardAI-API-Key": os.environ["GUARDAI_PROXY_API_KEY"]},
)
response = client.chat.completions.create(
model="gpt-4",
messages=[{"role": "user", "content": "Contact erika.mustermann@example.com or call +49 30 12345678"}]
)
# OpenAI sees: "Contact {{email:e_001:1bcaef1a4aff}} or call {{phone:ph_001:71453119be53}}"
# Response automatically restored with original valuesHow It Works
- Your app sends request to OGuardAI proxy (port 8081)
- Proxy masks PII in all user/assistant messages
- Proxy forwards to real provider API
- Provider responds with tokens (e.g., "
{{email:e_001:1bcaef1a4aff}}") - Proxy restores tokens to real values
- Your app receives the response with original values
Detection scope: By default (
GUARDAI_DETECTOR_MODEunset, orbuiltin) the proxy uses builtin regex detectors and covers emails, phones, IPs, IBANs, SSNs, VAT IDs, URLs, credit cards, and other structured patterns, but NOT person, company, or location names. To also mask names, run the Python NER sidecar and start the proxy withGUARDAI_DETECTOR_MODE=both(builtin plus NER, regex fallback if the sidecar is down) orGUARDAI_DETECTOR_MODE=advanced(NER required, fail closed on a sidecar outage) andGUARDAI_DETECTOR_ADVANCED_URL=http://localhost:9090. In the default builtin mode a raw name can reach the model, so enable NER for name-bearing traffic.
Inline images: The proxy OCR-scans inline image bytes (OpenAI
image_urldata:URIs and Anthropic base64 image blocks). If the image renders detectable PII or cannot be scanned, the request is rejected, so PII inside an image never reaches the LLM. Clean images are forwarded unchanged. OCR requires Tesseract on the proxy host (apt install tesseract-ocr); without it, inline images are rejected fail-closed rather than forwarded unscanned. The maximum decoded image size is 5 MiB, whichGUARDAI_PROXY_INLINE_IMAGE_MAX_BYTEScan only lower. Remote image URLs are not fetched: the proxy cannot OCR-scan the pixels behind a URL, so by default a remote image URL is rejected fail-closed. An operator can opt in withGUARDAI_PROXY_ALLOW_REMOTE_IMAGE_URLSto forward remote URLs after a URL-string PII check only (the image pixels are still not scanned).
Generated media (model output): The proxy protects text, so it fails closed on any model output it cannot scan. If the model returns audio, images, documents, or an inline
data:base64 payload that sits outside a text field, the response is refused with422 GUARDAI_OUTPUT_MEDIA_REJECTED; an output shape the proxy has no scanner for, such as a structured content array or an unrecognized block, is refused with422 GUARDAI_OUTPUT_UNSUPPORTED. The proxy cannot verify the PII inside generated pixels or audio, so it does not forward them unscanned. A request that asks the upstream for a non-text output modality (modalities,audio) is refused before the upstream is called. Text output is unaffected and byte-identical.
Other content blocks: Any non-text content block the proxy cannot scan (an Anthropic
documentblock, an OpenAIfileorinput_audioblock, or any unrecognized block type) is rejected fail-closed instead of being forwarded to the LLM unscanned. To send a document through AI safely, extract its text first and route it through the server/v1/transformAPI, which has a sandboxed document and PDF ingestion pipeline.
Provider credentials: The proxy forwards the upstream provider auth and routing headers (
Authorization,x-api-key,OpenAI-Organization,OpenAI-Project,anthropic-version,anthropic-beta) to the provider unchanged, because they are required to authenticate and route the call. These are credential and routing values, not request data, so they are intentionally never scanned and never logged. Your OGuardAI proxy key (X-GuardAI-API-Key) authenticates you to the proxy and is never forwarded upstream.
Setup
Availability: The proxy image used in the
docker runbelow is private and provided under a written agreement with Oronts. It is not a public pull; request access before setup at /request.
# Generate one proxy key; the proxy and every client must present the same value.
export GUARDAI_PROXY_API_KEY=$(openssl rand -hex 32)
# Start the OGuardAI proxy (with auth) from the published image, inheriting the exported key with a
# valueless -e. --network host serves it on the host's localhost:8081.
docker run --rm --network host \
-e GUARDAI_SESSION_SECRET=$(openssl rand -base64 32) \
-e GUARDAI_PROXY_API_KEY \
ghcr.io/oronts/oguardai/proxy:0.1.0 \
--target https://api.openai.com --policy default --port 8081 \
--single-tenant --rate-limit-enabledClients must include the
X-GuardAI-API-Key: <your-proxy-key>header. The upstream provider key (sk-...) is passed via the standardAuthorizationheader by the OpenAI SDK.
Client Setup
Run the client in the same shell where you exported GUARDAI_PROXY_API_KEY above, so the proxy and the client present the same key:
from openai import OpenAI
import os
client = OpenAI(
base_url="http://localhost:8081/v1",
default_headers={"X-GuardAI-API-Key": os.environ["GUARDAI_PROXY_API_KEY"]},
)Streaming Support
Streaming works transparently:
stream = client.chat.completions.create(
model="gpt-4",
messages=[...],
stream=True # Streaming works through proxy
)
for chunk in stream:
print(chunk.choices[0].delta.content, end="")Streaming budgets
On top of the per-chunk idle timeout, the proxy enforces three per-stream lifetime budgets on both the OpenAI-compatible and the Anthropic streaming paths. They bound a stream that keeps sending timely chunks the idle timeout never catches, or that never ends.
| Env var | Flag | Default | Purpose |
|---|---|---|---|
GUARDAI_PROXY_STREAM_MAX_DURATION_SECS | --stream-max-duration | 3600 | Total wall-clock deadline, in seconds, for one streaming response |
GUARDAI_PROXY_STREAM_MAX_TOTAL_BYTES | --stream-max-total-bytes | 67108864 (64 MiB) | Total received-byte cap for one streaming response |
GUARDAI_PROXY_STREAM_MAX_CONCURRENT | --stream-max-concurrent | 100 | Maximum concurrent in-flight streams |
Set an axis to 0 to disable it (unlimited). When a stream breaches the duration or byte cap
mid-flight, the proxy flushes any already-scanned held-back content, then ends the stream with a
terminal SSE error event of type GUARDAI_STREAM_BUDGET_EXCEEDED. No partial or raw content is
forwarded past the cap. When the concurrency cap is saturated, a new streaming request is rejected
after auth with 503 and a retry-after: 1 header. A non-dev proxy that disables all three axes
logs a startup warning.
A stream that starts emitting generated media the proxy cannot scan (an audio or image chunk, or an
unscannable content block) is ended the same way: the proxy flushes any already-scanned held-back
content, then emits one terminal SSE error event of type GUARDAI_OUTPUT_MEDIA_REJECTED or
GUARDAI_OUTPUT_UNSUPPORTED with no data: [DONE], so no media bytes reach the client.
Streaming terminal events
A streamed response is a 200 the moment its first byte leaves, so the proxy cannot withhold it the
way the buffered path does. Its only truthful channel after the status line is the terminal event.
When an upstream event cannot be protected, the proxy flushes any already-scanned held-back content
and then ends the stream with a terminal SSE error event instead of a clean data: [DONE] (OpenAI)
or message_stop (Anthropic). No unscanned or partial content is forwarded.
| Terminal event type | Reason label | When |
|---|---|---|
GUARDAI_STREAM_UNPROCESSABLE | malformed_event | A data line, terminal, or passthrough payload was not valid JSON |
GUARDAI_STREAM_UNPROCESSABLE | ambiguous_event | One event carried multiple content choices, content beside a refusal, content beside a tool call, or multiple tool-argument sources |
GUARDAI_STREAM_UNPROCESSABLE | unsupported_delta | An Anthropic content-block delta of a type the pipeline cannot protect |
GUARDAI_STREAM_UNPROCESSABLE | unknown_event | An Anthropic event name outside the documented allowlist |
GUARDAI_STREAM_UNPROCESSABLE | incomplete_tool_block | A tool-argument block was still open when the terminal arrived |
GUARDAI_STREAM_UNPROCESSABLE | buffer_limit | A tool-argument accumulation or SSE line buffer grew past its configured bound |
GUARDAI_STREAM_TRUNCATED | (none) | The upstream ended before its completion terminal (disconnect or transport error) |
GUARDAI_OUTPUT_BLOCKED | (none) | The output guard judged streamed model output and blocked it: newly generated PII in tool arguments, or a policy denial |
GUARDAI_OUTPUT_GUARD_UNAVAILABLE | (none) | The output guard could not complete a scan or restore of streamed model output, so the content was never judged. Covers the text channel, an extended-thinking block, tool arguments, and passthrough events (a detector or NER outage, a sibling scan that failed closed, or a token the session could not rehydrate) |
The OpenAI-shaped payload is {"error":{"message":"<reason>","type":"GUARDAI_STREAM_UNPROCESSABLE","code":"stream_unprocessable"}}; the Anthropic-shaped payload is {"type":"error","error":{"type":"GUARDAI_STREAM_UNPROCESSABLE","message":"<reason>"}}. The two output guard terminals reuse the buffered path's codes and the same shape: OpenAI carries "code":"output_blocked" or "code":"output_guard_unavailable". A guard block and a guard outage are distinct terminals: a block means the model output was judged and refused, an outage means the scan could not run and the same request may succeed once the deployment is fixed. The OpenAI and Anthropic SDKs raise on a terminal error event, so an SDK caller sees an exception rather than a truncated-but-clean response.
Scope of v1 streaming. The proxy protects a stream shaped like the first-party OpenAI and
Anthropic dialects: n=1 (streaming refuses any other n before the upstream call), one content
choice per event, and one tool call per event. A third-party OpenAI-compatible backend that batches
multiple choices or several tool calls into a single event, or an event that mixes content and a tool
call, cannot be guarded per item by the single per-stream pipeline, so it ends the stream with an
ambiguous_event error rather than forwarding a lossy result. Route those workloads through the
buffered (non-streaming) path. Because an undocumented Anthropic event name faults, a genuinely new
provider event type requires an allowlist update in the proxy.
Raw-SSE consumers. A client that reads the raw SSE stream itself, rather than through an SDK,
must treat event: error as terminal in addition to data: [DONE] / message_stop. A stream that
ends on event: error was not completed; the fault is recorded on the audit trail either way, and the
specific reason is carried in the error event, not the audit record.
What Gets Protected
| Message Role | Scanned? |
|---|---|
| system | Yes |
| user | Yes |
| assistant | Yes |
| tool calls | Yes (function arguments) |
| tool results | Yes |
Readiness and coverage
The proxy answers two operational endpoints locally instead of forwarding them upstream:
GET /readyz(unauthenticated) reports dependency health for a load balancer. It returns503when the proxy has no policy loaded, when a store it depends on is unhealthy, or, inboth/advanceddetector mode, when the NER sidecar is unreachable, so a deployment that would silently fall back to regex-only detection stops receiving traffic. In builtin mode it never probes a sidecar.GET /v1/capabilities(authenticated) reports the detector's EFFECTIVE coverage:{covered_entity_types, ner_active, detector_mode}, plusprotection_overrides(a list of the enabled override ids, omitted when none is enabled). Built-in format types (email, phone, IBAN, and so on) are always covered;person,company, andlocationappear only when NER is active, so a consumer can tell whether names are masked before sending traffic. The NER triple assumes a standard backend (the shipped GLiNER and spaCy backends both emit it); do not pair a subset backend (for example thenonebackend, which emits nothing) withboth/advancedmode, since capabilities would then advertise names it does not detect.
Anthropic Support
Same pattern for Anthropic:
import os
client = Anthropic(
api_key="sk-ant-...",
base_url="http://localhost:8081",
default_headers={"X-GuardAI-API-Key": os.environ["GUARDAI_PROXY_API_KEY"]},
)OpenAI-compatible providers (Mistral and others)
Any provider that speaks the OpenAI /v1/chat/completions wire format is protected by the same
handler. Point the proxy target at the provider and use it as usual. For Mistral:
docker run --rm --network host \
-e GUARDAI_SESSION_SECRET=$(openssl rand -base64 32) -e GUARDAI_PROXY_API_KEY \
ghcr.io/oronts/oguardai/proxy:0.1.0 \
--target https://api.mistral.ai --policy default --port 8081 --single-tenant --rate-limit-enabled
# or pass the target as an env var: -e GUARDAI_PROXY_TARGET=https://api.mistral.ai
# GUARDAI_PROXY_API_KEY is inherited from the value exported in Setup above.For a gateway or aggregator, point the target at the origin plus any prefix that sits before the
route the proxy appends. The proxy forwards the client's full path, which begins with /v1, so the
target is https://openrouter.ai/api and https://api.groq.com/openai, not a URL that already ends
in /v1. A target that ends in the mounted route's first segment, or that carries a percent sign, a
query string, or a fragment, is refused at startup with the reason to fix.
The client sends its normal Mistral request to http://localhost:8081/v1/chat/completions with its
Mistral API key in the Authorization: Bearer header (forwarded unchanged to the upstream). Message
content, tool-call arguments, tool/function definitions, top-level user, and metadata are
tokenized before the request leaves the runtime; buffered and streaming responses are rehydrated.
Restored tool-call arguments are redacted by default. The proxy cannot verify that the client's tool
runs inside the trust boundary, so it does not return raw values into tool_calls arguments. A
deployment whose tools are trusted and in-boundary opts into raw restoration with
GUARDAI_PROXY_TOOL_PAYLOAD_RAW=true (or --tool-payload-raw); this flag is the only lever, a policy
rule cannot widen it back to raw. It stays a production option (warned at startup, never gated), and
when enabled it is surfaced on the /v1/capabilities protection_overrides list and the startup log.
Mistral-specific request fields are handled correctly: prefix completion uses a boolean prefix flag
on the last assistant message and the prefix text lives in content, which is scanned like any other
message content; safe_prompt (boolean) and random_seed (number) carry no PII and pass through
unchanged.
Scope: only /v1/chat/completions (and Anthropic /v1/messages) are intercepted and protected.
/v1/embeddings is protected through the embedding path. Other Mistral endpoints such as
/v1/fim/completions are not supported and are rejected at the proxy boundary (fail-closed, not
forwarded), so raw PII never reaches an unprotected endpoint. Use the server /v1/transform API for
those payloads.
Azure OpenAI
Azure uses a different URL scheme (/openai/deployments/{deployment}/chat/completions?api-version=...)
and an api-key header instead of a Bearer token, so it needs an explicit provider mode. Set
--provider azure (or GUARDAI_PROXY_PROVIDER=azure) and point the target at your Azure resource:
docker run --rm --network host \
-e GUARDAI_SESSION_SECRET=$(openssl rand -base64 32) -e GUARDAI_PROXY_API_KEY \
ghcr.io/oronts/oguardai/proxy:0.1.0 \
--provider azure \
--target https://my-resource.openai.azure.com \
--policy default --port 8081 --single-tenant --rate-limit-enabledClients send their normal Azure request to
http://localhost:8081/openai/deployments/{deployment}/chat/completions?api-version=YYYY-MM-DD with
their api-key header. The proxy tokenizes the request through the same OpenAI-compatible pipeline,
preserves the deployment path and api-version query when forwarding, and forwards the api-key (and
Authorization, for Entra ID) header unchanged. Azure embeddings
(/openai/deployments/{deployment}/embeddings) are protected through the embedding path. The provider
mode is never auto-detected from the target URL: you must set it explicitly.
Vertex AI (OpenAI-compatible)
Vertex AI exposes an OpenAI-compatible chat endpoint at
/v1beta1/projects/{project}/locations/{location}/endpoints/{endpoint}/chat/completions (the endpoint
segment is openapi for managed Gemini, or your endpoint id for a self-deployed model) with an OAuth2
Bearer token. Set --provider vertex-openai and target your regional aiplatform host:
docker run --rm --network host \
-e GUARDAI_SESSION_SECRET=$(openssl rand -base64 32) -e GUARDAI_PROXY_API_KEY \
ghcr.io/oronts/oguardai/proxy:0.1.0 \
--provider vertex-openai \
--target https://us-central1-aiplatform.googleapis.com \
--policy default --port 8081 --single-tenant --rate-limit-enabledThe client sends its normal Vertex request to the same path on the proxy with its
Authorization: Bearer <token> header. The proxy tokenizes the request through the shared
OpenAI-compatible pipeline, preserves the full project/location/endpoint path when forwarding, and
forwards the Bearer token unchanged. Only this OpenAI-compatible endpoint is protected; the native
:generateContent API uses a different body schema and is not yet supported (use the server
/v1/transform API for those payloads).
Bedrock
Amazon Bedrock is not supported by the transparent proxy. Bedrock signs each request with AWS SigV4 over
a hash of the request body, so tokenizing the body would invalidate the signature; protecting Bedrock
would require the proxy to hold AWS credentials and re-sign every request. --provider bedrock therefore
refuses to start with an explanation. Use the server /v1/transform API for Bedrock payloads.
Configuration
The proxy respects all OGuardAI policy settings. Select a policy by adding --policy <name> to the
Setup command above; a builtin-only policy such as default needs nothing else.
strict-pii (blocks SSN/IBAN/passport and masks person and company) and german-support (German
formal salutations) declare person detection as required, so the proxy refuses to start them in the
default builtin detector mode. Run the NER sidecar and start the proxy in advanced mode:
# NER sidecar, bound to loopback (it runs unauthenticated, so keep it on 127.0.0.1)
docker run -d --rm --network host -e GUARDAI_DETECTOR_ALLOW_INSECURE=true \
--name oguardai-detector ghcr.io/oronts/oguardai/detector-py:0.1.0 \
--host 127.0.0.1 --port 9090
# Proxy in advanced (NER-required) mode with the strict-pii policy. GUARDAI_PROXY_API_KEY is inherited
# from the value exported in Setup above.
docker run --rm --network host \
-e GUARDAI_SESSION_SECRET=$(openssl rand -base64 32) -e GUARDAI_PROXY_API_KEY \
-e GUARDAI_DETECTOR_MODE=advanced -e GUARDAI_DETECTOR_ADVANCED_URL=http://localhost:9090 \
ghcr.io/oronts/oguardai/proxy:0.1.0 \
--target https://api.openai.com --policy strict-pii --port 8081 --single-tenant --rate-limit-enabledFor german-support, run the same two commands with --policy german-support.
Detection and tenancy are configured through environment variables:
| Env var | Default | Purpose |
|---|---|---|
GUARDAI_DETECTOR_MODE | builtin | builtin (regex only), both (regex + NER best-effort), or advanced (NER required, fails closed) |
GUARDAI_DETECTOR_ADVANCED_URL | http://localhost:9090 | URL of the Python NER detector service |
GUARDAI_DETECTOR_API_KEY | - | Shared secret for the proxy-to-detector hop (X-Detector-API-Key). When set it must be at least 16 characters and not a placeholder value, refused at startup outside dev |
GUARDAI_DETECTOR_LANGUAGE | auto | Language hint forwarded to detection (ISO 639-1) when requests are known to be single-language |
GUARDAI_DETECTOR_CUSTOM | - | JSON array of custom detector patterns (same shape as a policy's custom_patterns), so the proxy detects deployment-specific identifiers |
GUARDAI_PROXY_TENANT_ID | - | Tenant id bound to every session the proxy creates, isolating this proxy's sessions in multi-tenant deployments. When set: non-empty, at most 128 bytes, and no whitespace or control characters. A malformed value fails the proxy startup |
GUARDAI_PROXY_METRICS_API_KEY | - | Optional dedicated credential for the GET /metrics scrape (the Observability role). When set it must differ from every ingress key and requires the ingress key to be set; either rule fails startup. With it set, the ingress key serves only the data plane (chat, /v1/embeddings, /v1/capabilities) and this key serves only /metrics, carrying no tenant. Unset, the ingress key scrapes /metrics as before, so a single-key deployment is unchanged. Least-privilege for the scrape, not an exfiltration control: a leaked ingress key cannot call the upstream or read restored output. When set it must be at least 16 characters and not a placeholder value, refused at startup outside dev |
GUARDAI_PROXY_API_KEY_NEXT | - | Optional second accepted ingress key for a zero-downtime rotation overlap. It shares the tenant, rate bucket, and audit attribution of GUARDAI_PROXY_API_KEY; only the key differs. When set it must differ from the primary ingress key and from any metrics key, and requires the primary to be set; either rule fails startup. While set, GET /metrics gains a proxy-local guardai_proxy_ingress_auth_total{key_slot="current"|"next"} counter so an operator can watch the retiring key drain to zero. Unset, one ingress key is accepted and /metrics is unchanged. See the key rotation runbook. When set it must be at least 16 characters and not a placeholder value, refused at startup outside dev |
GUARDAI_PROXY_SYSTEM_PREAMBLE | false | Opt-in: inject the prompt-security system preamble into forwarded chats (off by default so token usage is unchanged) |
For a multi-replica (HA) proxy, share the request-replay store across replicas with
GUARDAI_REPLAY_BACKEND=redis + GUARDAI_REDIS_URL (default memory is per-replica, so a
replayed continuation could otherwise slip through on another replica). An unknown backend or an
unreachable Redis fails the proxy startup closed.
A multi-replica proxy MUST also share the revocation store, or a value revoked on the server or
another replica stays restorable through this ingress. Set GUARDAI_REVOCATION_BACKEND=redis +
GUARDAI_REDIS_URL (the same Redis the server uses) and the same GUARDAI_SESSION_SECRET as the
server: revocation digests are keyed to that secret, and the shared store binds to it on first use and
refuses to start a process presenting a different secret, so a mismatch fails closed rather than
silently splitting the two into separate revocation sets. Advertising HA with
GUARDAI_REPLAY_BACKEND=redis or GUARDAI_HIGH_AVAILABILITY=true while leaving revocation on the
per-replica memory default fails the proxy startup closed.
Rate Limiting
A non-dev proxy must run a bounded request limiter: startup is refused when it is off unless
GUARDAI_PROXY_DEV=1 is set, which is a local dev opt-out. Enable the per-client token-bucket limiter
with GUARDAI_PROXY_RATE_LIMIT_ENABLED=true (or --rate-limit-enabled; the env var wins over the
flag). Tune GUARDAI_PROXY_RATE_LIMIT_RPS (default 100) and GUARDAI_PROXY_RATE_LIMIT_BURST (default
200). Enabling it with an rps or burst below 1, or a non-numeric value, aborts startup. When the limit
is exceeded the proxy returns 429 with Retry-After: 1 in the caller's provider error shape.
The limiter runs in two layers. A per-client-IP layer runs ahead of authentication and charges only
failed authentication, so a flood of invalid X-GuardAI-API-Key values is answered with 429 before it
costs key comparison or log volume, while a valid caller and a relayed upstream 401 never spend its
budget. Tune it with GUARDAI_PROXY_AUTH_RATE_LIMIT_RPS (default 5) and
GUARDAI_PROXY_AUTH_RATE_LIMIT_BURST (default 10); set the rps to 0 to disable this layer, or with
GUARDAI_PROXY_RATE_LIMIT_ENABLED on, a positive rps with a zero burst aborts startup; with the switch
off the limiter does not run, but a malformed rps or burst value is still rejected at startup. The per-tenant layer runs after authentication and caps
the overall request rate with the GUARDAI_PROXY_RATE_LIMIT_RPS/_BURST knobs above. Both resolve the
client IP with the same trusted-proxy rules described below, and every 429 increments
guardai_rate_limit_rejections_total on GET /metrics.
Requests are bucketed by the authenticated tenant when auth is enabled. Without a tenant the bucket key
is the client IP. By default (GUARDAI_PROXY_RATE_LIMIT_TRUSTED_PROXY_DEPTH=0) that IP is the TCP peer
address, which a client cannot forge. If the proxy runs behind one or more reverse proxies that set
X-Forwarded-For, set GUARDAI_PROXY_RATE_LIMIT_TRUSTED_PROXY_CIDRS (or
--rate-limit-trusted-proxy-cidrs) to the proxy address range; the client IP is then resolved by a
rightmost-trusted walk over the header. The depth flag alone is refused outside a dev proxy
(GUARDAI_PROXY_DEV=1), because a direct peer could forge the header and dodge the rate limit. Leaving
both unset behind an L7 proxy shares one bucket across all clients, so set the trusted CIDRs. Behind a
shared NAT or corporate egress the per-client-IP layer groups distinct clients into one bucket, but it
counts only their failed authentications, so legitimate traffic from that address is not throttled while
the address still has failed-auth budget. Once a shared address has exhausted that budget, every request
from it is shed with 429 before authentication, valid keys included, until the budget refills at the
configured rate. Under a concurrent flood a handful of requests beyond the burst can still reach the key
compare; the per-second rate bounds the long-run cost. Set the trusted CIDRs to separate the failed-auth
budgets if several clients share the egress.
How Token Restoration Works
When the LLM generates output containing builtin-detected tokens like {{email:e_001:1bcaef1a4aff}}, the proxy
automatically restores them using the session state that was created during the
request transformation. The restore mode (full, partial, masked, formatted, abstract)
is controlled by the policy configuration (none removes the value entirely).
For example, with builtin detection:
- Input: "Contact erika.mustermann@example.com or IP 10.0.0.1 for help"
- To provider: "Contact
{{email:e_001:1bcaef1a4aff}}or IP{{ip:ip_001:cd9903b926e3}}for help" - From provider: "I'll reach out to
{{email:e_001:1bcaef1a4aff}}..." - To your app: "I'll reach out to erika.mustermann@example.com..."
In the default
builtinmode the proxy covers structured patterns only (emails, phones, IPs, IBANs, SSNs, etc.). To also mask person/company/location names, run the proxy with the NER sidecar (GUARDAI_DETECTOR_MODE=bothoradvanced, see Configuration), or use the full server API.
Architecture
Your App
|
| (standard OpenAI SDK calls)
v
OGuardAI Proxy (port 8081)
|
| 1. Transform: mask PII in request
| 2. Forward: send safe request to OpenAI
| 3. Rehydrate: restore tokens in response
|
v
OpenAI API (never sees real PII)Limitations
- The proxy adds a small latency overhead for PII detection and token restoration
- All message roles (including system) are scanned for PII
- The proxy requires network access to both your app and the target API
- Session state is per-request; multi-turn conversations need session continuity (the proxy handles this automatically via sealed session blobs)
- The proxy scans text output only. Generated media (audio, images, documents), an inline
data:payload outside a text field, and structured output shapes such as a content array are refused rather than forwarded, because their bytes cannot be scanned for PII - A bare base64 blob carried in a vendor extension field with no media declaration is governed by the text scanning posture, not by the generated-media refusal, so it is treated as opaque text rather than refused
Troubleshooting
Tokens not being restored:
Check that the LLM is faithfully reproducing the token format {{type:id:cap}}.
OGuardAI includes a 3-stage token repair pipeline (strict, repair, fuzzy) that
handles common LLM token mangling.
Performance:
In builtin (regex) mode the proxy adds only a small detection and restoration overhead on short inputs; enabling the NER sidecar (both or advanced) adds more. Measure it against your own inputs and hardware rather than assuming a fixed figure.
For large inputs, consider using the chunking API directly.
Runnable examples
Three end-to-end examples live in the repo examples/ directory. Each one points a stock provider
SDK at the proxy and changes nothing else about the calling code. Every example ships two run modes:
- Explain (the default): calls the same runtime contract the proxy uses internally (
/v1/transformand/v1/rehydrate) directly against the OGuardAI server, with no LLM call and no provider key. It uses only the language standard library, so it runs with nothing installed beyond the server. It prints what leaves your network, a leak check, the safe metadata the model receives, and the full restore lattice. - Live (
--live): the stock provider SDK pointed at the proxy. The only change from a normal integration is the base URL and theX-GuardAI-API-Keyheader.
Jump to:
- OpenAI Python, customer support desk
- OpenAI Node, sales copilot
- Anthropic Python, insurance claims desk
OpenAI Python, customer support desk
A support desk for Musterfirma (a fictional US online outdoor retailer) asks an OpenAI model
to draft the reply to a customer's order email, while the customer's name, email, phone, address,
customer number, order number, and bank IBAN never leave the network in the clear. Directory:
examples/openai-proxy-python/. Policy: customer-support-en.
Prerequisite (needs-NER): a running OGuardAI server. This example masks the person name and postal address, which needs the NER detector sidecar; the bundled Docker stack starts in
bothmode (builtin regex plus NER).customer-support-endeclares person and company detection as required, so a server without the NER sidecar rejects the transform (fail closed) rather than let an un-tokenized name reach the model.
Start the runtime, then run the offline explain path (no OpenAI key, no proxy, no pip install):
# Start the OGuardAI runtime (server + NER detector). NER is needed for person/address. The bundled
# stack requires these four secrets, and its client key ships least-privilege, so widen it to the
# customer-support-en policy and the channels this walkthrough restores on (user_output,
# internal_summary).
export GUARDAI_SESSION_SECRET=$(openssl rand -base64 32)
export GUARDAI_API_KEY_1=$(openssl rand -hex 32)
export GUARDAI_DETECTOR_API_KEY=$(openssl rand -hex 32)
export GUARDAI_REDIS_PASSWORD=$(openssl rand -hex 32)
export GUARDAI_API_KEY_1_ALLOWED_POLICIES=default,default-ner-required,german-support,customer-support-en
export GUARDAI_API_KEY_1_ALLOWED_OUTPUT_CHANNELS=user_output,internal_summary
docker compose -f deploy/docker/docker-compose.yml up -d
# Run the walkthrough. No OpenAI key, no proxy, no pip install required.
export GUARDAI_URL=http://localhost:3000
export GUARDAI_API_KEY="$GUARDAI_API_KEY_1" # the bundled stack runs api_key auth
python main.pyThe email the example reads (examples/openai-proxy-python/order_issue.txt):
From: erika.mustermann@example-mail.com
To: support@example.com
Subject: Wrong size delivered, order MO-2026-88214
Hello Musterfirma Support,
my name is Ms. Erika Mustermann (customer number CUST-583920). I ordered the
Ridgeline down jacket in medium, but order MO-2026-88214 arrived in extra small
and I would like to exchange it for the correct size.
You can reach me by email at erika.mustermann@example-mail.com or by phone at
+1 (415) 555-0182. My delivery address is 1420 Example Street, Portland, OR 97205.
If a refund turns out to be easier than an exchange, please send it to my bank
account, IBAN GB29 NWBK 6016 1331 9268 19.
Thanks for your help,
Erika MustermannScenario constants, the system prompt (which names no customer data), and the custom order-id
pattern that declares this tenant's order scheme to the detector
(examples/openai-proxy-python/main.py):
POLICY = "customer-support-en"
LANGUAGE = "en"
HERE = Path(__file__).resolve().parent
ORDER_FILE = HERE / "order_issue.txt"
# System instruction the agent gives the model. It names no customer data: the model works entirely
# from tokens plus the metadata OGuardAI attaches to each one.
SYSTEM_PROMPT = (
"You are a customer-support assistant for Musterfirma. Write a short, courteous reply "
"in English that confirms the order, acknowledges the wrong-size delivery, and explains the "
"exchange or refund next steps. The message contains {{type:id:cap}} placeholders that stand for "
"real data you must not see. Copy every placeholder verbatim; never decode, translate, "
"reformat, or invent one, and never write a real name, email, phone number, or IBAN yourself. "
"Do not write a salutation word before a person placeholder: the runtime inserts the correct "
"Mr./Ms. honorific itself. Open with 'Hello {{person:...}},'."
)
STRUCTURED_PII = {
"customer email": "erika.mustermann@example-mail.com",
"support email": "support@example.com",
"phone": "+1 (415) 555-0182",
"customer id": "CUST-583920",
"order": "MO-2026-88214",
"iban": "GB29 NWBK 6016 1331 9268 19",
}
NER_PII = {
"name": "Erika Mustermann",
"address": "1420 Example Street",
}
# The built-in order detector knows common prefixes, not this tenant's scheme, so the order id
# would reach the model raw. A custom pattern is how a deployment declares its own identifiers.
CUSTOM_PATTERNS = [
{"entity_type": "order_ref", "pattern": r"\bMO-\d{4}-\d{5}\b", "confidence": 0.99}
]The explain path drives the runtime through a thin client that posts to /v1/transform and
/v1/rehydrate (examples/openai-proxy-python/og_runtime.py):
def transform(
text: str,
policy: str,
language: str | None = None,
context: dict[str, Any] | None = None,
custom_patterns: list[dict[str, Any]] | None = None,
) -> dict[str, Any]:
"""Detect + tokenize `text` under `policy`. Returns safe_text, session_state, entity_context."""
payload: dict[str, Any] = {"input": text, "policy": policy}
if language:
payload["language"] = language
if context:
payload["context"] = context
if custom_patterns:
payload["custom_patterns"] = custom_patterns
return _post("/v1/transform", payload)
def rehydrate(
output: str,
session_state: str,
*,
output_channel: str | None = None,
restore_mode: str | None = None,
restore_overrides: dict[str, str] | None = None,
) -> dict[str, Any]:
"""Restore tokens in `output` under the session, governed by channel / restore mode."""
payload: dict[str, Any] = {"output": output, "session_state": session_state}
if output_channel:
payload["output_channel"] = output_channel
if restore_mode:
payload["restore_mode"] = restore_mode
if restore_overrides:
payload["restore_overrides"] = restore_overrides
return _post("/v1/rehydrate", payload)Run it live against the real proxy and OpenAI. The live proxy pins the SAME policy the explain path
uses (customer-support-en), which requires NER for the person and company, so the proxy runs in
advanced mode with the NER sidecar up. The proxy also takes the tenant order pattern through the
GUARDAI_DETECTOR_CUSTOM env var, because the SDK request cannot carry the per-request custom
patterns the explain path passes to /v1/transform. With both in place, live and explain protect
the same values:
pip install -r requirements.txt # OpenAI SDK, only needed for --live
# This example masks person names (and exposes gender/formality metadata), which needs NER, so run
# the detector in the host network namespace and put the proxy in `advanced` mode (NER required, fail
# closed on a sidecar outage). `customer-support-en` declares person and company detection as
# required, so the proxy refuses to start in `builtin` or `both` mode. `--host 127.0.0.1` keeps the
# unauthenticated detector bound to loopback, reachable by the proxy on the same host but not exposed
# on any other interface.
docker run -d --rm --network host -e GUARDAI_DETECTOR_ALLOW_INSECURE=true \
--name oguardai-detector ghcr.io/oronts/oguardai/detector-py:0.1.0 \
--host 127.0.0.1 --port 9090 # NER sidecar bound to loopback :9090
# Start the proxy in front of OpenAI, from the published image (the release archive ships the CLI and
# server only, not a proxy binary). --network host lets the proxy reach the detector on
# localhost:9090 and serve on the host's localhost:8081. GUARDAI_PROXY_API_KEY is the value the client
# sends as X-GuardAI-API-Key; the SDK's OPENAI_API_KEY bearer token is forwarded upstream unchanged.
# --policy pins the same policy the explain path uses, and GUARDAI_DETECTOR_CUSTOM declares the tenant
# order id pattern the explain path passes per request, so the proxy masks it too.
export GUARDAI_SESSION_SECRET=$(openssl rand -base64 32)
export GUARDAI_PROXY_API_KEY=$(openssl rand -hex 32)
export GUARDAI_DETECTOR_MODE=advanced
export GUARDAI_DETECTOR_ADVANCED_URL=http://localhost:9090
export GUARDAI_DETECTOR_CUSTOM='[{"entity_type":"order_ref","pattern":"\\bMO-\\d{4}-\\d{5}\\b","confidence":0.99}]'
docker run --rm --network host \
-e GUARDAI_SESSION_SECRET -e GUARDAI_PROXY_API_KEY \
-e GUARDAI_DETECTOR_MODE -e GUARDAI_DETECTOR_ADVANCED_URL -e GUARDAI_DETECTOR_CUSTOM \
ghcr.io/oronts/oguardai/proxy:0.1.0 \
--port 8081 --target https://api.openai.com --policy customer-support-en \
--single-tenant --rate-limit-enabled
# In another shell, point the SDK at the proxy and run for real.
export OPENAI_API_KEY="sk-..." # your real OpenAI key, read from the environment
export GUARDAI_PROXY_API_KEY="<same-as-above>"
export GUARDAI_MODEL="gpt-4o-mini" # optional, via OPENAI_MODEL
python main.py --liveThe only change from a normal OpenAI integration (examples/openai-proxy-python/main.py, run_live):
def run_live(order: str) -> int:
# Imported lazily so the explain path needs no third-party packages.
try:
from openai import OpenAI
except ImportError:
print("Live mode needs the OpenAI SDK: pip install openai", file=sys.stderr)
return 2
proxy_url = os.environ.get("GUARDAI_PROXY_URL", "http://localhost:8081") + "/v1"
proxy_key = os.environ.get("GUARDAI_PROXY_API_KEY", "")
if not os.environ.get("OPENAI_API_KEY"):
print("Live mode needs OPENAI_API_KEY (forwarded by the proxy to OpenAI).", file=sys.stderr)
return 2
# The only change from a normal OpenAI integration: base_url points at the proxy, and the
# proxy's own auth key rides in X-GuardAI-API-Key. The SDK still sends OPENAI_API_KEY as its
# bearer token, which the proxy forwards upstream. Everything else is stock OpenAI.
client = OpenAI(
base_url=proxy_url,
default_headers={"X-GuardAI-API-Key": proxy_key},
)
model = os.environ.get("OPENAI_MODEL", "gpt-4o-mini")
print(RULE)
print(f"LIVE MODE: OpenAI SDK -> OGuardAI proxy ({proxy_url}) -> OpenAI ({model})")
print(RULE)
completion = client.chat.completions.create(
model=model,
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
{"role": "user", "content": order},
],
)
# The model saw only tokens; the proxy restored real values on the user_output channel before
# this response reached the SDK.
print("\nRestored draft returned by the proxy:\n")
print(completion.choices[0].message.content)
return 0Dependencies (examples/openai-proxy-python/requirements.txt), only needed for --live:
openai>=1.0.0Expected output (explain path). Token ids and fingerprints vary per run; the shape is what matters:
[1] What leaves your network (sent to OpenAI):
... my name is {{person:p_001:...}} (customer number {{customer_id:cid_001:...}}) ...
... order {{order:o_001:...}} ... email at {{email:e_001:...}} ... phone at {{phone:ph_001:...}} ...
... IBAN {{iban:ib_001:...}}.
[1b] Leak check on the outbound text:
PASS, none of the 8 raw values are present. detector: builtin_and_ner
[2] Safe metadata the model receives with the tokens:
type gender formality role lang
email - - customer en
person female formal customer en
...
[4] What your OpenAI client receives back (user_output restore):
Hello Ms. Erika Mustermann, ... (IBAN stays masked)
[5] Restore-mode ladder on the salutation (same session, same draft):
full -> Hello Erika Mustermann,
formatted -> Hello Ms. Erika Mustermann,
partial -> Hello E. Mustermann,
masked -> Hello E**** *********n,
none -> Hello [REDACTED],
[6] Role-gated tightening (caller_role=support_agent, internal_summary channel):
agent console sees -> Hello E. Mustermann,
[7] Fail-closed guards:
tampered session -> rejected with GUARDAI_INVALID_INPUT (HTTP 400); no raw value returned
forged token -> resolved=0 unresolved=['{{iban:ib_999:deadbeef0000}}'] (left verbatim, never guessed)OpenAI Node, sales copilot
A sales copilot for Musterfirma (a fictional B2B SaaS vendor) drafts a follow-up to a prospect's
inbound email, while the prospect's name, company, email, phone, the CRM opportunity and account ids,
and a lead score never leave the network in the clear. Directory: examples/openai-proxy-node/.
Policy: crm-sales. The explain path uses the global fetch built into modern Node.js.
Prerequisite (needs-NER): a running OGuardAI server with the NER detector active.
crm-saleslists person and company underdetection.required_for, so a runtime without NER refuses the transform (fail closed) rather than let a prospect or account name reach the model. Run the bundled Docker stack (bothmode) so the full walkthrough executes.
Start the runtime, then run the offline explain path (no OpenAI key, no proxy, no npm install of
openai):
# Start the OGuardAI runtime (the product under test). NER is needed for person/company/address. The
# bundled stack requires these four secrets, and its client key ships least-privilege, so widen it to
# the crm-sales policy and the channels this walkthrough restores on (user_output, crm_sync,
# sales_export, log_safe).
export GUARDAI_SESSION_SECRET=$(openssl rand -base64 32)
export GUARDAI_API_KEY_1=$(openssl rand -hex 32)
export GUARDAI_DETECTOR_API_KEY=$(openssl rand -hex 32)
export GUARDAI_REDIS_PASSWORD=$(openssl rand -hex 32)
export GUARDAI_API_KEY_1_ALLOWED_POLICIES=default,default-ner-required,german-support,crm-sales
export GUARDAI_API_KEY_1_ALLOWED_OUTPUT_CHANNELS=user_output,crm_sync,sales_export,log_safe
docker compose -f deploy/docker/docker-compose.yml up -d
# Run the walkthrough. No OpenAI key, no proxy, no npm install of openai required.
export GUARDAI_URL=http://localhost:3000
export GUARDAI_API_KEY="$GUARDAI_API_KEY_1" # the bundled stack runs api_key auth
npx tsx index.tsThe prospect email the example reads (examples/openai-proxy-node/prospect_inbound.txt):
From: erika.mustermann@example.com
To: max.mustermann@example.com
Subject: Re: Musterfirma proposal, opportunity OPP-2026-4471
Hi Max,
thanks for the walkthrough yesterday. This is Erika Mustermann, VP Operations at
Beispielfirma and the economic buyer on our side.
We are moving forward internally on opportunity OPP-2026-4471 against account
ACCT-88213. Our lead score in your system is lead score: 82, which sounds about
right given our timeline.
Could you send a follow-up summarizing the enterprise tier and next steps? You can
reach me at erika.mustermann@example.com or on +49 89 4455123.
Best,
ErikaThe fixture that proves the hard financial floor. crm-sales uses on_redact: reject, so a pasted
card blocks the whole request and the model is never called (examples/openai-proxy-node/card_paste.txt):
Quick internal note before the call: the prospect accidentally pasted a corporate
card, 4111 1111 1111 1111, into the email thread for opportunity OPP-2026-4471.
Please scrub it from the record.Scenario constants, the system prompt, and the tracked raw values
(examples/openai-proxy-node/index.ts):
const POLICY = "crm-sales";
const LANGUAGE = "en";
const HERE = dirname(fileURLToPath(import.meta.url));
// System instruction the rep gives the model. It names no prospect data: the model works entirely
// from tokens plus the metadata OGuardAI attaches to each one.
const SYSTEM_PROMPT = [
"You are a B2B sales copilot for Musterfirma. Draft a concise, friendly follow-up email to a",
"prospect that summarizes the enterprise tier and proposes next steps. The message contains",
"{{type:id:cap}} placeholders that stand for real data you must not see. Copy every placeholder",
"verbatim; never decode, translate, reformat, or invent one, and never write a real name, company,",
"email, or deal id yourself. Do not write a salutation word before a person placeholder: the",
"runtime inserts the correct honorific. Open with 'Hi {{person:...}},'.",
].join(" ");
// Structured identifiers the runtime detects deterministically: builtin regex for the emails and
// phone, and the crm-sales custom_patterns for the opportunity and account ids. They must never
// reach the model on any deployment, so their presence in the outbound text is a hard failure.
const STRUCTURED_PII = [
"erika.mustermann@example.com",
"max.mustermann@example.com",
"+49 89 4455123",
"OPP-2026-4471",
"ACCT-88213",
];
// Prospect and account names. These are NER-only: the regex floor detects no names, so a runtime
// without NER cannot tokenize them. crm-sales lists person and company under detection.required_for,
// so such a runtime refuses the request instead of leaking them.
const NER_PII = ["Erika Mustermann", "Beispielfirma"];The explain path drives the runtime through a thin client that posts to /v1/transform and
/v1/rehydrate (examples/openai-proxy-node/og_runtime.ts):
export function transform(
input: string,
opts: { policy: string; language?: string; context?: TransformContext },
): Promise<TransformResult> {
return post<TransformResult>("/v1/transform", {
input,
input_type: "text",
policy: opts.policy,
language: opts.language,
context: opts.context,
});
}
export function rehydrate(opts: {
output: string;
sessionState: string;
channel?: string;
restoreMode?: string;
}): Promise<RehydrateResult> {
return post<RehydrateResult>("/v1/rehydrate", {
output: opts.output,
session_state: opts.sessionState,
output_channel: opts.channel,
restore_mode: opts.restoreMode,
});
}Run it live against the real proxy and OpenAI. Start the NER sidecar and the proxy as in the OpenAI
Python example above (target https://api.openai.com, port 8081, advanced mode), but pass
--policy crm-sales, the policy the explain path uses. crm-sales requires NER for the prospect and
account names, and it declares its deal, account, and lead-score patterns in the policy file, so no
GUARDAI_DETECTOR_CUSTOM is needed. Then run the Node client:
npm install # installs the openai SDK (declared in package.json), only needed for --live
# In another shell, point the SDK at the proxy and run for real.
export OPENAI_API_KEY="sk-..." # your real OpenAI key, read from the environment
export GUARDAI_PROXY_API_KEY="<same-as-above>"
export OPENAI_MODEL="gpt-4o-mini" # optional
npx tsx index.ts --liveThe only change from a normal OpenAI integration (examples/openai-proxy-node/index.ts, runLive):
async function runLive(): Promise<number> {
const proxyUrl = `${process.env.GUARDAI_PROXY_URL ?? "http://localhost:8081"}/v1`;
const proxyKey = process.env.GUARDAI_PROXY_API_KEY ?? "";
if (!process.env.OPENAI_API_KEY) {
console.error("Live mode needs OPENAI_API_KEY (forwarded by the proxy to OpenAI).");
return 2;
}
// Imported lazily so the explain path needs no third-party packages.
const { default: OpenAI } = await import("openai");
// The only change from a normal OpenAI integration: baseURL points at the proxy, and the proxy's
// own auth key rides in X-GuardAI-API-Key. The SDK still sends OPENAI_API_KEY as its bearer token,
// which the proxy forwards upstream. Everything else is stock OpenAI.
const client = new OpenAI({
baseURL: proxyUrl,
defaultHeaders: { "X-GuardAI-API-Key": proxyKey },
});
const model = process.env.OPENAI_MODEL ?? "gpt-4o-mini";
console.log(RULE);
console.log(`LIVE MODE: OpenAI SDK -> OGuardAI proxy (${proxyUrl}) -> OpenAI (${model})`);
console.log(RULE);
const completion = await client.chat.completions.create({
model,
messages: [
{ role: "system", content: SYSTEM_PROMPT },
{ role: "user", content: fixture("prospect_inbound.txt") },
],
});
// The model saw only tokens; the proxy restored real values on the user_output channel before
// this response reached the SDK.
console.log("\nRestored draft returned by the proxy:\n");
console.log(completion.choices[0]?.message.content ?? "");
return 0;
}Dependencies and scripts (examples/openai-proxy-node/package.json):
{
"name": "guardai-openai-proxy-node",
"version": "1.0.0",
"description": "OpenAI Node SDK with OGuardAI transparent PII proxy",
"type": "module",
"scripts": {
"typecheck": "tsc --noEmit",
"test": "npx tsx --test og_runtime.test.ts",
"start": "npx tsx index.ts",
"explain": "npx tsx index.ts --explain",
"live": "npx tsx index.ts --live"
},
"dependencies": {
"openai": "^4.0.0"
},
"devDependencies": {
"@types/node": "^20",
"tsx": "^4.0",
"typescript": "^5.5"
}
}Expected output (explain path). Token ids and fingerprints vary per run; the shape is what matters:
[1] What leaves your network (sent to OpenAI):
... This is {{person:p_001:...}}, VP Operations at {{company:c_001:...}} ...
... opportunity {{custom:deal_id:x_001:...}} against account {{custom:account_id:x_002:...}} ...
... lead score: {{custom:lead_score:x_003:...}} ... reach me at {{email:e_001:...}} or on {{phone:ph_001:...}}.
[1b] Leak check on the outbound text:
PASS, none of the 7 raw values are present. detector: builtin_and_ner
[4] One session, four readers (same draft, different restore):
--- channel=user_output (...) --- opportunity OPP-2026-4471 for Beispielfirma, real email
--- channel=crm_sync (...) --- ids and contacts masked
--- channel=sales_export (...) --- names abstracted, contacts dropped
--- channel=log_safe (...) --- everything redacted
[5] Role-gated deal id (user_output channel, same inbound):
caller_role=sales_rep -> ... opportunity OPP-2026-4471 ...
no role -> ... opportunity ******** ...
[6] Fail-closed guards:
a) tampered session -> rejected with GUARDAI_INVALID_INPUT (HTTP 400); no raw value returned
b) forged token -> resolved=0 unresolved=["{{custom:deal_id:x_999:deadbeef0000}}"] (left verbatim, never guessed)
c) pasted card -> whole request blocked: GUARDAI_POLICY_DENIED (HTTP 403); the model was never calledAnthropic Python, insurance claims desk
An AI claims desk for Musterversicherung AG (a fictional German motor insurer) asks Claude to
draft the formal German acknowledgement of a claim email, while the policyholder name, two emails,
phone, postal address, German tax ID, bank IBAN, policy and claim numbers, vehicle VIN and licence
plate, and a reserve amount never leave the network in the clear. The same proxy binary that fronts
OpenAI also fronts Anthropic on /v1/messages. Directory: examples/anthropic-proxy-python/.
Policy: insurance-de. All data in the fixture is fictional.
Prerequisite (needs-NER): a running OGuardAI server with the NER detector active (person and address detection need it). The
insurance-depolicy requires NER for the policyholder name, so a runtime without NER fails closed rather than let an un-tokenized name cross. The live path runs the proxy inadvancedmode (NER required, fail closed on a sidecar outage).
Start the runtime, then run the offline explain path (Python standard library only, nothing to install):
# The bundled stack requires these four secrets, and its client key ships least-privilege, so widen it
# to the insurance-de policy and the channels this walkthrough restores on (user_output,
# customer_email, reinsurer_export, log_safe).
export GUARDAI_SESSION_SECRET=$(openssl rand -base64 32)
export GUARDAI_API_KEY_1=$(openssl rand -hex 32)
export GUARDAI_DETECTOR_API_KEY=$(openssl rand -hex 32)
export GUARDAI_REDIS_PASSWORD=$(openssl rand -hex 32)
export GUARDAI_API_KEY_1_ALLOWED_POLICIES=default,default-ner-required,german-support,insurance-de
export GUARDAI_API_KEY_1_ALLOWED_OUTPUT_CHANNELS=user_output,customer_email,reinsurer_export,log_safe
docker compose -f deploy/docker/docker-compose.yml up -d
export GUARDAI_BASE_URL=http://localhost:3000
export GUARDAI_API_KEY="$GUARDAI_API_KEY_1" # the bundled stack runs api_key auth
python main.pyThe claim email the example reads (examples/anthropic-proxy-python/claim_email.txt):
Von: erika.mustermann@example-mail.de
An: schaden@example.de
Betreff: Schadenmeldung Kaskoversicherung, Auffahrunfall vom 04.07.2026
Sehr geehrte Damen und Herren,
ich, Frau Dr. Erika Mustermann, Versicherungsnehmerin, moechte einen Schaden
melden. Am 04.07.2026 ist mir auf der A9 ein Fahrzeug aufgefahren.
Meine Kontaktdaten fuer Rueckfragen: E-Mail erika.mustermann@example-mail.de,
Telefon +49 89 1122334. Anschrift: Musterstrasse 47, 80331 Muenchen.
Zum Vertrag: Versicherungsschein-Nr. VS-4820091, Schadennummer SCH-2026-55831.
Meine Steuer-Identifikationsnummer lautet 86 095 742 719.
Betroffen ist mein Fahrzeug, Fahrgestellnummer WVWZZZ1KZAW008421,
amtliches Kennzeichen M-KV 2847.
Bitte ueberweisen Sie eine etwaige Erstattung auf meine IBAN
DE97 9900 0000 0123 4567 89. Nach Einschaetzung meiner Werkstatt liegt der
Schaden bei rund 18.500 EUR.
Bitte bestaetigen Sie mir kurz den Eingang und die weiteren Schritte.
Mit freundlichen Gruessen
Dr. Erika MustermannScenario constants, the German system prompt, and the tracked raw values
(examples/anthropic-proxy-python/main.py):
POLICY = "insurance-de"
LANGUAGE = "de"
HERE = Path(__file__).resolve().parent
CLAIM_FILE = HERE / "claim_email.txt"
# System instruction the agent gives Claude. It intentionally names no customer data: the model
# works entirely from tokens plus the metadata OGuardAI attaches to each one.
SYSTEM_PROMPT = (
"Du bist die Schadenassistenz der Musterversicherung AG. Formuliere eine kurze, formelle "
"Eingangsbestaetigung auf Deutsch (Sie-Form). Sprich die Person mit korrekter Anrede an, "
"nenne Versicherungsschein- und Schadennummer, bestaetige die hinterlegte Bankverbindung fuer "
"eine etwaige Erstattung und kuendige die Pruefung an. Behandle jeden {{typ:id:pruefsumme}}-Platzhalter als "
"unveraenderlichen Baustein: niemals dekodieren, umschreiben oder erfinden."
)
STRUCTURED_PII = [
"erika.mustermann@example-mail.de",
"schaden@example.de",
"+49 89 1122334",
"86 095 742 719",
"DE97 9900 0000 0123 4567 89",
"18.500",
"VS-4820091",
"SCH-2026-55831",
"WVWZZZ1KZAW008421",
"M-KV 2847",
]
NER_PII = [
"Erika Mustermann",
]The explain path drives the runtime through a thin client that posts to /v1/transform and
/v1/rehydrate (examples/anthropic-proxy-python/og_runtime.py):
def transform(text: str, policy: str, language: str | None = None) -> dict[str, Any]:
"""Detect + tokenize `text` under `policy`. Returns safe_text, session_state, entity_context."""
payload: dict[str, Any] = {"input": text, "policy": policy}
if language:
payload["language"] = language
return _post("/v1/transform", payload)
def rehydrate(
output: str,
session_state: str,
*,
output_channel: str | None = None,
restore_mode: str | None = None,
restore_overrides: dict[str, str] | None = None,
) -> str:
"""Restore tokens in `output` under the session, governed by channel / restore mode."""
payload: dict[str, Any] = {"output": output, "session_state": session_state}
if output_channel:
payload["output_channel"] = output_channel
if restore_mode:
payload["restore_mode"] = restore_mode
if restore_overrides:
payload["restore_overrides"] = restore_overrides
return _post("/v1/rehydrate", payload)["restored_text"]Run it live against the real proxy and Claude. Start the proxy pointed at Anthropic, with the
insurance-de policy and the NER detector in advanced mode:
export GUARDAI_SESSION_SECRET=$(openssl rand -base64 32)
export GUARDAI_PROXY_API_KEY=$(openssl rand -hex 32)
export GUARDAI_DETECTOR_MODE=advanced
export GUARDAI_DETECTOR_ADVANCED_URL=http://localhost:9090
# NER detector in the host network namespace, from the published image. `--host 127.0.0.1` keeps the
# unauthenticated detector bound to loopback, reachable by the proxy on the same host (localhost:9090)
# but not exposed on any other interface.
docker run -d --rm --network host -e GUARDAI_DETECTOR_ALLOW_INSECURE=true \
--name oguardai-detector ghcr.io/oronts/oguardai/detector-py:0.1.0 \
--host 127.0.0.1 --port 9090
# The proxy runs from the published image (the release archive ships the CLI and server only, not a
# proxy binary). --network host lets it reach the detector on localhost:9090 and serve on
# localhost:8081.
docker run --rm --network host \
-e GUARDAI_SESSION_SECRET -e GUARDAI_PROXY_API_KEY \
-e GUARDAI_DETECTOR_MODE -e GUARDAI_DETECTOR_ADVANCED_URL \
ghcr.io/oronts/oguardai/proxy:0.1.0 \
--provider anthropic --target https://api.anthropic.com \
--port 8081 --policy insurance-de --single-tenant --rate-limit-enabledThen run the client. The Anthropic SDK reads ANTHROPIC_API_KEY from the environment and the proxy
forwards it upstream. No key is hardcoded:
pip install -r requirements.txt
export ANTHROPIC_API_KEY=sk-ant-...
export GUARDAI_PROXY_API_KEY=<same value the proxy was started with>
export GUARDAI_PROXY_URL=http://localhost:8081
python main.py --liveThe only change from a normal Anthropic integration (examples/anthropic-proxy-python/main.py,
run_live):
def run_live(claim: str) -> int:
# Imported lazily so the explain path needs no third-party packages.
try:
import anthropic
except ImportError:
print("Live mode needs the Anthropic SDK: pip install anthropic", file=sys.stderr)
return 2
proxy_url = os.environ.get("GUARDAI_PROXY_URL", "http://localhost:8081")
proxy_key = os.environ.get("GUARDAI_PROXY_API_KEY", "")
if not os.environ.get("ANTHROPIC_API_KEY"):
print("Live mode needs ANTHROPIC_API_KEY (forwarded by the proxy to Claude).", file=sys.stderr)
return 2
# The only change from a normal Anthropic integration: base_url points at the proxy, and the
# proxy's own auth key rides in X-GuardAI-API-Key. The SDK still sends ANTHROPIC_API_KEY, which
# the proxy forwards upstream. Everything else is stock Anthropic.
client = anthropic.Anthropic(
base_url=proxy_url,
default_headers={"X-GuardAI-API-Key": proxy_key},
)
model = os.environ.get("ANTHROPIC_MODEL", "claude-sonnet-4-5")
print(RULE)
print(f"LIVE MODE: Anthropic SDK -> OGuardAI proxy ({proxy_url}) -> Claude ({model})")
print(RULE)
message = client.messages.create(
model=model,
max_tokens=512,
system=SYSTEM_PROMPT,
messages=[{"role": "user", "content": claim}],
)
text = "".join(block.text for block in message.content if block.type == "text")
# The model saw only tokens; the proxy restored real values on the user_output channel before
# this response reached the SDK.
print("\nRestored draft returned by the proxy:\n")
print(text)
return 0Dependencies (examples/anthropic-proxy-python/requirements.txt), only needed for --live:
anthropic>=0.40How the policy governs restoration. The insurance-de policy
(policies/insurance-de/policy.yaml) decides each value's base restore mode, then each channel may
only tighten it, which is why the same draft renders differently for each reader:
| Value | customer_email | reinsurer_export | log_safe |
|---|---|---|---|
| policyholder name | gender aware anrede | abstract | none |
| policy / claim no. | full | masked | masked |
| IBAN | masked | none | none |
| VIN / licence plate | masked | none | none |
| reserve amount | band (partial) | band (partial) | none |
Expected output (explain mode):
[1] What leaves your network (sent to Claude):
... ich, Frau Dr. {{person:p_001:...}}, Versicherungsnehmerin, ...
Versicherungsschein-Nr. {{custom:policy_number:x_001:...}}, Schadennummer {{custom:claim_id:x_002:...}}.
Meine Steuer-Identifikationsnummer lautet {{custom:de_tax_id:x_003:...}}.
[1b] Leak check on the outbound text:
PASS, none of the 11 raw values are present. detector: builtin_and_ner
[2] Safe metadata Claude receives with the tokens:
type gender formality role lang
person female formal policyholder de
...
[4] Restoration is governed per destination (same session, different reader):
--- channel=customer_email ---
... Versicherungsschein-Nr. VS-4820091 ... Bankverbindung D*** **** **** **** **** *0 ...
Rueckstellung liegt bei moderate (5k-50k).
--- channel=reinsurer_export ---
... Versicherungsschein-Nr. V********1 ... Bankverbindung [REDACTED] ...
--- channel=log_safe ---
Sehr geehrte [REDACTED], ... Versicherungsschein-Nr. [REDACTED] ...
[5] Fail-closed: a tampered session must not restore anything:
PASS, rejected with GUARDAI_INVALID_INPUT (HTTP 400); no raw value returned