OpenAI Drop-In Proxy

The Killer Use Case

Change one URL. Get enterprise PII protection. Zero code changes.

Before (unprotected)

client = OpenAI(api_key="sk-...")
response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Customer Julia Schneider (julia@example.com) needs help"}]
)
# Julia's name and email sent directly to OpenAI

After (protected by OGuardAI proxy)

The proxy runs on port 8081 by default (separate from the OGuardAI server on port 3000 or 8080). Point your SDK's base_url at the proxy:

client = OpenAI(
    api_key="sk-...",
    base_url="http://localhost:8081/v1"  # Only change: point to OGuardAI proxy (port 8081)
)
response = client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Contact julia@example.com or call +49 30 12345678"}]
)
# OpenAI sees: "Contact {{email:e_001:1bcaef1a4aff}} or call {{phone:ph_001:71453119be53}}"
# Response automatically restored with original values

How It Works

1. Your app sends request to OGuardAI proxy (port 8081)
2. Proxy masks PII in all user/assistant messages
3. Proxy forwards to real provider API
4. Provider responds with tokens (e.g., "{{email:e_001:1bcaef1a4aff}}")
5. Proxy restores tokens to real values
6. Your app receives the response with original values

Detection scope: By default (GUARDAI_DETECTOR_MODE unset, or builtin) the proxy uses builtin regex detectors and covers emails, phones, IPs, IBANs, SSNs, VAT IDs, URLs, credit cards, and other structured patterns, but NOT person, company, or location names. To also mask names, run the Python NER sidecar and start the proxy with GUARDAI_DETECTOR_MODE=both (builtin plus NER, regex fallback if the sidecar is down) or GUARDAI_DETECTOR_MODE=advanced (NER required, fail closed on a sidecar outage) and GUARDAI_DETECTOR_ADVANCED_URL=http://localhost:9090. In the default builtin mode a raw name can reach the model, so enable NER for name-bearing traffic.

Inline images: The proxy OCR-scans inline image bytes (OpenAI image_url data: URIs and Anthropic base64 image blocks). If the image renders detectable PII or cannot be scanned, the request is rejected, so PII inside an image never reaches the LLM. Clean images are forwarded unchanged. OCR requires Tesseract on the proxy host (apt install tesseract-ocr); without it, inline images are rejected fail-closed rather than forwarded unscanned. The maximum decoded image size is 5 MiB, which GUARDAI_PROXY_INLINE_IMAGE_MAX_BYTES can only lower. Remote image URLs are not fetched: the proxy cannot OCR-scan the pixels behind a URL, so by default a remote image URL is rejected fail-closed. An operator can opt in with GUARDAI_PROXY_ALLOW_REMOTE_IMAGE_URLS to forward remote URLs after a URL-string PII check only (the image pixels are still not scanned).

Other content blocks: Any non-text content block the proxy cannot scan (an Anthropic document block, an OpenAI file or input_audio block, or any unrecognized block type) is rejected fail-closed instead of being forwarded to the LLM unscanned. To send a document through AI safely, extract its text first and route it through the server /v1/transform API, which has a sandboxed document and PDF ingestion pipeline.

Provider credentials: The proxy forwards the upstream provider auth and routing headers (Authorization, x-api-key, OpenAI-Organization, OpenAI-Project, anthropic-version, anthropic-beta) to the provider unchanged, because they are required to authenticate and route the call. These are credential and routing values, not request data, so they are intentionally never scanned and never logged. Your OGuardAI proxy key (X-GuardAI-API-Key) authenticates you to the proxy and is never forwarded upstream.

Setup

# Start OGuardAI proxy (with auth)
GUARDAI_SESSION_SECRET=$(openssl rand -base64 32) \
GUARDAI_PROXY_API_KEY=$(openssl rand -hex 32) \
  oguardai-proxy --target https://api.openai.com --policy default --port 8081 --single-tenant

# Or with Docker
docker run -p 8081:8081 \
  -e GUARDAI_SESSION_SECRET=$(openssl rand -base64 32) \
  -e GUARDAI_PROXY_API_KEY=$(openssl rand -hex 32) \
  ghcr.io/oronts/oronts-guardai/oguardai-proxy:latest \
  --target https://api.openai.com --single-tenant \
  --policy default

Clients must include X-GuardAI-API-Key: <your-proxy-key> header. The upstream provider key (sk-...) is passed via the standard Authorization header by the OpenAI SDK.

Client Setup

from openai import OpenAI
import os

client = OpenAI(
    base_url="http://localhost:8081/v1",
    default_headers={"X-GuardAI-API-Key": os.environ["GUARDAI_PROXY_API_KEY"]},
)

Streaming Support

Streaming works transparently:

stream = client.chat.completions.create(
    model="gpt-4",
    messages=[...],
    stream=True  # Streaming works through proxy
)
for chunk in stream:
    print(chunk.choices[0].delta.content, end="")

What Gets Protected

Anthropic Support

Same pattern for Anthropic:

import os
client = Anthropic(
    api_key="sk-ant-...",
    base_url="http://localhost:8081",
    default_headers={"X-GuardAI-API-Key": os.environ["GUARDAI_PROXY_API_KEY"]},
)

OpenAI-compatible providers (Mistral and others)

Any provider that speaks the OpenAI /v1/chat/completions wire format is protected by the same handler. Point the proxy target at the provider and use it as usual. For Mistral:

oguardai-proxy --target https://api.mistral.ai --policy default --port 8081 --single-tenant
# or: GUARDAI_PROXY_TARGET=https://api.mistral.ai oguardai-proxy ...

The client sends its normal Mistral request to http://localhost:8081/v1/chat/completions with its Mistral API key in the Authorization: Bearer header (forwarded unchanged to the upstream). Message content, tool-call arguments, tool/function definitions, top-level user, and metadata are tokenized before the request leaves the runtime; buffered and streaming responses are rehydrated.

Restored tool-call arguments are redacted by default. The proxy cannot verify that the client's tool runs inside the trust boundary, so it does not return raw values into tool_calls arguments. A deployment whose tools are trusted and in-boundary opts into raw restoration with GUARDAI_PROXY_TOOL_PAYLOAD_RAW=true (or --tool-payload-raw); this flag is the only lever, a policy rule cannot widen it back to raw.

Mistral-specific request fields are handled correctly: prefix completion uses a boolean prefix flag on the last assistant message and the prefix text lives in content, which is scanned like any other message content; safe_prompt (boolean) and random_seed (number) carry no PII and pass through unchanged.

Scope: only /v1/chat/completions (and Anthropic /v1/messages) are intercepted and protected. /v1/embeddings is protected through the embedding path. Other Mistral endpoints such as /v1/fim/completions are not supported and are rejected at the proxy boundary (fail-closed, not forwarded), so raw PII never reaches an unprotected endpoint. Use the server /v1/transform API for those payloads.

Azure OpenAI

Azure uses a different URL scheme (/openai/deployments/{deployment}/chat/completions?api-version=...) and an api-key header instead of a Bearer token, so it needs an explicit provider mode. Set --provider azure (or GUARDAI_PROXY_PROVIDER=azure) and point the target at your Azure resource:

oguardai-proxy --provider azure \
  --target https://my-resource.openai.azure.com \
  --policy default --port 8081 --single-tenant

Clients send their normal Azure request to http://localhost:8081/openai/deployments/{deployment}/chat/completions?api-version=YYYY-MM-DD with their api-key header. The proxy tokenizes the request through the same OpenAI-compatible pipeline, preserves the deployment path and api-version query when forwarding, and forwards the api-key (and Authorization, for Entra ID) header unchanged. Azure embeddings (/openai/deployments/{deployment}/embeddings) are protected through the embedding path. The provider mode is never auto-detected from the target URL: you must set it explicitly.

Vertex AI (OpenAI-compatible)

Vertex AI exposes an OpenAI-compatible chat endpoint at /v1beta1/projects/{project}/locations/{location}/endpoints/{endpoint}/chat/completions (the endpoint segment is openapi for managed Gemini, or your endpoint id for a self-deployed model) with an OAuth2 Bearer token. Set --provider vertex-openai and target your regional aiplatform host:

oguardai-proxy --provider vertex-openai \
  --target https://us-central1-aiplatform.googleapis.com \
  --policy default --port 8081 --single-tenant

The client sends its normal Vertex request to the same path on the proxy with its Authorization: Bearer <token> header. The proxy tokenizes the request through the shared OpenAI-compatible pipeline, preserves the full project/location/endpoint path when forwarding, and forwards the Bearer token unchanged. Only this OpenAI-compatible endpoint is protected; the native :generateContent API uses a different body schema and is not yet supported (use the server /v1/transform API for those payloads).

Bedrock

Amazon Bedrock is not supported by the transparent proxy. Bedrock signs each request with AWS SigV4 over a hash of the request body, so tokenizing the body would invalidate the signature; protecting Bedrock would require the proxy to hold AWS credentials and re-sign every request. --provider bedrock therefore refuses to start with an explanation. Use the server /v1/transform API for Bedrock payloads.

Configuration

The proxy respects all OGuardAI policy settings:

# Use a specific policy
oguardai-proxy --target https://api.openai.com --policy strict-pii --port 8081

# With German formal restore
oguardai-proxy --target https://api.openai.com --policy german-support --port 8081

Detection and tenancy are configured through environment variables:

For a multi-replica (HA) proxy, share the request-replay store across replicas with GUARDAI_REPLAY_BACKEND=redis + GUARDAI_REDIS_URL (default memory is per-replica, so a replayed continuation could otherwise slip through on another replica). An unknown backend or an unreachable Redis fails the proxy startup closed.

A multi-replica proxy MUST also share the revocation store, or a value revoked on the server or another replica stays restorable through this ingress. Set GUARDAI_REVOCATION_BACKEND=redis + GUARDAI_REDIS_URL (the same Redis the server uses) and the same GUARDAI_SESSION_SECRET as the server: revocation digests are keyed to that secret, and the shared store binds to it on first use and refuses to start a process presenting a different secret, so a mismatch fails closed rather than silently splitting the two into separate revocation sets. Advertising HA with GUARDAI_REPLAY_BACKEND=redis or GUARDAI_HIGH_AVAILABILITY=true while leaving revocation on the per-replica memory default fails the proxy startup closed.

Rate Limiting

Off by default. Enable a per-client token-bucket limiter with GUARDAI_PROXY_RATE_LIMIT_ENABLED=true (or --rate-limit-enabled; the env var wins over the flag). Tune GUARDAI_PROXY_RATE_LIMIT_RPS (default 100) and GUARDAI_PROXY_RATE_LIMIT_BURST (default 200). Enabling it with an rps or burst below 1, or a non-numeric value, aborts startup. When the limit is exceeded the proxy returns 429 with Retry-After: 1 in the caller's provider error shape.

Requests are bucketed by the authenticated tenant when auth is enabled. Without a tenant the bucket key is the client IP. By default (GUARDAI_PROXY_RATE_LIMIT_TRUSTED_PROXY_DEPTH=0) that IP is the TCP peer address, which a client cannot forge. If the proxy runs behind one or more reverse proxies that set X-Forwarded-For, set the depth to the number of trusted hops so the real client IP is read from that header instead; do not leave depth at 0 behind an L7 proxy, or every client will share the proxy's bucket.

How Token Restoration Works

When the LLM generates output containing builtin-detected tokens like {{email:e_001:1bcaef1a4aff}}, the proxy automatically restores them using the session state that was created during the request transformation. The restore mode (full, partial, masked, formatted, abstract) is controlled by the policy configuration (none removes the value entirely).

For example, with builtin detection:

• Input: "Contact julia@example.com or IP 10.0.0.1 for help"
• To provider: "Contact {{email:e_001:1bcaef1a4aff}} or IP {{ip:ip_001:cd9903b926e3}} for help"
• From provider: "I'll reach out to {{email:e_001:1bcaef1a4aff}}..."
• To your app: "I'll reach out to julia@example.com..."

In the default builtin mode the proxy covers structured patterns only (emails, phones, IPs, IBANs, SSNs, etc.). To also mask person/company/location names, run the proxy with the NER sidecar (GUARDAI_DETECTOR_MODE=both or advanced, see Configuration), or use the full server API.

Architecture

Your App
  |
  | (standard OpenAI SDK calls)
  v
OGuardAI Proxy (port 8081)
  |
  | 1. Transform: mask PII in request
  | 2. Forward: send safe request to OpenAI
  | 3. Rehydrate: restore tokens in response
  |
  v
OpenAI API (never sees real PII)

Limitations

• The proxy adds a small latency overhead for PII detection and token restoration
• All message roles (including system) are scanned for PII
• The proxy requires network access to both your app and the target API
• Session state is per-request; multi-turn conversations need session continuity (the proxy handles this automatically via sealed session blobs)

Troubleshooting

Tokens not being restored: Check that the LLM is faithfully reproducing the token format {{type:id:cap}}. OGuardAI includes a 3-stage token repair pipeline (strict, repair, fuzzy) that handles common LLM token mangling.

Performance: In builtin (regex) mode the proxy adds only a small detection and restoration overhead on short inputs; enabling the NER sidecar (both or advanced) adds more. Measure it against your own inputs and hardware rather than assuming a fixed figure. For large inputs, consider using the chunking API directly.