Latency, throughput, and session blob size characteristics by detector mode and payload size

Transform Latency by Detector Mode

The figures below are indicative, from local profiling, not guarantees. Actual latency depends on hardware, the NER model, and text length. For service-level targets see SLOs; for the measurement approach see Benchmarks.

Mode	p50	p99	Notes
`builtin` (regex only)	~1-2ms	~5ms	Fastest. No external dependencies.
`both` (builtin + NER healthy)	~80ms	up to several hundred ms	NER adds network round-trip + model inference. Grows with text length.
`both` (NER configured, unavailable)	timeout	timeout	Waits for the NER timeout (`detector.timeout_secs`), then falls back to builtin.
`advanced` (NER only)	~80ms	up to several hundred ms	All detection via Python sidecar.

The two NER rows reflect the same GLiNER inference path. At roughly 780 characters, NER detection measured around 80ms p50 and into the several-hundred-ms range at p99 on a single CPU core. These are orders of magnitude above the builtin regex path; size NER capacity accordingly.

Rehydrate Latency

Size	p50	p95	p99
Small (under 100 chars)	Under 0.1ms	Under 0.5ms	Under 1ms
Medium (1-5KB)	Under 0.5ms	Under 1ms	Under 2ms
Large (10-100KB)	Under 2ms	Under 5ms	Under 10ms

Rehydrate is typically fast: it's a local string replacement operation with no external calls.

Session Seal/Unseal

Operation	Latency
Seal (AES-256-GCM encrypt)	Under 0.1ms
Unseal (decrypt + validate)	Under 0.1ms

Session Blob Size

Entities	Blob Size	Per-Entity
1	~400 bytes	~400 bytes
10	~2 KB	~200 bytes
50	~10 KB	~200 bytes
100	~20 KB	~200 bytes
500	~100 KB	~200 bytes
1000	~200 KB	~200 bytes

Blob size grows linearly, approximately 200 bytes per entity.

Throughput

Mode	Ops/sec (single core)
Builtin only	>1000
Builtin + NER	20-50 (limited by NER round-trip)

Recommendations

For lowest latency

detector:
  mode: builtin

Detects the named regex types (email, phone, SSN, IBAN, credit card, IP, URL, order, customer ID, passport, health ID, address, DOB) plus the built-in custom patterns (monetary amount, German tax ID, German social-security number, quote ID, case reference). Does NOT detect: person names, company names, locations (these require the NER sidecar).

For best coverage

detector:
  mode: both
  advanced_url: http://localhost:9090

Detects all entity types including person, company, location via GLiNER/spaCy. Latency depends on NER sidecar performance.

If NER sidecar is sometimes unavailable

The system automatically falls back to builtin-only detection when the NER sidecar is unreachable (after the configured NER timeout, detector.timeout_secs). The /readyz probe actively checks the sidecar and returns 503 when it is down, and guardai_ner_degraded_total counts fallbacks; /v1/health always reports the detector healthy, so use /readyz for live status.

To avoid the NER-timeout penalty on each request when NER is down:

Monitor /readyz and the guardai_ner_degraded_total metric for detector status
If NER is down, switch to mode: builtin until it recovers
Or accept the NER-timeout penalty per request (subsequent requests in the same connection are fast if using connection pooling)

Performance Characteristics