Performance Characteristics

This page carries no latency, throughput, or blob-size numbers. Benchmarks sets the rule and holds it: a figure is published only with the named run, hardware, and corpus behind it, and no such run exists yet. What is below is the shape of the cost, read off the code, plus what to run to get numbers for your own deployment.

What dominates a request

The detector mode you configure is the dominant factor.

A network hop plus model inference is a different order of cost from an in-process regex scan, and inference grows with text length. Size NER capacity separately from API capacity: NER throughput scales with sidecar instances, not with server instances.

The "no external call" property in builtin describes the detection stage, not the whole request. A detector webhook, Redis-backed sessions, replay protection, revocation, or telemetry export each add network I/O regardless of detector mode.

Rehydrate

Rehydrate is local token-to-value replacement against the session state that travels with the request. It runs no detection. With sealed sessions, in-memory revocation, and output NER scanning off, it makes no external call.

Two settings put it back on the network:

• detector.scan_output_with_ner adds a sidecar round trip on the output guard. When the key is unset it follows detector.mode, so both and advanced scan output as well as input. Set it to false explicitly to opt out; that is the documented latency trade.
• Redis-backed sessions load over the network, and Redis-backed revocation checks each carried value (a revoked person additionally suppresses a within-session token only when its link is explicitly marked exclusive ownership; no runtime path currently marks one) over the network too.

Session state size

The sealed session blob grows with the number of entities it carries, roughly linearly, and the session layer enforces an upper bound on session size. That is a size relationship, not a measured byte figure. Measure the blob your own payload mix produces before sizing a client-side store around it.

Measuring it on your hardware

The criterion bench guardai-detector-builtins gives us the builtin detection stage number at each release. For your own deployment, measure at the HTTP level: drive /v1/transform or /v1/detect against a running server (the container image or the release binary). For detection quality and end-to-end answer quality, see the two harnesses in Benchmarks; both compute metrics only from a run you execute.

Recommendations

For lowest latency

detector:
  mode: builtin

Detects the named regex types (email, phone, SSN, IBAN, credit card, IP, URL, order, customer ID, passport, health ID, address, DOB) plus the built-in custom patterns (monetary amount, German tax ID, German social-security number, quote ID, case reference). Does NOT detect: person names, company names, locations (these require the NER sidecar).

For best coverage

detector:
  mode: both
  advanced_url: http://localhost:9090

Detects all entity types including person, company, and location via GLiNER or spaCy. Latency depends on the NER sidecar.

If the NER sidecar is sometimes unavailable

In both mode the runtime falls back to builtin-only detection when the sidecar is unreachable, after waiting up to detector.timeout_secs. In advanced mode there is no fallback: a sidecar outage fails the request closed with GUARDAI_DETECTION_UNAVAILABLE (503, retriable). A policy that names an entity type in detection.required_for fails closed in either mode rather than under-detecting.

To avoid paying the NER timeout on every request while the sidecar is down:

1. Watch /readyz, which probes the sidecar and returns 503 when it cannot answer. /v1/health always reports the detector healthy by design, so it is not the live signal.
2. Track guardai_ner_degraded_total. It records a detection outcome, not a request outcome; see Reading the NER degradation counter before alerting on it.
3. Switch to mode: builtin until the sidecar recovers, or accept the timeout wait per request.