Performance Benchmarks

No published benchmark numbers yet

There is no reproducible, end-to-end OGuardAI benchmark artifact yet. This page does not quote throughput, latency percentiles, scaling factors, or memory figures, because none have been produced by a real, named run against a dataset we can point to.

The repository ships two benchmark harnesses under benchmarks/ that compute metrics only from a run you execute. They carry no result numbers on purpose, and neither harness prints a metric it did not measure. See benchmarks/README.md. When we publish figures, they will come from one of those runs, with the hardware, payload corpus, and method stated alongside them, not from estimates.

Until then, use the qualitative expectations below for architecture-level reasoning only, not for capacity planning against a specific target.

What shapes performance

OGuardAI has two detection paths, and which one you configure is the dominant factor.

Builtin-only path

The builtin detectors are pure Rust regex and string work with no external calls. The detection stage itself stays inside the process: detect, tokenize, transform, and (on the way back) rehydrate are all in-process compute. Cost scales with payload length and entity count, because the regex scan covers more text and more matches are tokenized. The builtin detector stage is bounded by CPU and the size of the input rather than by any external service.

This "no network I/O" property is about the detection stage, not the whole request. A full request can still cross the process boundary if you configure it to: a detector webhook, Redis-backed sessions, replay protection, revocation, or outbound telemetry each add network I/O regardless of the detection path. The in-process bound holds for a sealed-local deployment (sealed sessions, in-memory revocation, no configured webhook or telemetry).

You can measure this path today with the criterion benches that ship in the workspace:

cargo bench -p guardai-detector-builtins

That gives you real, hardware-specific numbers for the builtin detection stage on your own machine.

Builtin + NER path

When NER is enabled, each detection additionally makes a synchronous round-trip to the Python detector sidecar (GLiNER or spaCy) and waits for model inference. That round-trip plus inference dominates the request: it is a network hop to another process and a model forward pass, both far more expensive than in-process regex. Inference cost grows with text length. Expect this mode to be materially slower per request than builtin-only, and to depend heavily on the NER model, batch behavior, and the sidecar's own hardware (CPU vs GPU). We do not publish a factor here because it has not been measured end to end.

Rehydrate

Rehydrate performs local token-to-value replacement using the session state that travels with the request, and this replacement work is unaffected by whether NER is enabled. With sealed sessions, in-memory revocation, and output NER scanning disabled the rehydrate call makes no external calls. Enabling detector.scan_output_with_ner adds a round-trip to the NER sidecar on the output guard even under those settings. Shared backends also cross the boundary: a Redis-backed session loads over the network, and Redis-backed revocation checks each carried value (and cascades a revoked person to its within-session belongs_to tokens) over the network too. We do not rank rehydrate against the other stages because that comparison has not been measured end to end.

Degradation and fail-closed behavior

If NER is configured but the sidecar times out or is unreachable, an optional NER setup degrades: the request falls back to builtin-only detection and continues. The per-request wait before that fallback is bounded by the configured detector.timeout_secs.

This fallback is not unconditional. A policy that marks an entity type with detection.required_for fails the request closed instead of falling back, so an NER-only-protected type is never allowed to pass raw when NER did not run. Additionally, a sidecar response the runtime cannot trust (a caller-rejected request, or an over-cap or inconsistent span) fails closed even when NER is optional, rather than silently degrading. The fallback path applies only to a genuine transport outage or timeout on an optional NER configuration. See crates/runtime/src/detection.rs and crates/detector-client/src/merged.rs for the exact decision.

Horizontal scaling (expectation, not measurement)

The core data path is designed to scale horizontally: each server instance is stateless because session state travels with the request as a sealed, encrypted blob, so instances do not coordinate on the transform path. Adding instances is expected to add capacity for that path.

This has not been measured across multiple instances, and several factors bound real scaling:

• NER throughput scales with the number of detector sidecar instances, not with the API instances, so a NER-heavy workload needs proportional sidecar capacity.
• Optional features that use a shared backend, such as Redis-backed sessions and replay protection, introduce a coordination point that a purely sealed-session deployment does not have. Revocation is the exception: a multi-replica deployment (sealed or not) must run revocation_backend=redis so a value revoked on one instance is refused by all, a required coordination point rather than an optional one.
• Audit and other I/O add per-request cost that a load test would capture.

We will publish multi-instance numbers only from a real load run, not by extrapolating a single-instance figure.

Memory (structural notes, not profiler figures)

The Rust runtime has no garbage collector, so it does not incur GC pause behavior. We do not publish process RSS, per-request overhead, or peak-under-load figures here, because no memory profiler run supports them.

What the code and its tests do establish is the shape of session state: the sealed session blob grows with the number of entities carried, roughly linearly, and the session layer enforces upper bounds on session size. That is a size relationship, not a process-memory measurement. For an actual memory footprint on your workload, profile a running server with your allocator, concurrency, and payload mix.

Measuring it yourself

1. Builtin detection stage: cargo bench -p guardai-detector-builtins for hardware-specific criterion results.
2. Model utility: benchmarks/model_utility/ answers whether routing a prompt through OGuardAI changes answer quality versus the raw prompt. By default it makes only the guarded LLM call and scores it against each row's reference field with a scorer you provide. The raw call, which sends the unprotected prompt straight to the model, is off by default: it runs whenever you pass --unsafe-raw-baseline. Adding --baseline raw (which requires that flag) then scores the guarded output against the raw output instead of each row's reference; without the flag a raw baseline is refused so real PII is never sent unprotected. The two shipped scorers are placeholders labeled as such in the code; swap in a real metric before trusting the output.
3. NER quality: benchmarks/ner_bakeoff/ sends your labeled, multilingual data to /v1/detect and reports precision, recall, and F1 per language and per entity type, under a span match rule you pick (--match exact or --match overlap).

Both harnesses need a running server (cargo run -p oguardai-cli -- run). They differ on datasets. model_utility defaults to a bundled synthetic fixture set (fictional PII only), so a run with no --dataset still executes against that fixture rather than exiting. ner_bakeoff has no default corpus: with no --dataset it exits without scoring, because it needs your labeled gold spans to compute precision, recall, and F1.

Planned end-to-end harness

A reproducible end-to-end harness that drives the full transform path, including the NER sidecar, is planned. When it lands, the intended setup is:

• Tool: a constant-throughput load generator with scripted POST payloads.
• Payloads: synthetic text with realistic PII density (emails, phones, SSNs, IBANs).
• Warm-up: a warm-up window discarded before measurement.
• NER sidecar: a single detector instance on a stated host.
• Repetition: each run repeated, with the method and hardware reported next to the numbers.

Any figure this page eventually carries will come from that harness on a named run, so it can be reproduced and challenged.