Plugin and Extension Development

OGuardAI has no in-process plugin loader, no entry-point registry, and no hot-loaded native plugin SDK. Nothing is linked into the runtime from a directory at request time, so a dropped-in binary can never widen the trust boundary. "Extending" OGuardAI instead happens through explicit, reviewable seams at several altitudes, and every one of them is additive and fail-closed: a seam can only ADD or TIGHTEN detection, never weaken the built-in floor.

Most of what people call a "plugin" for a data-protection runtime (a new entity, a new vertical, a new audience, a new language) is the top rows: policy, a request field, or config, no compile. This guide maps every seam, goes deep on the one that genuinely needs code (the NER backend interface), and closes with how to build and test a change.

For the day-to-day "add my own entity" path see Extending Entities. For the full catalog of config-only surfaces see Extensibility. For dev setup, branch strategy, and PR rules see Contributing. This page is the contributor-facing map of where each extension lives.

Workspace layout

Know which plane you are touching before you start. The repo is a Cargo workspace (Rust runtime), a uv-managed Python tree (the NER detector and Python SDK), and a pnpm workspace (TypeScript SDK, MCP tools).

crates/                 # Runtime Plane (Rust)
  core/                 # EntityType, errors, kernel types
  detector-builtins/    # built-in regex detectors (patterns.rs)
  detector-client/      # HTTP bridge to the Python NER service + MergedDetector
  policy/               # policy engine (custom_patterns, rules, channels)
  rehydrate/            # restore modes, restore_floor.yaml, restore strategies
  tokenizer/ transformer/ session/ auth/ ...
python/
  detector-core/        # the NER detector: backends, patterns, enrichment, languages
apps/
  server/  cli/  proxy/           # Rust binaries (oguardai-server, oguardai-cli)
  detector-py/          # FastAPI wrapper around detector-core (the NER sidecar)
  mcp-server/  platform-web/
packages/
  mcp-tools/  shared-types/
sdks/
  typescript/  python/  go/  java/   # the four client SDKs
policies/               # shipped policy YAML per vertical

The Rust server never runs NER itself. It calls the Python detector over HTTP through crates/detector-client (DetectorClient, MergedDetector), reaching it at GUARDAI_DETECTOR_URL (for example http://detector:9090). The detector handles raw PII and is meant to sit on an internal network only. Keep that trust boundary in mind whenever you extend detection: a new backend runs inside the sidecar, never in the caller.

Seam 1: custom entity type (policy, no code)

A deployment declares its own domain identifiers (an MRN, a matter id, a bank account) under custom_patterns in policy YAML. Each match becomes {{custom:<type>:id}} and is governed by a normal rule. Custom patterns are additive: they can only add detections, never disable or weaken a built-in.

custom_patterns:
  - entity_type: "employee_id"
    pattern: '\bEMP-\d{6}\b'
    confidence: 0.9
    context_words: ["employee", "staff"]
rules:
  - entity_type: "employee_id"
    action: "tokenize"
    restore_mode: "masked"
    protection_level: 1
    conditions: []

That is the entire change, no fork. The full treatment (partial value_group, restore_strategies, NER-label types, the compatibility matrix) lives in Extending Entities. Contribute Rust only to promote a detector into the shipped built-in floor, and that is the advanced section of that same guide (crates/detector-builtins/src/patterns.rs, crates/core/src/types.rs, crates/rehydrate/src/restore.rs).

Seam: a custom pattern for a single request (no restart)

When a pattern is caller-specific or one-off, a request can carry its own custom_patterns without touching any policy or restarting the server. The patterns apply to that request only, are validated on use, and are never persisted:

POST /v1/transform
{
  "input": "Ticket TKT-4821 for julia@firma.example",
  "policy": "default",
  "custom_patterns": [
    { "entity_type": "ticket_id", "pattern": "\\bTKT-\\d{4,6}\\b", "confidence": 0.9 }
  ]
}

The same validation the policy loader runs applies here: a pattern that matches everything is rejected, a type may not shadow a built-in named type, and the match merges additively (it fills gaps the built-in and NER detectors left, never overrides them). Two validate-only endpoints let a caller check a type name or a pattern before sending traffic: POST /v1/entity-types and POST /v1/patterns. Nothing is stored: this is a per-request extension, not registration.

Seam: an external HTTP detector (config)

A deployment can register an external detector over HTTP under detector.webhook in oguardai.yaml. Per request the runtime POSTs {text, language} to the endpoint and merges the returned spans additively, so a domain-specific detector (a model or service you own) augments the built-in and NER detectors without a fork:

detector:
  mode: both
  webhook:
    url: https://detector.internal/scan
    entity_types: [employee_id, case_ref]   # only these types are accepted
    api_key: ${DETECTOR_KEY}                 # sent as X-Detector-API-Key
    timeout_secs: 5
    required: false                          # true => a failure rejects the request

The plugin can only ADD coverage: any span it returns that overlaps a built-in or NER span is dropped, a declared type may not shadow a built-in named type, and an offset that disagrees with a returned value is rejected. Failure handling is explicit: with required: true a timeout, HTTP error, or malformed response fails the request closed (the detector's types are mandatory); with required: false it degrades with a warning and the built-in floor still runs. Invalid webhook config aborts startup rather than serving a half-configured detector.

Seam: push-webhook notifications (config)

Set notifications.webhook_url to receive the runtime's audit events at an external endpoint (a SIEM, an observability pipeline):

notifications:
  webhook_url: https://sink.internal/guardai-events
  api_key: ${SINK_KEY}                       # sent as X-Notification-Key
  events: [transform, policy_denied, session_expired]   # empty => all events

Delivery is fire-and-forget and fail-open: it runs on a spawned task, so a slow or down endpoint never blocks or fails a request, and the primary audit sink is unaffected. The payload is the same PII-free audit event the runtime already records: entity types and counts, never values, with the session id, trace id, and tenant id carried only as one-way fingerprints. It reuses the audit pipeline, so no request path emits an event twice.

Seam 2: language pack overlay (data, no source change)

OGuardAI ships enrichment lexicons for roughly 30 languages as an immutable data floor under python/detector-core/guardai_detector/languages/data/<code>.json. There are no per-language code modules: every language is data, driven by the generic DataEnricher. An operator adds, extends, or replaces a language at deploy time by pointing GUARDAI_LANGUAGE_PACKS at a JSON file that maps a language code to a lexicon. It loads at startup and a malformed pack fails loudly, it never silently no-ops.

{
  "_comment": "operator language packs",

  "yue": { "female_titles": ["女士"], "male_titles": ["先生"] },

  "de": { "male_names": ["acmebob"], "informal_markers": ["servus "] },

  "fr": { "_mode": "replace", "female_titles": ["Mme "], "male_titles": ["M. "] }
}

The optional _mode on each entry selects the operation:

• No built-in for the code: the overlay is registered as a brand new language, as-is.
• _mode absent or "extend" (the default): the overlay only widens the built-in. The built-in's gender and honorific results stay authoritative (the overlay fills gaps only, so a confident built-in classification is never flipped), new formality markers join the built-in's vote, and structural scalars (rtl, name_order, default_formality) keep the built-in value.
• _mode: "replace": the built-in for that code is dropped and only the overlay applies. To truly override a built-in, start from its checked-in <code>.json, edit it, and load it with replace.

The lexicon schema (female_titles, male_titles, formal_markers, informal_markers, female_names, male_names, patronymic_suffixes, honorific_suffixes, rtl, name_order, default_formality) is documented in python/detector-core/guardai_detector/languages/data/README.md and validated by validate_lexicon at load. Programmatic equivalents exist for tests and embedding: LanguageRegistry.register_overlay(code, lexicon) for one entry and LanguageRegistry.load_overlays_from_path(path) for a file.

Two adjacent data seams round this out, both no-code:

• spaCy model mapping. GUARDAI_SPACY_MODELS (JSON {language: model_name}) points a language at a different installed spaCy model or adds one, merged over the built-in map. Only relevant when the active backend is spacy.
• Restore output. restore_templates in oguardai.yaml (or GUARDAI_RESTORE_TEMPLATES JSON for the proxy) localizes honorifics and abstract labels over the crates/rehydrate/data/restore_floor.yaml defaults. See Extensibility for both.

A language pack changes detection enrichment (gender, formality, honorifics), not which raw values are found. Detection vocabulary is a separate seam: detector.vocab in oguardai.yaml per deployment and custom_patterns per policy.

Seam 3: the NER backend interface (code)

This is the one true "plugin" seam, and it requires source. The detection strategy is a fixed, known set. A backend is selected by name, not discovered, so a new engine (a proprietary model, a hosted NER API, a different local model) is a code contribution to python/detector-core, not a config knob.

The interface

Every backend implements the NERBackend abstract base class in python/detector-core/guardai_detector/backends/base.py:

class NERBackend(ABC):
    @abstractmethod
    def detect(
        self, text: str, language: str = "en", extra_labels: list[str] | None = None
    ) -> list[DetectedEntity]: ...

    @abstractmethod
    def is_available(self) -> bool: ...

    @abstractmethod
    def name(self) -> str: ...

    def available_languages(self) -> list[str]:
        return []          # override if you have per-language models (spaCy does)

    def supports_zero_shot(self) -> bool:
        return False       # override to True if you honor per-request extra_labels

Contract notes that matter:

• detect returns DetectedEntity objects with entity_type, text, start, end, confidence, and metadata. Entity types must be OGuardAI types (person, company, location, address); map your engine's labels yourself, the way SpaCyBackend maps PERSON/ORG/GPE. The pattern-based detector runs separately and its results are merged, so a backend returns NER entities only.
• extra_labels are per-request zero-shot labels a policy asks for via detection.ner_labels. If your engine supports zero-shot, honor them and set supports_zero_shot() to True; report a per-request label back under its own name (the Rust client remaps it per policy). If not, ignore them and leave the default False, which makes the runtime fail closed on a required label-derived type rather than under-detect.
• available_languages() defaults to empty on purpose: a language-general engine like GLiNER declares no per-language models, and the detector derives coverage from the enrichment registry instead. Only override it if you load language-specific models.
• is_available() must reflect real readiness (model loaded, dependency present). Load failures are expected and handled by the selector below, so return False rather than raising in the constructor.

The three shipped backends

none is the regex-only mode: NER returns nothing and only the pattern detector runs. It is the honest floor, not a stub to fill in.

How a backend is selected

DetectorConfig.ner_backend comes from the NER_BACKEND env var (default gliner). PIIDetector._init_backend in python/detector-core/guardai_detector/detector.py accepts either a built-in name validated against _KNOWN_BACKENDS = {"gliner", "spacy", "none"}, or a module.path:ClassName spec that it imports and constructs (load_custom_backend). If a built-in is not available, it degrades down the chain gliner -> spacy -> none; a custom backend that fails aborts under strict_backend (env NER_BACKEND_STRICT=true) or else degrades to the regex-only floor, never silently to a different engine. The default production compose sets NER_BACKEND_STRICT: "true" so the requested backend is mandatory.

Adding a new backend

No fork (a deployment's own engine). Put your NERBackend subclass on the detector service's import path and set NER_BACKEND=my_pkg.module:MyBackend (or route a single language to it via GUARDAI_NER_LANG_BACKENDS). The class must construct with no required arguments, reading its model, endpoint, or API key from the environment, and return False from is_available() until it is ready. The detector imports and constructs it (load_custom_backend); a load failure aborts under NER_BACKEND_STRICT or else degrades to the regex-only floor. This is the plug-and-play path for a proprietary model, a hosted NER, or a different GLiNER wrapper, no source change to detector-core.

In tree (shipping it with detector-core). To make a backend a first-class built-in name, wiring is explicit and small:

1. Add backends/mybackend.py implementing NERBackend. Map your engine's labels to OGuardAI entity types. Do not raise in __init__; set an internal available flag and return it from is_available().
2. Export it from backends/__init__.py alongside the others.
3. Register the name in PIIDetector._KNOWN_BACKENDS and add a construction branch in _init_backend (mirror the gliner/spacy branches, including the strict_backend abort and the logger.info("ner_backend_loaded", ...) line).
4. If your backend needs config (a model name, an endpoint, an API key), add fields to DetectorConfig in guardai_detector/models.py and read the env in the service lifespan (apps/detector-py/guardai_detector_service/main.py), validating eagerly so a bad value fails at startup.
5. Add a backend test under python/detector-core/tests/ (see test_detector.py), covering is_available(), the label mapping, and supports_zero_shot() behavior.

The rest of the pipeline (dedup, enrichment, the merge with regex, the Rust-side MergedDetector) is untouched: it consumes DetectedEntity and does not care which backend produced it.

You usually do not need a new backend

Before writing one, confirm the dynamic seams do not already cover you. A new vertical NER label (a diagnosis, a matter subject) is zero-shot on the existing GLiNER backend via a policy's detection.ner_labels, no code. A different local spaCy model for a language is GUARDAI_SPACY_MODELS, no code. A new backend is justified only when you are swapping the detection engine itself, for example a proprietary or hosted model.

Building and testing a change

Match the plane you touched. All three ecosystems have their own gate; run the one you changed, and the E2E suites when behavior crosses the HTTP boundary.

Rust runtime

cargo build --workspace
cargo test --workspace
cargo fmt --check
cargo clippy --workspace --all-targets -- -D warnings

make test, make lint, and make build wrap these. Per-crate: cargo test -p guardai-policy, cargo test -p guardai-rehydrate, and so on.

Python detector

cd python/detector-core && uv run pytest
cd sdks/python && uv run pytest

make python-test runs both. The detector service wrapper has its own tests under apps/detector-py/tests (test_auth.py, test_config.py). Lint and format with ruff check python/ and ruff format --check python/. For a backend or enrichment change, the fast loop is uv run pytest tests/test_detector.py and tests/test_enrichment.py in python/detector-core.

End-to-end suites (live server, real HTTP, no mocks)

The suites in tests/ hit a running server on http://localhost:3000. They use only the Python standard library. Start the server first, then run them.

# Start the full stack (server + detector + revocation store)
docker compose -f deploy/docker/docker-compose.yml up -d

# Or a local build
cargo run --release -p oguardai-server -- --config oguardai.yaml

# Point the suites at the server and run in order
export GUARDAI_BASE_URL=http://localhost:3000
export GUARDAI_API_KEY=...            # a key the server accepts

rm -f revocations.dat                 # revocation tests persist state; start clean
python3 tests/e2e.py                  # every API endpoint
python3 tests/enterprise.py           # business workflows (support, RAG, healthcare)
python3 tests/features.py             # feature verification against the corpora
python3 tests/languages.py            # language coverage and packs
python3 tests/revocation.py           # run LAST, it writes persistent state

Run revocation last (or restart the server between runs): it creates persistent state that affects round-trip tests. Other suites cover specific seams: tests/customization.py and tests/verticals.py exercise the config-only extension surfaces, tests/ha.py needs the HA compose stack (deploy/docker/docker-compose.ha.yml), and tests/proxy.py auto-skips unless an OPENAI_API_KEY or ANTHROPIC_API_KEY is set.

For E2E against the production auth mode, layer the E2E override, which grants the bootstrap key admin scope and mounts policies/ read-only so policy edits apply without a rebuild:

docker compose \
  -f deploy/docker/docker-compose.yml \
  -f deploy/docker/docker-compose.e2e.yml up -d

Never use that override in production. See tests/README.md for the full suite matrix and corpus reference.

What does not exist (by design)

Do not document or expect these; they are deliberately absent so a config file cannot weaken the trust model:

• No in-process plugin discovery, no entry-point registry, no native hot-loading. The in-process NER backends are a named, validated set (gliner, spacy, none); a fourth is a source change wired in explicitly. An external detector is supported over HTTP (detector.webhook, above): that is the config-declared, fail-closed, additive-only way to add one without a fork or a linked binary.
• No config-defined policy actions. redact, tokenize, and abstract are a closed, security-reviewed set. A novel action (route, quarantine, notify) is code, not a knob. A raw pass-through is expressed via the policy whitelist, not a new action.
• No config-defined restore modes. full, formatted, partial, masked, abstract, and none are closed. restore_strategies customize the partial reveal without code, but a brand new mode is code.
• No config-defined token format. {{type:id:cap}} and {{custom:type:id:1ea13a62b528}} are fixed.

Everything domain-specific (entity types, entity roles, caller roles, output channels, egress destinations, the RAG classification lattice, NER labels, languages, the partial-restore algorithm) is yours to define in policy and data. The fixed core is exactly the part a deployment should not be able to loosen from outside the runtime. See the "What requires code" section of Extensibility for the reasoning behind each line.