Languages

OGuardAI is not limited to a fixed set of languages. Structured PII is detected in any language, named-entity detection covers the languages the NER model ships with plus any you add, and restoration is extensible per language. Nothing about a language is hardcoded as a ceiling.

Three layers of language coverage

1. Structured PII (any language). Email, phone, IBAN, credit card, IP, URL, customer and order identifiers, dates, and the built-in custom detectors are matched by format. Format does not depend on the surrounding language, so these are protected in text of any script or language, with no configuration.
2. Named entities (NER model set, extensible). Person, company, and location names are detected by the optional NER sidecar. The GLiNER model ships with a broad multilingual set, and you can extend or override its labels per language with a language pack. Detection quality is strongest for high-resource languages.
3. Restoration (per language, extensible). Gendered salutations, formal and informal register, and grammatical formatting are driven by restore templates. A language without a dedicated template still restores correctly by substitution; it just does not add honorific adaptation until you supply one.

Using a language

Pass the language on the request. Any well-formed base ISO code is accepted and preserved end to end, even one the NER model does not list, so an operator-added pack or restore template keeps working under its own code (a region-subtagged tag such as de-DE falls back to the base language):

curl -X POST http://localhost:3000/v1/transform \
  -H "X-API-Key: your-key-here" -H "Content-Type: application/json" \
  -d '{ "input": "Neem contact op met mevrouw Anouk de Vries, anouk@example.nl", "policy": "default", "language": "nl" }'

If you omit language, the runtime uses the configured detector.default_language (English if unset); it does not auto-detect the language. An unknown but well-formed code (for example a locale the NER model was not trained on) is not coerced to English: structured PII is still protected by format, and the code is carried through so your language pack and restore templates apply.

Honest NER coverage

Named-entity detection depends on the NER model, so it is not uniform across every language:

• Strong: German, English, Dutch, French, Spanish, Italian, Portuguese, Romanian, Ukrainian, Russian, Greek, Finnish, Arabic, and other high-resource languages return person, company, and location spans reliably.
• Entity-type precision varies: for some languages a name is still detected and tokenized (so it is protected) but may be typed as company or location rather than person (observed for Polish, Czech, and Hebrew in some phrasings). The value is masked either way; only role-specific handling that keys off the person type is affected.
• Weaker recall, some low-resource languages: a person name may not be returned as a span at all, so it is not tokenized (observed for Hindi and Thai). Structured PII in those languages is unaffected. This is a model-quality limitation; where a specific name form matters, anchor it with a policy custom_patterns rule, add a language pack, or route the language to a stronger backend (see below).
• Known limitation, space-free scripts: short Chinese and Japanese person names in a space-free sentence are frequently not returned as a discrete span by the current NER model. This is a model limitation, not a policy or code restriction, and structured PII in those languages is unaffected. A honorific-anchored custom_patterns rule can catch strongly delimited cases, but regex is brittle for CJK precisely because there are no word boundaries: a pattern such as [\p{Han}]{2,3}(先生|女士) can over-capture the preceding character (for example matching 系王伟 rather than 王伟 in 联系王伟先生). The robust fix is a CJK-tuned NER backend, for example spaCy ja_core_news or zh_core_web, whose tokenizers segment space-free scripts. Route those languages to spaCy without any code change by setting GUARDAI_NER_LANG_BACKENDS={"zh":"spacy","ja":"spacy"} on the detector service; every other language keeps the default backend, and the routed backend is used only if it can serve the language, so a missing model never silently degrades detection. The spaCy CJK models must be installed in the detector image, and GUARDAI_SPACY_MODELS can point a language at a specific installed model. See the NER backend section of the plugin development guide.

Treat NER as a data-minimization control, not an absolute guarantee: a span the model does not surface cannot be tokenized. When a specific name form matters, anchor it with a policy custom_patterns rule.

Adding or extending a language

You can extend both detection and restoration without touching Rust or Python source.

Add NER labels or lexicon for a language (data)

Point GUARDAI_LANGUAGE_PACKS at a JSON file that overlays the built-in enrichment lexicon and NER labels for a language. Built-ins remain the floor; a pack only adds or tightens. See the language-pack overlay section of the plugin development guide.

Add locale-specific structured PII (config, no code)

Add a custom_patterns or detector.vocab.extra_pattern_specs rule to detect a locale's postal codes, national identifiers, or name forms. Patterns can require a nearby context word so a broad shape does not over-detect. See Address Detection for Any Language and Extending Entities.

Add restoration for a language (config, no code)

Supply restore_templates in the policy to add gendered salutations and register for a language the runtime does not template out of the box. Restoration without a template still works by substitution. See the language packs and restore templates section of the extensibility guide and the language coverage section of policy authoring.

Verifying a language end to end

# Structured PII in any language (works with no configuration):
curl -s -X POST http://localhost:3000/v1/transform -H "X-API-Key: your-key-here" \
  -H "Content-Type: application/json" \
  -d '{ "input": "Kontakt: pieter@voorbeeld.co, IBAN NL91ABNA0417164300", "language": "af" }'

# Named entities (needs detector.mode both or advanced):
curl -s -X POST http://localhost:3000/v1/transform -H "X-API-Key: your-key-here" \
  -H "Content-Type: application/json" \
  -d '{ "input": "Bitte kontaktieren Sie Herrn Klaus Müller in München.", "language": "de" }'

The first request tokenizes the email and IBAN even for a language the NER model does not list. The second additionally tokenizes the person and location once the NER sidecar is enabled.