How OGuardAI processes PDF files with quality tiers, preprocessing tips, and known limitations

OGuardAI processes PDF files through POST /v1/transform/file. Text extraction quality varies significantly depending on PDF type. This document describes what to expect at each tier and how to get the best results.

Quality Tiers

Tier 1: Text-Based PDFs (Highest Quality)

PDFs created by word processors, report generators, or "Save as PDF" workflows contain embedded text. OGuardAI extracts this text directly using pdf-extract, preserving all characters exactly as authored. PII detection is reliable.

Expect: Near-perfect extraction. All emails, phone numbers, SSNs, IBANs, and names are detected accurately.

Tier 2: Scanned PDFs with Clean OCR (Good Quality)

Scanned documents require Tesseract OCR. The official server Docker image already bundles Tesseract with English, German, and Arabic language data, so OCR works without extra setup there. Tesseract is a separate dependency only for the native release binary (install the tesseract-ocr package on that host); the separate proxy image does not include it. Quality depends on scan resolution and image clarity. 300+ DPI with black text on white background yields the best results.

Expect: Good extraction for clean scans. Some character substitutions may occur (e.g., 0 vs O, 1 vs l), which can cause missed PII.

Note: /v1/transform/file does not fall back to OCR. A PDF with no extractable text on any page (a scanned or image-only document) is rejected with 400, naming the image endpoint, rather than returned as empty text, so an unread scan is never mistaken for a document that holds no PII. To process scanned pages, convert them to images first and use /v1/transform/image.

Tier 3: Mixed PDFs (Variable Quality)

Some PDFs contain both text-based and scanned pages. pdf-extract returns the embedded text from the text pages while the scanned pages contribute nothing. Because at least one page yields text, the whole-document textless rejection does not fire, so the request succeeds on the partial extraction. There is currently no automatic per-page OCR fallback.

Expect: Text pages extract well. Scanned pages produce no text, and because the request is not rejected, PII on those pages is silently missed. Split scanned pages out and send them to /v1/transform/image.

Tier 4: Complex Layouts (Limited)

Multi-column layouts, tables, forms, and overlapping text boxes often produce merged or reordered text. Handwriting is not supported.

Expect: Text is extracted but word order may be wrong. Table cells may merge across columns. PII split across layout regions may not be detected.

Preprocessing Recommendations

Setting	Recommendation
Resolution	300 DPI minimum, 600 DPI for small text
Color	Grayscale or black-on-white for OCR
Orientation	Correct rotation before upload
Format	Use text-based PDF when possible (not scanned images)
File size	Server enforces a configurable upload size limit
Multi-column	Convert to single-column text before upload if feasible

Extraction Sandbox

PDF text extraction does not run in the request thread. The runtime re-execs its own binary in a short-lived subprocess that caps its address space and runs under a wall-clock timeout, so a decompression bomb aborts the child rather than the server. Two limits keys bound it, and both are tighten-only: config may lower them, never raise them past the built-in ceiling.

Key	Default and ceiling	Env override
`limits.pdf_extract_memory_mib`	256	`GUARDAI_PDF_EXTRACT_MEMORY_MIB`
`limits.pdf_extract_timeout_secs`	10	`GUARDAI_PDF_EXTRACT_TIMEOUT_SECS`

A large or slow but otherwise valid PDF can hit one of these ceilings. That is a 413, not a 400, because the document is not what failed. See the classification below.

API Usage

Transform a PDF file:

curl -F "file=@document.pdf" \
     -H "X-API-Key: $GUARDAI_API_KEY" \
     http://localhost:3000/v1/transform/file

With optional policy and language:

curl -F "file=@document.pdf" \
     -F "policy=strict-pii" \
     -F "language=de" \
     -H "X-API-Key: $GUARDAI_API_KEY" \
     http://localhost:3000/v1/transform/file

Response Fields

The file transform response includes document metadata alongside the standard transform output:

Field	Description
`safe_text`	Transformed text with PII replaced by tokens
`entities`	Array of detected entities with types and spans. `span.start` and `span.end` are UTF-8 byte offsets into the extracted text, not character or UTF-16 indices
`session_id`	Session identifier for subsequent rehydration
`session_state`	Encrypted session blob for rehydration
`original_format`	Detected format (e.g., `Pdf`, `Docx`, `Txt`)
`original_size`	Original file size in bytes
`stats`	Processing statistics

Note on quality indicators: The response does not currently include an extraction quality score or confidence metric for PDF text extraction. The original_format field confirms the file was recognized as a PDF, and original_size can help identify suspiciously small files (which may be image-only scans). For image-based processing via /v1/transform/image, the response exposes bounding boxes (geometry only) plus per-entity detection confidence in entities[]. OCR per-word confidence is computed internally but is not returned in the response.

Failure Classification

A document that does not process is not automatically a bad document. Ingestion failures are classified into three kinds, and the HTTP status tells you which one you have. Do not treat every failure as a format problem with the upload.

Status	Code	Meaning	What to do
400	`GUARDAI_BAD_REQUEST`	The format is unsupported, or the document is malformed. The caller's input is at fault.	Fix or convert the document. Retrying the same bytes will fail the same way.
413	`GUARDAI_PAYLOAD_TOO_LARGE`	A configured safety ceiling refused it: the upload size limit, the extraction wall clock, the sandbox address-space cap, or extracted text over the budget. The document may be perfectly valid.	Raise the relevant limit if the workload justifies it, or split the document.
503	`GUARDAI_DOCUMENT_EXTRACTION_UNAVAILABLE`	The runtime could not perform the extraction at all (the sandbox could not be started, its pipes were unavailable). The document is not implicated.	Retriable. Check the server log for the recorded cause; the fixed 503 body deliberately carries no detail.

The 503 body is a fixed string. The underlying error can carry filesystem paths and document fragments, so it is written to the server log and never to the response.

Known Limitations

No automatic OCR fallback for scanned PDFs. The /v1/transform/file endpoint uses pdf-extract for text extraction only. Scanned pages with no embedded text produce empty results. Use /v1/transform/image for scans.
Handwriting is not supported. Tesseract OCR does not reliably read handwritten text.
Multi-column text may merge. Column boundaries are not detected; text from adjacent columns may interleave.
Password-protected PDFs are rejected. Encrypted PDFs fail as a parse error (400), because they are not extractable as authored.
No per-page quality reporting. There is no way to know which pages extracted cleanly and which did not.
PDF forms (AcroForms) field values may not be extracted depending on how the form was authored.

Future Improvements

Planned work includes automatic OCR fallback for pages with no extractable text (hybrid PDF processing) and per-page extraction quality scores in the response.

PDF Support