PageTrace

Understand documents. Trace every answer to its source.

PageTrace — Multimodal Document Intelligence Platform is an early-stage project exploring how document understanding systems can make evidence, provenance, and evaluation first-class concerns.

Vision

PageTrace is guided by four ideas:

• Traceability: outputs should retain precise, inspectable links to source evidence.
• Provenance: transformations should record where information came from and how it changed.
• Evaluation: measurable quality should shape development from the first real capability.
• Untrusted-document security: document content is hostile data, never executable instruction.

Current status

PageTrace v1.0.1 is a published portfolio release. Milestones 0 through 11 are implemented and accepted within their documented scope. Version 1.0 describes the portfolio/API milestone, not a claim that the local service is suitable for remote or multi-user production deployment. The current package can:

• stage untrusted local PDF, PNG, and JPEG files under explicit byte/page/pixel limits;
• identify supported media from content signatures and confirm it with pypdf or Pillow;
• compute full SHA-256 content fingerprints and deterministic document/page identities;
• persist canonical immutable manifests and the exact validated source bytes atomically;
• verify manifest structure, identity, source size, and source fingerprint during readback;
• extract embedded PDF text with pypdf while preserving the returned text;
• reject extracted output above typed per-page and per-document character limits;
• route PDF pages with no usable embedded text and all PNG/JPEG pages as explicit OCR candidates;
• represent mixed digital/scanned-style PDFs with independent page-level states;
• persist deterministic, versioned text artifacts separately from immutable ingestion manifests;
• render only OCR-candidate and policy-eligible sparse pages using PDFium;
• recognize routed page images with bundled RapidOCR PP-OCR models on CPU ONNX Runtime;
• persist immutable OCR artifacts linked to both document and text-routing provenance;
• measure exact match, character error rate, and word error rate without hidden normalization;
• position embedded PDF words and replay-verified OCR lines in explicit top-left coordinates;
• detect line-delimited PDF tables and preserve positioned cells; and
• persist immutable structured artifacts linked to the exact document, text, and OCR artifacts;
• create deterministic page-bounded chunks without dropping oversized spans;
• retain exact source-span, character-offset, page-region, coordinate-unit, and processor lineage; and
• persist immutable corpus artifacts whose chunks can be reconstructed and reverified from the exact structured source;
• rank verified corpus chunks with a transparent, versioned Unicode BM25 baseline; and
• evaluate binary relevance judgments with Precision@k, Recall@k, MRR, MAP, and nDCG;
• produce deterministic extractive answers made only from exact retrieved-text slices; and
• preserve complete retrieval, chunk, page, region, and character-offset evidence or abstain with an explicit reason;
• consolidate text, retrieval, answer, provenance, safety, human-review, and resource metrics;
• durably queue bounded ingestion, QA, and quality workflows with idempotency, retries, cancellation, recovery, lifecycle events, and persistent operational counts;
• expose those workflows through an authenticated loopback JSON API and a separate background worker/operator CLI;
• inspect source identity, page geometry, exact citations, retrieval context, abstention, quality findings, durable activity, and system counters in a packaged evidence-first web interface; and
• apply explicit quality gates and directional baseline-regression rules with CI-friendly status codes; and
• reproduce a two-page evidence demonstration with an exact cited answer, explicit abstention, canonical retrieval/quality evaluations, file digests, and machine-readable limitations.

An OCR-candidate status remains routing information rather than proof that OCR is necessary or accurate. OCR output is untrusted derived data with measured confidence, not verified truth. PageTrace does not yet build persistent or vector indexes or perform abstractive/model-assisted answer generation. It does not use language or vision models. Its Milestone 6 QA baseline only selects exact retrieved excerpts and cannot synthesize across them. Corpus chunks currently cover positioned text spans; tables remain traceable through the linked structure artifact rather than being duplicated as speculative table text. Table extraction is limited to deterministic line-based PDF detection.

Implemented pipeline

untrusted documents
        |
secure deterministic ingestion (Milestone 1 complete)
        |
embedded PDF text + explicit OCR routing (Milestone 2A complete)
        |
real OCR engine and measured OCR evaluation (Milestone 2B complete)
        |
positioned text and line-based PDF tables (Milestone 3 complete)
        |
provenance-aware page-bounded chunks (Milestone 4 complete)
        |
transparent BM25 retrieval and relevance metrics (Milestone 5 complete)
        |
extractive evidence-grounded QA with explicit abstention (Milestone 6 complete)
        |
cross-stage metrics, human rubrics, gates, and regressions (Milestone 7 complete)
        |
durable authenticated product backend (Milestone 8 complete)
        |
evidence-first web product (Milestone 9 complete)
        |
local security and operational hardening (Milestone 10 complete)
        |
reproducible evidence-backed portfolio (Milestone 11 complete)

See ROADMAP.md for milestone scope boundaries and docs/ACCEPTANCE.md for the v1.0 acceptance record.

Milestone 12 is planned around representative real-document evaluation and reliability work. Its scope and acceptance requirements are documented in the Milestone 12 plan.

Published release

Download the verified wheel, source distribution, and SHA256SUMS from the PageTrace v1.0.1 release. Verify the downloaded distribution before installation:

sha256sum --check SHA256SUMS
python -m pip install pagetrace-1.0.1-py3-none-any.whl

On Windows PowerShell, use Get-FileHash -Algorithm SHA256 and compare the result with SHA256SUMS.

Development setup

PageTrace requires Python 3.11 or later.

python -m venv .venv
# Linux/macOS: source .venv/bin/activate
# Windows PowerShell: .venv\Scripts\Activate.ps1
python -m pip install --upgrade pip
python -m pip install -e ".[dev,ocr,structure]"

The base install remains sufficient for ingestion and Milestone 2A extraction. Install the ocr extra only when running Milestone 2B OCR; it adds RapidOCR, CPU ONNX Runtime, and pypdfium2. The structure extra adds pdfplumber for positioned PDF words and line-based table detection. OCR geometry replay also requires the ocr extra when the selected OCR artifact contains routed pages.

Reproducible v1.0 portfolio

Install the output-affecting reference versions and run the complete evidence demonstration:

python -m pip install \
  -c requirements/portfolio-constraints.txt \
  -e ".[dev,ocr,structure]"
pagetrace portfolio-demo portfolio-output

The command creates a deterministic PDF, processes it through the complete local pipeline, and writes canonical evidence, evaluation, quality, digest, and limitation records. It demonstrates one exact source-cited answer and one correct no-evidence abstention. See the v1.0 portfolio guide for reproduction and claim boundaries.

Visual evidence

Document intake — Successful ingestion with verified SHA-256 identity and page metadata.

Evidence-grounded answer — Extractive answer backed by cited source slices.

Explicit abstention — PageTrace refuses to answer when no retrieval evidence passes the configured threshold.

These screenshots were captured from PageTrace v1.0.1 using the deterministic synthetic portfolio fixture and do not demonstrate general real-world accuracy.

Python API

from pathlib import Path

from pagetrace.corpus import build_corpus, load_corpus_artifact
from pagetrace.documents import ingest_document, load_document
from pagetrace.extraction import extract_document_text, load_text_extraction
from pagetrace.ocr import evaluate_ocr, load_ocr_artifact, ocr_document
from pagetrace.qa import QaStatus, answer_question
from pagetrace.quality import AnswerQualityCase, create_quality_suite, evaluate_quality
from pagetrace.retrieval import (
    RetrievalJudgment,
    create_retrieval_dataset,
    evaluate_retrieval,
    retrieve,
)
from pagetrace.structure import load_structured_document, structure_document

manifest = ingest_document(Path("annual-report.pdf"), store=Path(".pagetrace"))
loaded = load_document(manifest.document_id, store=Path(".pagetrace"))
assert loaded == manifest

artifact = extract_document_text(manifest.document_id, store=Path(".pagetrace"))
loaded_artifact = load_text_extraction(
    artifact.artifact_id,
    document_id=manifest.document_id,
    store=Path(".pagetrace"),
)
assert loaded_artifact == artifact

ocr_artifact = ocr_document(
    manifest.document_id,
    artifact.artifact_id,
    store=Path(".pagetrace"),
)
assert (
    load_ocr_artifact(
        ocr_artifact.artifact_id,
        document_id=manifest.document_id,
        store=Path(".pagetrace"),
    )
    == ocr_artifact
)

evaluation = evaluate_ocr("expected text", ocr_artifact.pages[0].text or "")
print(evaluation.character_error_rate, evaluation.word_error_rate)

structured = structure_document(
    manifest.document_id,
    artifact.artifact_id,
    ocr_artifact.artifact_id,
    store=Path(".pagetrace"),
)
assert (
    load_structured_document(
        structured.artifact_id,
        document_id=manifest.document_id,
        store=Path(".pagetrace"),
    )
    == structured
)

corpus = build_corpus(
    manifest.document_id,
    structured.artifact_id,
    store=Path(".pagetrace"),
)
assert (
    load_corpus_artifact(
        corpus.artifact_id,
        document_id=manifest.document_id,
        store=Path(".pagetrace"),
    )
    == corpus
)

results = retrieve(
    manifest.document_id,
    corpus.artifact_id,
    "revenue growth",
    store=Path(".pagetrace"),
)

dataset = create_retrieval_dataset(
    corpus.artifact_id,
    (
        RetrievalJudgment(
            query_id="revenue-growth",
            query="revenue growth",
            relevant_chunk_ids=(corpus.chunks[0].chunk_id,),
        ),
    ),
)
metrics = evaluate_retrieval(corpus, dataset)
print(results.returned_hit_count, metrics.mean_recall)

qa_result = answer_question(
    manifest.document_id,
    corpus.artifact_id,
    "revenue growth",
    store=Path(".pagetrace"),
)
if qa_result.answer is None:
    print("abstained:", qa_result.abstention_reason)
else:
    print(qa_result.answer, qa_result.citations)

quality_suite = create_quality_suite(
    answer_cases=(
        AnswerQualityCase(
            case_id="revenue-growth-answer",
            result=qa_result,
            expected_status=QaStatus.ANSWERED,
        ),
    ),
)
quality_report = evaluate_quality(quality_suite)
print(quality_report.status, quality_report.metrics)

Expected document failures derive from DocumentError. More specific public errors distinguish unsupported or malformed content, resource limits, encrypted PDFs, storage failures, and persistence-integrity failures. IngestionLimits configures the Python API defaults:

Limit	Default
Maximum file size	100 MiB
Maximum PDF pages	2,000
Maximum image pixels	100,000,000

Limits can be changed deliberately for a deployment context:

from pagetrace.documents import IngestionLimits, ingest_document

manifest = ingest_document(
    Path("small.png"),
    store=Path(".pagetrace"),
    limits=IngestionLimits(max_file_bytes=5 * 1024 * 1024),
)

Expected extraction failures derive from ExtractionError; source-document integrity failures remain explicit DocumentError values. ExtractionLimitError distinguishes output-policy rejection from parser, integrity, storage, and OCR-routing outcomes.

TextExtractionConfig captures the pypdf mode, routing threshold, and immutable ExtractionLimits in artifact identity. Defaults bound accepted output to 2,000,000 characters per page and 20,000,000 characters per document. Counts include every character returned by pypdf. Limits are inclusive (count <= limit succeeds); PageTrace raises before accepting the page when a count is greater, never truncates text, and never persists a partial artifact. The 2-million/20-million defaults are intentionally far above ordinary digital page text while placing a conservative pre-alpha ceiling on accepted page and artifact output.

from pagetrace.extraction import ExtractionLimits, TextExtractionConfig

configuration = TextExtractionConfig(
    limits=ExtractionLimits(
        max_characters_per_page=250_000,
        max_characters_per_document=2_000_000,
    )
)
artifact = extract_document_text(
    manifest.document_id,
    store=Path(".pagetrace"),
    configuration=configuration,
)

With the default sufficiency threshold of four non-whitespace characters, zero becomes OCR_CANDIDATE, one to three becomes SPARSE_EMBEDDED_TEXT, and four or more becomes EMBEDDED_TEXT. This deterministic rule is not an OCR-quality classifier. Sparse means embedded text exists below the configured sufficiency threshold; it does not mean the page is satisfactorily resolved, and the page remains eligible for future Milestone 2B OCR policy/evaluation. Sparse and embedded text is stored exactly as returned by pypdf. Whitespace-only output is not claimed as extracted text.

By default, Milestone 2B selects both OCR_CANDIDATE and SPARSE_EMBEDDED_TEXT pages. A candidates-only policy is available. PDFs render at 200 DPI; image documents use their verified native pixels. Defaults allow at most 100 selected pages, 25 million pixels per page, 100 million pixels per document, 1,000 detected lines per page, 2 million OCR characters per page, and 20 million OCR characters per document. Violations fail without truncation or partial persistence.

Milestone 3 emits display-oriented, top-left coordinates: PDF pages use points and image pages use pixels. Embedded PDF text is represented as positioned words. OCR-selected pages are rendered and recognized again with the exact processor and configuration recorded by the source OCR artifact; text, line count, and mean confidence must match before geometry is accepted. Defaults allow at most 100,000 text spans per page, 1,000,000 spans per document, 100 tables per page, 1,000 tables per document, 10,000 cells per table, and 100,000 cells per document. Violations fail without partial persistence.

Milestone 4 emits page-bounded chunks in structured page/span order. The default maximum is 4,000 characters per chunk, with recorded spaces between embedded PDF words and newlines between OCR lines or source types when both fragments fit in one chunk. Long spans are split into exact, gap-free character slices. Every fragment records its source span identity/index, source and chunk offsets, evidence source, and bounding box. Defaults allow at most 10,000 chunks per page, 100,000 chunks per document, 10,000 fragments per chunk, 1,000,000 fragments per document, and 50,000,000 accepted chunk characters per document.

Milestone 5 uses a fixed NFKC/case-folded Unicode word tokenizer and Okapi BM25 with explicit defaults k1=1.2, b=0.75, and top_k=10. Positive-scoring chunks are ordered by score and then stable corpus order. Results preserve the exact corpus identity/content fingerprint plus chunk, page, coordinate, bounding-region, text, matched-term, and processor/tokenizer provenance. Queries are limited to 10,000 characters and 1,000 tokens. Binary relevance evaluation reports Precision@k, Recall@k, reciprocal rank, average precision, and nDCG per case and as deterministic means. Query results and evaluations are returned or serialized explicitly; they are not silently persisted.

Milestone 6 deterministically splits retrieved chunks at line and sentence-terminal boundaries, ranks candidate excerpts by unique query-term coverage and retrieval rank, and returns at most three exact excerpts and 4,000 answer characters by default. Every citation records the retrieval rank, chunk identity, exact character offsets, matched terms, and coverage; the embedded retrieval result carries page, bounding-region, coordinate, corpus, and processor provenance. An answer is valid only when it equals its cited excerpts joined by the configured separator. PageTrace abstains explicitly when retrieval has no hits, no excerpt reaches the default 0.25 coverage threshold, or the answer limit cannot retain a supported excerpt. QA results are serializable but are not silently persisted.

Milestone 7 uses canonical quality suites to combine named text-error fixtures, existing retrieval evaluations, expected QA behavior, relevant citation judgments, structured 1-to-5 human reviews, and an explicit policy. Reports aggregate exact-match/CER/WER, retrieval MRR/Precision/Recall/MAP/ nDCG, answer status and exact-match accuracy, citation precision/recall, supported-answer and false-answer rates, human rubric means, and deterministic query/retrieved/answer/evidence size counters. External request count and estimated external cost are zero for the current local pipeline. Static metric gates and directional baseline rules are never silently skipped: missing metrics, missing baselines, or insufficient samples produce incomplete; violations produce failed; suites without policy checks are clearly observational. Findings identify affected cases without storing human free-text notes or reviewer identities.

CLI

pagetrace ingest annual-report.pdf --store .pagetrace
pagetrace inspect sha256-<64-lowercase-hex-characters> --store .pagetrace
pagetrace extract-text sha256-<64-lowercase-hex-characters> --store .pagetrace
pagetrace inspect-text sha256-<64-lowercase-hex-characters> \
  text-sha256-<64-lowercase-hex-characters> --store .pagetrace
pagetrace ocr sha256-<64-lowercase-hex-characters> \
  text-sha256-<64-lowercase-hex-characters> --store .pagetrace
pagetrace inspect-ocr sha256-<64-lowercase-hex-characters> \
  ocr-sha256-<64-lowercase-hex-characters> --store .pagetrace
pagetrace evaluate-ocr reference.txt prediction.txt --json
pagetrace structure sha256-<64-lowercase-hex-characters> \
  text-sha256-<64-lowercase-hex-characters> \
  ocr-sha256-<64-lowercase-hex-characters> --store .pagetrace
pagetrace inspect-structure sha256-<64-lowercase-hex-characters> \
  structure-sha256-<64-lowercase-hex-characters> --store .pagetrace --json
pagetrace build-corpus sha256-<64-lowercase-hex-characters> \
  structure-sha256-<64-lowercase-hex-characters> --store .pagetrace
pagetrace inspect-corpus sha256-<64-lowercase-hex-characters> \
  corpus-sha256-<64-lowercase-hex-characters> --store .pagetrace --json
pagetrace retrieve sha256-<64-lowercase-hex-characters> \
  corpus-sha256-<64-lowercase-hex-characters> "revenue growth" --store .pagetrace --json
pagetrace evaluate-retrieval sha256-<64-lowercase-hex-characters> \
  corpus-sha256-<64-lowercase-hex-characters> retrieval-dataset.json \
  --top-k 10 --store .pagetrace --json
pagetrace answer sha256-<64-lowercase-hex-characters> \
  corpus-sha256-<64-lowercase-hex-characters> "revenue growth" \
  --top-k 10 --minimum-coverage 0.25 --store .pagetrace --json
pagetrace evaluate-quality quality-suite.json --baseline prior-quality-report.json --json
pagetrace portfolio-demo portfolio-output

# The module entry point is equivalent.
python -m pagetrace ingest annual-report.pdf --store .pagetrace --json

Plain output reports document or artifact identity, provenance, page counts, and per-page routing. --json emits the relevant canonical manifest, artifact, or metric object. Expected document, extraction, OCR, structure, corpus, retrieval/evaluation, and QA errors are concise, have a non-zero exit status, and do not display a traceback.

evaluate-quality returns status 0 for passed or observational reports, 1 for failed policy checks, 3 when required gates/regressions could not be evaluated, and 2 for malformed or unsafe inputs. This makes missing baseline data distinguishable from a measured regression.

Artifact and identity design

Storage is content-addressed and never uses an input filename as a destination path:

STORE/
|-- .staging/
`-- documents/
    `-- sha256-<full-digest>/
        |-- manifest.json
        |-- source.pdf | source.png | source.jpg
        `-- artifacts/
            |-- text/
            |   `-- text-sha256-<full-digest>.json
            |-- ocr/
            |   `-- ocr-sha256-<full-digest>.json
            |-- structure/
            |   `-- structure-sha256-<full-digest>.json
            `-- corpus/
                `-- corpus-sha256-<full-digest>.json

The canonical manifest uses schema version 1, stable UTF-8 JSON serialization, a full SHA-256 fingerprint, and 1-based page numbering. Identical bytes produce identical document IDs, page IDs, and manifests regardless of the input path or filename. Source filename, source path, timestamps, temporary paths, usernames, and host information are deliberately excluded from the canonical manifest.

Input is copied once into a same-store staging directory in bounded chunks while it is hashed. Validation and final persistence use those staged bytes. A complete artifact is atomically moved into its final directory, and re-ingestion verifies an existing artifact instead of overwriting it.

Text artifacts use their own schema version 1 and identity. The artifact ID is derived from the document fingerprint, extraction schema, extractor/backend versions, and output-affecting configuration, including extraction output limits—not from a filename, path, timestamp, machine, username, or random UUID. Page results retain the exact Milestone 1 page IDs. Canonical text-result bytes include an integrity fingerprint; complete files are promoted atomically without overwriting contradictory artifacts. Readback re-verifies both the text artifact and its underlying Milestone 1 source.

OCR artifacts use a separate schema-v1 identity derived from the document fingerprint, exact source text-artifact identity and content fingerprint, routing/rendering/limit configuration, processor version, installed RapidOCR/ONNX Runtime/PDFium versions, and SHA-256 of the three bundled model files. Page results record source routing status, whether OCR ran, rendered dimensions, exact OCR text, bounded counts, and rounded mean line confidence. Readback re-verifies every provenance link.

Structured artifacts use schema version 1 and derive identity from the document fingerprint, exact source text and OCR artifact identities/content fingerprints, structure configuration, and installed pdfplumber version for PDFs. Page results preserve source page IDs, coordinate units, rotation, positioned text spans, detected table grids, and cells. Canonical content is checksummed, promoted atomically without overwriting contradictions, and revalidated against all three earlier stages during readback.

Corpus artifacts use schema version 1 and derive identity from the document fingerprint, exact source structure identity/content fingerprint, corpus processor version, chunk size, separator, and resource limits. Each chunk has its own stable corpus-scoped identifier and content checksum plus exact page, region, coordinate, span, and character-slice provenance. Readback reloads the structured source, regenerates the expected corpus deterministically, and rejects any contradiction.

Retrieval results and evaluation datasets use schema version 1. Result identity captures the exact corpus identity/content fingerprint, query, BM25 configuration, processor/tokenizer versions, and Unicode data version. Canonical results checksum the complete ranked hit list. Evaluation dataset identity captures ordered queries and binary relevant chunk IDs; evaluation identity binds that dataset to the exact corpus and retrieval configuration. These objects are serializable but are not placed in artifact storage automatically because queries may contain sensitive data.

Evidence-grounded QA results also use schema version 1. Answer identity binds the exact retrieval result/content, extractive processor/segmenter versions, coverage policy, separator, and output limits. The answer checksum covers status, abstention reason, answer, and every citation. The full canonical retrieval result is embedded so citation slices can be checked against their ranked source text without a hidden index. QA results are not placed in artifact storage automatically because questions, retrieved text, and answers may be sensitive.

Quality suites and reports use schema version 1. Suite identity covers ordered text fixtures, retrieval evaluations, QA judgments/results, structured human reviews, and the complete policy. Report identity binds the suite, processor version, optional baseline report, policy, metrics, gates, regressions, findings, and a content checksum. Canonical strict JSON is explicit caller output and is not silently persisted; suites may embed sensitive QA retrieval text and must be handled accordingly.

The Milestone 8 backend persists canonical workflow requests, results, public errors, and ordered lifecycle events in a schema-v1 SQLite database. Submission idempotency binds one caller-provided key to the workflow and canonical request fingerprint. Workers claim jobs atomically, enforce bounded attempts and result sizes, and recover jobs left running after a coordinated single-host process stop. The built-in HTTP server binds only to loopback, requires a 32–512 character bearer token for all /v1 routes, and exposes unauthenticated liveness/readiness checks without document data. It also validates the exact loopback Host and optional Origin, bounds stalled connections, and emits same-origin browser protections.

Initialize a local backend, run a worker, or serve the API with:

pagetrace-backend --database .pagetrace/backend.sqlite3 init
pagetrace-backend --database .pagetrace/backend.sqlite3 worker
# Set PAGETRACE_BACKEND_TOKEN to a high-entropy secret first.
pagetrace-backend --database .pagetrace/backend.sqlite3 serve
# Preview, then confirm, deletion of terminal jobs older than seven days.
pagetrace-backend --database .pagetrace/backend.sqlite3 purge --older-than-hours 168
pagetrace-backend --database .pagetrace/backend.sqlite3 \
  purge --older-than-hours 168 --confirm

Open http://127.0.0.1:8765/ and enter the same bearer token. The Evidence Desk keeps that token only in page memory. It does not use cookies, local storage, or session storage. Ingestion accepts a path under the backend's configured --input-root; browsers do not upload source bytes. The web product can start all three built-in workflows, follow their durable state and ordered events, cancel active work, reopen a known job ID, and inspect canonical results and provenance.

Authenticated routes are POST /v1/jobs, GET /v1/jobs/{job_id}, GET /v1/jobs/{job_id}/events, POST /v1/jobs/{job_id}/cancel, and GET /v1/metrics. GET /healthz and GET /readyz expose status only. A submission body has the following stable envelope:

{
  "workflow": "ingest_document",
  "request": {"source_path": "incoming/example.pdf"},
  "idempotency_key": "upload-2026-0001",
  "max_attempts": 3
}

ingest_document accepts source_path under the configured input root. answer_question accepts document_id, corpus_artifact_id, question, and optional QA/retrieval bounds. evaluate_quality accepts a canonical suite object and an optional baseline report object. Unknown workflow fields are rejected rather than ignored.

The HTTP boundary is deliberately single-host and standard-library based. It accepts only exact loopback Host/Origin authorities on the listening port. Do not expose, forward, or tunnel the built-in server to an untrusted network; remote and multi-user deployment are unsupported.

The packaged web assets are public so the login screen can load, while all /v1 data routes remain bearer-authenticated. The server accepts only its generated asset-name pattern, rejects symlinked or out-of-root files, caps asset size, sends a restrictive content security policy, blocks framing and MIME sniffing, and disables caching for HTML. Fingerprinted JavaScript and CSS receive immutable caching. Evidence text is inserted as text content rather than interpreted as markup.

Security boundary and limits

Milestone 1 rejects missing paths, directories, symbolic-link inputs, empty/unsupported/malformed files, encrypted PDFs, excessive file/page/pixel counts, and multi-frame images. Pillow decompression-bomb conditions become explicit document-limit errors. Readback rejects unsafe IDs, unknown schema versions, malformed manifests, missing/unsafe files, and source size/fingerprint mismatches.

Character limits bound text accepted into each page result and the cumulative canonical artifact; they are checked only after pypdf.extract_text returns. They do not sandbox pypdf or prevent a malicious compressed/content stream from consuming parser CPU or RAM first. pypdf and Pillow remain in-process, and OS/subprocess memory and time isolation does not exist. Parser vulnerabilities and filesystem, race, or deeply nested PDF denial-of-service scenarios remain possible. See SECURITY.md for the precise implemented controls and residual risks.

OCR page/pixel limits are checked before rendering; actual dimensions are checked again after rendering, and line/text limits are checked after RapidOCR returns. RapidOCR, ONNX Runtime, PDFium, and image codecs still run in-process without hard CPU, memory, or wall-clock isolation. The bundled models avoid runtime model downloads, but optional dependency installation remains a supply-chain operation.

Structure limits bound accepted spans, tables, and cells only after pdfplumber or OCR inference returns. pdfplumber and OCR geometry replay remain in-process without hard CPU, memory, or wall-clock isolation. Line-based table detection is a parser result, not a semantic guarantee that a detected grid is correct.

Corpus construction introduces no new parser or model. Its typed chunk, fragment, and character limits are enforced before persistence. Canonical corpus files are additionally capped at 512 MiB. Construction still occurs in-process and may consume memory proportional to accepted structured text and canonical output.

BM25 retrieval tokenizes all accepted chunks in-process for each query; there is no persistent index, cache, subprocess, or hard CPU/memory/time isolation. Query character/token limits bound accepted input, while the existing corpus limits bound scanned content. Queries remain untrusted data and are never executed or sent to an external service.

Evidence-grounded QA adds no model, parser, network call, or execution path. It processes bounded retrieval results in-process and emits only exact source substrings separated by a configured delimiter. Coverage is lexical evidence selection, not a truth or completeness score. Document instructions remain untrusted text even when they appear in an answer. Questions, retrieval results, and QA results are not silently retained or disclosed externally.

Quality evaluation adds no parser, model, network request, or document execution path. It consumes already validated metric/result objects, records no reviewer identity or free-text review, and applies a 512 MiB CLI input ceiling. Quality metrics and gates describe measured fixtures only; they are not proof of general correctness, safety, truth, fairness, or production readiness.

The product backend stores workflow requests and results, which may include source paths, questions, retrieved evidence, answers, and complete quality suites. Operators must protect and expire the SQLite database and artifact store according to their data-retention requirements. Bearer authentication is not transport encryption; the built-in server is loopback-only and does not provide TLS, user accounts, per-user authorization, distributed leases, hard execution timeouts, or operating-system isolation. Cancellation of a running in-process handler takes effect when that handler returns. The backend defaults to 10,000 retained jobs and rejects new work at the ceiling. Its explicit purge command permanently deletes only bounded batches of terminal jobs and their events; it does not delete content-addressed artifacts, inputs, logs, exports, or backups.

See deployment and operations for the supported local topology, retention, backup, monitoring, and upgrade checklist, and incident response for containment, evidence preservation, recovery, privacy deletion, and post-incident review.

All extracted text remains untrusted document data. PageTrace does not execute embedded commands, scripts, URLs, or instruction-like text, and no extracted content should be treated as a system or model instruction.

Quality and validation

ruff check .
ruff format --check .
mypy src tests
pytest
pip-audit --local --skip-editable
pagetrace portfolio-demo portfolio-run-one
pagetrace portfolio-demo portfolio-run-two
python -m build
python -m twine check --strict dist/*
git diff --check

pytest enforces branch-aware coverage with a 90% minimum. The contribution guide documents the clean-wheel import and CLI smoke tests used by continuous integration.

Project policies

• CONTRIBUTING.md — local workflow and quality expectations

• SECURITY.md — implemented controls, residual risks, and reporting

• ROADMAP.md — milestone responsibilities

• CHANGELOG.md — unreleased changes

• docs/DEPLOYMENT.md — supported deployment and operational controls

• docs/INCIDENT_RESPONSE.md — incident handling runbook

• docs/PORTFOLIO.md — reproducible evidence demonstration and claim limits

• docs/RELEASING.md — tagged candidate verification and release checklist

Author / Maintainer

Asadullah Hussaini (@HivMindAI) — Project owner and primary developer/maintainer.

License

PageTrace is available under the MIT License.