DocLens

An OutSystems Developer Cloud (ODC) External Library for reliable text extraction from PDFs and standalone images — built as a drop-in replacement for the parts of TEXTractor that were crashing.

Not tied to any specific app or flow. Usable from any ODC app or library that needs to read text out of a document, whether it's a normal PDF, a scanned/photographed page with no text layer, or a standalone image.

The bug this fixes

TEXTractor's GetText2 action crashes with:

System.IO.IOException: Failed to load image from memory
(wrapped in OutSystems.Application.ErrorHandling.ExtensionException)

on PDFs whose embedded JPEGs were re-saved by tools like iLovePDF, which chain two filters on the image stream:

/Filter [/FlateDecode /DCTDecode]

The raw stream bytes then start with a zlib header (78 9C), not the JPEG SOI marker (FF D8) — a JPEG decoder handed those bytes directly crashes immediately. TEXTractor is closed-source (built on a modified fork of Toxy), so the bug can't be patched directly.

Root-cause fix, in DocLens.TryGetImageBytes: every embedded image used internally for OCR is checked for a zlib header before being treated as JPEG data, and inflated first if needed. Confirmed against real documents that reliably crashed TEXTractor.

What it does

Action	Input	Output
ReadTextFromPdf	PDF bytes, password, OCR language	Text — native text layer where available, OCR fallback per page otherwise
ReadTextFromPdfFromUrl	Same, but downloads the PDF from a URL first	Same
ReadTextFromImage	Standalone image bytes (JPEG/PNG/TIFF/BMP/GIF/WEBP)	OCR text — same OCR path used internally for image-only PDF pages
ReadTextFromImageFromUrl	Same, but downloads the image from a URL first	Same

That's the whole surface, by design — four actions, one job each: get text out of a PDF or an image, inline or by URL. All of them accept a fileName (used only to make error messages traceable) and return (bool Success, ..., string ErrorMessage) — nothing throws out to the caller; failures are always reported via ErrorMessage.

The *FromUrl variants exist because ODC External Library actions have a 5.5MB inline payload limit; TEXTractor documents the same pattern as _FromREST/_WithREST action suffixes. Above that size, host the file somewhere reachable by URL (e.g. a Binary Data entity attribute) and use the *FromUrl action instead — the library downloads it itself, so there's no practical size ceiling beyond normal execution limits.

Password-protected PDFs

Fully supported and tested (encrypt/decrypt round-tripped with a real password-protected PDF): pass the password in the password parameter. Wrong or missing password on an encrypted PDF returns a clear ErrorMessage instead of throwing.

OCR quality

Every OCR call tries RapidOcrNet (PP-OCRv6 small, ONNX models run via Microsoft.ML.OnnxRuntime) first, falling back to Tesseract only if RapidOcrNet's model files are missing or it produces no text at all — see RapidOcrEngine.cs and DocLens.RunOcr. This is what makes handwritten text (a name, a date, a note filled into a printed form) come back correctly in the same call as the printed text around it, instead of needing a separate action or engine choice — RapidOcrNet reads both in one pass. This was decided after side-by-side testing against a real handwritten medical certificate: Tesseract reads printed text well but turns handwriting into garbage; a transformer OCR model built for isolated clean handwriting lines (TrOCR) did worse than Tesseract on this kind of mixed printed-form-with-handwritten-fields document; RapidOcrNet/PaddleOCR got most fields right for both. Full writeup: docs/superpowers/plans/2026-09-11-rapidocr-handwriting.md.

RapidOcrNet needs no native library compilation — unlike Tesseract/ Leptonica (see "Linux native libraries" below), Microsoft.ML.OnnxRuntime ships its own official linux-x64 binaries as a plain NuGet dependency.

Before the Tesseract fallback runs (RapidOcrNet does its own preprocessing internally), images are deskewed (Leptonica's own skew estimator — already bundled, since it's what Tesseract itself depends on), upscaled (if their long edge is under ~2200px), and converted to grayscale. Deskewing only corrects rotation; if the estimator isn't confident there's a skew to fix, it safely leaves the image unchanged rather than guessing. Upscaling measurably improves results on photographed documents (e.g. a phone photo of a payslip), where the actual document content is often much smaller in pixel terms than the full image once background is accounted for. On one test document this took OCR output from 0 characters to full field-level extraction (name, surname, sex, nationality on a South African ID card).

OCR accuracy still depends on source image quality — a blurry, low-light phone photo won't OCR as cleanly as a proper scan, no matter how good the preprocessing (deskewing a blurry photo doesn't sharpen it — confirmed against a low-quality test payslip, where OCR output was unchanged before and after adding deskew, since that particular photo isn't actually tilted). If OCR quality matters for a given flow, encourage flatter, better-lit photos, enforce a minimum resolution, or scan instead of photographing.

Project layout

src/DocLens/              The ODC External Library itself
  IDocLens.cs              Interface — the ODC-facing contract
  DocLens.cs                Implementation
  RapidOcrEngine.cs         Primary OCR engine (PP-OCRv6 via RapidOcrNet/ONNX)
  tessdata/                Local-testing-only trained data (eng, por)
  rapidocr-models/          Local-testing-only PP-OCRv6 ONNX models + dict

test/DocLens.TestHarness/ Console harness — runs text extraction against
                            every PDF in Files/ and reports pass/fail

Files/                     Real-world test documents (not committed to
                            the repo — see .gitignore)

Building and testing locally

Requires the .NET 8 SDK.

cd test/DocLens.TestHarness
dotnet build
dotnet run --no-build

macOS-only: getting OCR to actually run locally

The Tesseract NuGet package only bundles Windows native binaries. Its loader looks for two specific, version-pinned file names (libleptonica-1.82.0.dylib, libtesseract50.dylib) that don't exist on macOS or Linux by default. This is a local-testing convenience problem only — it has no bearing on the real deployment target.

The test harness's .csproj has a post-build step (StageMacOsTesseractNatives) that copies a real Homebrew-installed Tesseract under those exact expected names, so dotnet run just works:

brew install tesseract

Then always run with:

DYLD_LIBRARY_PATH=/opt/homebrew/lib dotnet run --no-build

This step is not needed for ODC deployment — the real Linux runtime ships its own bundled native binaries (see below).

Deploying to ODC

1. Build the release DLL for the real target runtime:

   cd src/DocLens
   dotnet publish -c Release -r linux-x64 --self-contained false

2. Package as a ZIP — the built DLL + its dependencies at the zip root, per the ODC External Libraries SDK convention.

3. Model files for OCR — both the PP-OCRv6 ONNX models (rapidocr-models/PP-OCRv6_det_small.onnx, rapidocr-models/PP-OCRv6_rec_small.onnx, rapidocr-models/ppocrv6_dict.txt, ~31MB combined — the "medium" preset was tried first but at ~138MB doesn't fit the ODC Portal's 90MB library upload limit once tessdata/native libs are added; see docs/ODC-EXTERNAL-LIBRARY-NOTES.md section 16) and the Tesseract fallback's trained data (eng.traineddata/por.traineddata from tessdata_fast, ~6MB) are bundled directly inside the library's own zip, the same way the native .so files are — dotnet publish copies them there automatically (see the .csproj), and both RapidOcrEngine.ModelPath and TessDataPath already default to paths under AppContext.BaseDirectory, which resolve correctly since External Libraries run as their own isolated Lambda function with their own bundled files. No separate app-side resource setup needed. RapidOcrNet also needs the PP-OCRv5 line-orientation classifier it bundles itself (models/v5/ch_PP-LCNet_x0_25_textline_ori_cls_mobile.onnx, copied automatically by the NuGet package's own build target) — confirm it's present in the publish output before zipping (see RapidOcrEngine.ClsModelPath).

4. Upload and publish the external library via the ODC Portal (Integrate → External Logic → Create library), or via the ODC REST API if you want to automate this.

5. Test in an app before releasing the first stable version (the ODC Portal has a "Try library in an app" option on the library's version history).

Deployment status

Published and confirmed working end-to-end on a real ODC tenant, library DocLens, all 4 actions live. OCR verified with real extracted text (not just "no crash") directly against the production runtime, not inferred from local testing.

Linux native libraries (Tesseract/Leptonica) — the real story

The Tesseract NuGet package only bundles Windows binaries, and getting OCR working correctly on ODC's actual runtime took three separate fixes, each found by testing against the real environment rather than assuming:

1. Confirming what the runtime actually is. ODC External Libraries run as AWS Lambda functions on Amazon Linux 2023, not a generic Linux/Debian image — found via a small diagnostic action called directly against the live environment. An earlier attempt built against a Debian bookworm container, which looked identical locally but crashed in production (native ABI mismatch).
2. Compiling libtesseract/liblept from source inside a real amazonlinux:2023 container (neither NuGet nor that distro's package repos provide them), stripped of debug symbols (~15MB total).
3. libdl doesn't exist as a separate file on Amazon Linux 2023. Since glibc 2.34, dlopen/dlsym/etc. live directly in libc.so.6; AL2023 dropped the old libdl.so.2 compatibility stub entirely (Debian still ships it, which is why this only surfaced against the real target). Fixed via NativeLibrary.SetDllImportResolver, redirecting libdl lookups straight to libc.so.6.

Also needed: explicitly pre-loading every native dependency by absolute path via NativeLibrary.Load() in a fixed-point loop (DocLens.EnsureLinuxNativeLibrariesLoaded), since LD_LIBRARY_PATH set from managed C# code has no effect — Linux's dynamic linker only reads it once, very early at process startup, before any of this code runs.

See docs/ODC-EXTERNAL-LIBRARY-NOTES.md for the full story, including how each of these was actually diagnosed (not guessed) — reusable notes for the next ODC component with native dependencies.

Known follow-ups

• RapidOcrNet works on real Linux x64 (verified via Docker), but two things still need checking on the real ODC tenant. A real publish -r linux-x64 was run inside mcr.microsoft.com/dotnet/runtime:10.0 (QEMU-emulated on Apple Silicon) and correctly extracted the handwritten sample — no native library issues at all, unlike the Tesseract saga (see "Linux native libraries" above). Two open items from that same test run, detailed in docs/ODC-EXTERNAL-LIBRARY-NOTES.md section 16:
  1. Latency: one OCR call took 112s under QEMU emulation. Very likely inflated by QEMU's known weakness emulating the SIMD instructions ONNX Runtime relies on, but this can't be confirmed without testing on real Lambda hardware — and section 14 already documents that OCR can exceed OutSystems' ~10s synchronous action limit even with Tesseract alone, so this needs measuring for real before use in a synchronous screen call.
  2. Two crashes unrelated to OCR (NullReferenceException and a segfault) surfaced in pure PdfPig code during the same test — one of them on the exact file section 15 already diagnosed as a QEMU-only false positive. Per that section's own lesson, not acted on here, but worth re-checking if they ever show up outside QEMU.
  • Deployment size, measured for real: the built zip is 58MB (after trimming ~12MB of unused Windows Tesseract DLLs that were being bundled unnecessarily even before this change) — comfortably under the Portal's 90MB manual-upload limit, but over the MCP extlib_upload tool's 50MB hard cap. Upload this version via the ODC Portal directly, not the MCP tool.
• .NET runtime version: ODC flagged that .NET 8 will be discontinued Nov 10, 2026 (expected — that's .NET 8's normal LTS end-of-support date) and recommends net10.0. The project already targets <TargetFramework>net10.0</TargetFramework> locally (not yet published under that target) — republish once convenient, well before the deadline.
• A real crash was found in image-extraction logic on one large, complex PDF under local Docker/QEMU emulation testing. Raw image-extraction (ExtractImagesFromPdf) was removed from the public interface entirely rather than shipped with a known crash — the library now only ever returns text, which sidesteps the issue and keeps the API surface small and focused. Worth investigating properly before ever re-adding an image-extraction action.