Repository navigation
Read the documents the JPEG, empty-name, glyph-box and date faults refused - #34
Open
CastleSoft wants to merge 4 commits into
Open
CastleSoft wants to merge 4 commits into
CastleSoft wants to merge 4 commits into
Conversation
…ng and corpora A writer that only reads its own output back cannot see the mistakes it shares with itself. validation.yml now puts the example output and a validation corpus (written by tools/SharpCorePDF.Validation, with a manifest of the text, passwords, permissions, revisions and signature coverage each document must have) through qpdf, pypdf, pdfminer.six, MuPDF, pdf.js, fontTools, OTS, PDFium and pyHanko, checks the incremental-update invariants, and replays the fuzz seeds and four hostile documents. robustness.yml runs libFuzzer over the parser and the DCT, CCITT, LZW, Flate and RunLength decoders, reads the veraPDF and pdf.js corpora one process per file, and judges the PDF/A writer with veraPDF over the veraPDF corpus. compliance.yml cross-validates PDF/A-4. The checks found defects that are not fixed here. Each is pinned by a skipped test (PdfSignatureAttackTests, PdfIncrementalInvariantTests) or a strict --known-defect flag that fails the run once the defect is fixed, and docs/validation.md lists them with their evidence. No shipped package changes. The constitution's statement that no Python validator runs in CI is corrected as a proposed PATCH amendment (2.0.1) for the maintainer to ratify. Local AI assistant state (.claude/, CLAUDE.local.md) is ignored.
pdf.js 6 has no PDFDocumentProxy.destroy(), so the encryption check's pdf.js leg reported every document as unopenable: the call threw inside the try that was meant to catch a wrong password. The loading task owns the document and destroying it releases both. The fix was made before the previous commit but an editor saved a stale copy of the file over it.
Signature wrapping: the validator required only that the byte range start at 0 and end at the end of the file, so bytes written into the unsigned gap could close the signature value early and add entries while the digest still matched, and the signature was reported valid and covering the whole document. The gap must now be exactly one hexadecimal string equal to the dictionary's /Contents; anything else is MalformedByteRange, integrity Unknown and CoversWholeDocument false. Decompression bombs: Flate, LZW and RunLength decoded without a limit, so a 1.1 MB document took the process past 1 GiB. PdfEngineOptions.MaxDecodedStreamBytes (128 MiB) now stops them; decode buffers above 16 MiB no longer stay in the shared array pool, and buffer growth no longer overflows. Stack overflow: a chain of streams whose /Length is the next stream was resolved by recursion, and 60,000 links ended the process. The chain is followed no deeper than MaxNestingDepth, then the stream is found by its endstream keyword, as for a /Length cycle. Allocations sized by the file: an object stream's /N is bounded by what the stream can hold, and a JPEG whose output is above the caller's budget is measured from its frame header instead of being decoded first; the working sizes are computed in 64 bits. Tests that fail without the fixes: PdfSignatureAttackTests (gap wrapping, gap shape), PdfFilterTests (limits per filter, content stream past the limit), PdfReaderRecoveryTests (length chain, indirect lengths, object stream /N), PdfJpegFilterTests (oversized frames, working buffers). The three hostile documents of validation.yml are read under a 1 GiB heap, and their --known flags are removed.
…fused Four defects turned documents into exceptions that were not PdfExceptions, so a caller that handles the contract's refusal still saw the read fail: - A JPEG scan selected Huffman tables with 4-bit selectors and a progressive scan named its spectral selection, and both were used as indexes unchecked (IndexOutOfRangeException). ParseScan now refuses a selector above 3, and a progressive scan unless Ss <= Se <= 63 and Ah, Al <= 13. - The empty name, a solidus on its own, is a name PDF allows, and four documents of the pdf.js test corpus use it as a key; PdfDictionary.Set refused it, so they did not open at all. - A glyph drawn upside down (a negative font size) gave PdfGlyph.BoundingBox a bottom above its top, and the rectangle refused it, failing text extraction for the page. - PdfDate.TryParse threw for a time-zone offset beyond fourteen hours, so one malformed /CreationDate failed the inspection of the whole document; such an offset is now read as no offset. PdfContainerTests.Refuses_an_entry_without_a_key asserted the second defect and is rewritten: a null key is still refused, the empty name is kept. Tests that fail without the fixes: PdfJpegFilterTests (table selectors, progressive scan bounds), PdfObjectModelTests (empty-name key, in a dictionary and in a document), PdfTextScannerTests (upside-down glyph), PdfSerializationTests (offsets past fourteen hours). The pdf.js corpus and the hostile documents now read with no fault, crash or timeout, and their --known flags are removed.
This was referenced Nov 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
Four kinds of document that ended a read in an exception the contract does not name — not a
PdfException— are now read: a JPEG whose scan header names tables or coefficients the decoder does nothave, a document that uses the empty name as a dictionary key, a page with a glyph drawn upside down, and a
document whose
/CreationDatestates a time-zone offset no time zone has. With these fixed, the pdf.js testcorpus (987 files) and the four hostile documents read with no fault, crash or timeout.
Stacked on #33 (itself on #32). Until those merge, the diff here includes them; review
9e2ab83alone.Why, and what is wrong today
a progressive scan's spectral selection indexes the 64-entry zig-zag table; both were used unchecked
(
IndexOutOfRangeExceptionfromJpegDecoder.DecodeBlock).ParseScannow refuses a selector above 3, anda progressive scan unless
Ss ≤ Se ≤ 63andAh, Al ≤ 13; the image is reported undecodable as any other./on its own is a name ISO 32000-1 §7.3.5 allows.PdfDictionary.Setrefused anempty key with
ArgumentException, which surfaced while parsing, so four pdf.js corpus documents(
issue3188,issue5896,pdfjs_wikipedia,poppler-742-0-fuzzed) did not open at all.PdfGlyph.BoundingBoxhandedPdfRectanglea bottom above its top (issue14415). The box is now takenbetween the two whichever way round they are.
PdfDate.TryParseclamped hours to 0–23 and handed the result toDateTimeOffset, which throws past ±14:00 (TAMReview). Such an offset is now read as no offset, so themethod keeps its contract of answering rather than throwing.
The test that fails without this change
Run with the four source changes reverted: 14 fail, each with the original exception.
PdfJpegFilterTests.Refuses_a_scan_that_selects_a_table_beyond_the_four_there_are(4 cases),Refuses_a_progressive_scan_outside_the_block(the spectral cases)PdfObjectModelTests.Keeps_an_entry_whose_key_is_the_empty_name,Opens_a_document_that_uses_the_empty_name_as_a_keyPdfTextScannerTests.Gives_a_glyph_drawn_upside_down_a_boxPdfSerializationTests.Reads_an_offset_no_time_zone_has_as_no_offset(the out-of-range cases; thein-range cases pass either way on purpose — they pin that ±14:00 and +05:30 are kept)
Checklist
dotnet build .\SharpCorePDF.slnx -c Releaseis clean on both legs, with zero warningsCHANGELOG.mdhas entries under[Unreleased]→ Fixeddocs/validation.mdmoves the four defects to the fixed table; the--knownflags they had invalidation.ymlandrobustness.ymlare removedAnything else a reviewer should know
PdfContainerTests.Refuses_an_entry_without_a_keyasserted thatan empty key is refused — the defect itself. It becomes
Refuses_an_entry_without_a_key_and_keeps_one_whose_key_is_the_empty_name: anullkey is still aprogrammer error (
ArgumentNullException), the empty name is kept./ID[0]defect,next) on net10.0 and on net11.0. The pdf.js corpus plus the hostile documents: 991 files, 987 read,
4 refused (
PdfException, as before), 0 faults — with no--knownflag left in either workflow.survivors were read:
≥ 3on a selector is only observable with a fixture defining table 3 (ours define 0and 1, so a selector of 3 fails later anyway),
>>>equals>>on a byte, andAh/Alat exactly 13 is alegal value no test meaningfully exercises.