Repository navigation
Refuse signature wrapping and bound what hostile input can allocate - #33
Open
CastleSoft wants to merge 3 commits into
Open
CastleSoft wants to merge 3 commits into
CastleSoft wants to merge 3 commits into
Conversation
…ng and corpora A writer that only reads its own output back cannot see the mistakes it shares with itself. validation.yml now puts the example output and a validation corpus (written by tools/SharpCorePDF.Validation, with a manifest of the text, passwords, permissions, revisions and signature coverage each document must have) through qpdf, pypdf, pdfminer.six, MuPDF, pdf.js, fontTools, OTS, PDFium and pyHanko, checks the incremental-update invariants, and replays the fuzz seeds and four hostile documents. robustness.yml runs libFuzzer over the parser and the DCT, CCITT, LZW, Flate and RunLength decoders, reads the veraPDF and pdf.js corpora one process per file, and judges the PDF/A writer with veraPDF over the veraPDF corpus. compliance.yml cross-validates PDF/A-4. The checks found defects that are not fixed here. Each is pinned by a skipped test (PdfSignatureAttackTests, PdfIncrementalInvariantTests) or a strict --known-defect flag that fails the run once the defect is fixed, and docs/validation.md lists them with their evidence. No shipped package changes. The constitution's statement that no Python validator runs in CI is corrected as a proposed PATCH amendment (2.0.1) for the maintainer to ratify. Local AI assistant state (.claude/, CLAUDE.local.md) is ignored.
pdf.js 6 has no PDFDocumentProxy.destroy(), so the encryption check's pdf.js leg reported every document as unopenable: the call threw inside the try that was meant to catch a wrong password. The loading task owns the document and destroying it releases both. The fix was made before the previous commit but an editor saved a stale copy of the file over it.
Signature wrapping: the validator required only that the byte range start at 0 and end at the end of the file, so bytes written into the unsigned gap could close the signature value early and add entries while the digest still matched, and the signature was reported valid and covering the whole document. The gap must now be exactly one hexadecimal string equal to the dictionary's /Contents; anything else is MalformedByteRange, integrity Unknown and CoversWholeDocument false. Decompression bombs: Flate, LZW and RunLength decoded without a limit, so a 1.1 MB document took the process past 1 GiB. PdfEngineOptions.MaxDecodedStreamBytes (128 MiB) now stops them; decode buffers above 16 MiB no longer stay in the shared array pool, and buffer growth no longer overflows. Stack overflow: a chain of streams whose /Length is the next stream was resolved by recursion, and 60,000 links ended the process. The chain is followed no deeper than MaxNestingDepth, then the stream is found by its endstream keyword, as for a /Length cycle. Allocations sized by the file: an object stream's /N is bounded by what the stream can hold, and a JPEG whose output is above the caller's budget is measured from its frame header instead of being decoded first; the working sizes are computed in 64 bits. Tests that fail without the fixes: PdfSignatureAttackTests (gap wrapping, gap shape), PdfFilterTests (limits per filter, content stream past the limit), PdfReaderRecoveryTests (length chain, indirect lengths, object stream /N), PdfJpegFilterTests (oversized frames, working buffers). The three hostile documents of validation.yml are read under a 1 GiB heap, and their --known flags are removed.
This was referenced Nov 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
Fixes the four security defects the independent validation checks found (#32). A document can no longer
be changed after signing and still be reported as covering the whole file, and hostile input can no
longer exhaust memory or end the process: the decoded size of a stream is bounded, a
/Lengthchain isbounded, and an object stream's
/Nand a JPEG frame's size no longer size allocations unchecked. Oneoption is added,
PdfEngineOptions.MaxDecodedStreamBytes(128 MiB).Stacked on #32. This branch is #32's two commits plus one: until #32 merges, the diff here includes
them. Review
7020f3balone, or merge #32 first.Why, and what is wrong today
of the file. Bytes written into the unsigned gap can close the signature value early and add entries
while every signed byte, and so the digest, is unchanged; the signature was reported valid and covering
the whole document. The gap must now be exactly one hexadecimal string equal to the dictionary's
/Contents; anything else isMalformedByteRange, integrityUnknown,CoversWholeDocumentfalse.OutOfMemoryException). Flate, LZW andRunLength now stop at
MaxDecodedStreamBytes: a content stream past it is reported unreadable throughthe warning list, an object or cross-reference stream refuses the document with
PdfSourceFormatException. Decode buffers above 16 MiB no longer stay in the shared array pool, whichotherwise kept hundreds of megabytes across read passes.
/Lengthis the next stream ended the process — a stackoverflow no caller can catch. The chain is followed no deeper than
MaxNestingDepth, then the stream isfound by its
endstreamkeyword, as a/Lengthcycle already was./N 2147483647asked for 16 GiB;/Nis now bounded by what the stream can hold. A JPEG frame of 65,535 × 65,535 allocated its sampleplanes before the caller's budget was consulted; above the budget it is now measured from the header
(the same answer as before — format and size, no pixels — without the allocation), and the working
sizes are computed in 64 bits.
The test that fails without this change
Each was run with its fix reverted and failed with the original defect:
PdfSignatureAttackTests.Does_not_report_a_gap_that_holds_more_than_the_signature_value_as_covering_the_document(was skipped as a known defect in Validate the output with independent readers and the input with fuzzing and corpora #32),
Refuses_a_gap_that_is_not_one_hexadecimal_stringPdfFilterTests.Refuses_a_flate_stream_that_decodes_past_the_limit, the LZW, RunLength and literal-runcompanions, and
Reports_a_content_stream_past_the_limit_as_unreadable_instead_of_inflating_itPdfReaderRecoveryTests.Reads_a_stream_whose_length_is_a_chain_of_sixty_thousand_streams(without thefix this takes the test host down — which is the defect),
Reads_an_object_stream_whose_count_its_header_cannot_hold(
OutOfMemoryException),Resolves_the_indirect_length_of_every_stream_of_a_documentPdfJpegFilterTests.Measures_a_frame_too_large_for_the_budget_without_allocating_it(
ArgumentException/ArgumentOutOfRangeExceptionfrom 32-bit overflow),Refuses_a_frame_whose_working_buffers_cannot_fit_an_arrayChecklist
dotnet build .\SharpCorePDF.slnx -c Releaseis clean on both legs, with zero warningsPublicAPI.Unshipped.txtupdated in the same commit (PdfEngineOptions.MaxDecodedStreamBytes)CHANGELOG.mdhas an entry under[Unreleased](Added, and Fixed — marked as security fixes)MaxNestingDepth's new effect, the untrusted-input note),docs/validation.mdAnything else a reviewer should know
/ID[0]defect) on net10.0 and on net11.0. The three hostile documents in
validation.ymlare read underDOTNET_GCHeapHardLimit= 1 GiB, five runs out of five, and their--knownflags are removed;jpeg-selector.pdfkeeps its flag (a separate defect, next). The pdf.js corpus (987 files) reads exactlyas before the change — 977 read, 4 refused, the 6 known faults — so the new limits refuse nothing
legitimate. pyHanko, qpdf, the text extractors and the encryption matrix agree as before, and a genuine
signature still validates
Valid/Trusted.refused only intermittently inside a 1 GiB heap: the buffer doubles on the large-object heap, which the
collector does not compact by default, and every read pass met the stream again. 128 MiB is far above
any real content stream, object stream or font program and refused reliably. The source read keeps its
pooled buffer, so the large-document numbers in
docs/measurements.mdare unaffected.(130 of 161). The survivors were read: the working-buffer arithmetic can only be pinned at a 2 GB
budget, the pool flag is not observable from a test (it is covered by the heap-limit runs instead), the
checkedcasts and the depth>/>=boundary are equivalent at reachable sizes, and the encrypted-document branch of the gap check is unreachable in-repo because the signer refuses encrypted documents.
choice, on the basis that this fix follows immediately.