Skip to content

Refuse signature wrapping and bound what hostile input can allocate - #33

Open
CastleSoft wants to merge 3 commits into
MPCoreDeveloper:mainfrom
CastleSoft:security-fixes
Open

CastleSoft wants to merge 3 commits into
MPCoreDeveloper:mainfrom
CastleSoft:security-fixes

Conversation

@CastleSoft

Copy link
Copy Markdown

What this changes

Fixes the four security defects the independent validation checks found (#32). A document can no longer
be changed after signing and still be reported as covering the whole file, and hostile input can no
longer exhaust memory or end the process: the decoded size of a stream is bounded, a /Length chain is
bounded, and an object stream's /N and a JPEG frame's size no longer size allocations unchecked. One
option is added, PdfEngineOptions.MaxDecodedStreamBytes (128 MiB).

Stacked on #32. This branch is #32's two commits plus one: until #32 merges, the diff here includes
them. Review 7020f3b alone, or merge #32 first.

Why, and what is wrong today

  • Signature wrapping. The validator only checked that the byte range starts at 0 and ends at the end
    of the file. Bytes written into the unsigned gap can close the signature value early and add entries
    while every signed byte, and so the digest, is unchanged; the signature was reported valid and covering
    the whole document. The gap must now be exactly one hexadecimal string equal to the dictionary's
    /Contents; anything else is MalformedByteRange, integrity Unknown, CoversWholeDocument false.
  • Decompression bombs. A 1.1 MB document inflated past 1 GiB (OutOfMemoryException). Flate, LZW and
    RunLength now stop at MaxDecodedStreamBytes: a content stream past it is reported unreadable through
    the warning list, an object or cross-reference stream refuses the document with
    PdfSourceFormatException. Decode buffers above 16 MiB no longer stay in the shared array pool, which
    otherwise kept hundreds of megabytes across read passes.
  • Stack overflow. 60,000 streams whose /Length is the next stream ended the process — a stack
    overflow no caller can catch. The chain is followed no deeper than MaxNestingDepth, then the stream is
    found by its endstream keyword, as a /Length cycle already was.
  • Allocations sized by the file. A 513-byte object stream declaring /N 2147483647 asked for 16 GiB;
    /N is now bounded by what the stream can hold. A JPEG frame of 65,535 × 65,535 allocated its sample
    planes before the caller's budget was consulted; above the budget it is now measured from the header
    (the same answer as before — format and size, no pixels — without the allocation), and the working
    sizes are computed in 64 bits.

The test that fails without this change

Each was run with its fix reverted and failed with the original defect:

  • PdfSignatureAttackTests.Does_not_report_a_gap_that_holds_more_than_the_signature_value_as_covering_the_document
    (was skipped as a known defect in Validate the output with independent readers and the input with fuzzing and corpora #32), Refuses_a_gap_that_is_not_one_hexadecimal_string
  • PdfFilterTests.Refuses_a_flate_stream_that_decodes_past_the_limit, the LZW, RunLength and literal-run
    companions, and Reports_a_content_stream_past_the_limit_as_unreadable_instead_of_inflating_it
  • PdfReaderRecoveryTests.Reads_a_stream_whose_length_is_a_chain_of_sixty_thousand_streams (without the
    fix this takes the test host down — which is the defect), Reads_an_object_stream_whose_count_its_header_cannot_hold
    (OutOfMemoryException), Resolves_the_indirect_length_of_every_stream_of_a_document
  • PdfJpegFilterTests.Measures_a_frame_too_large_for_the_budget_without_allocating_it
    (ArgumentException / ArgumentOutOfRangeException from 32-bit overflow),
    Refuses_a_frame_whose_working_buffers_cannot_fit_an_array

Checklist

  • dotnet build .\SharpCorePDF.slnx -c Release is clean on both legs, with zero warnings
  • A test fails when this change is reverted, and it is in this pull request
  • PublicAPI.Unshipped.txt updated in the same commit (PdfEngineOptions.MaxDecodedStreamBytes)
  • CHANGELOG.md has an entry under [Unreleased] (Added, and Fixed — marked as security fixes)
  • Documentation updated: manual chapter 10 (the option, MaxNestingDepth's new effect, the untrusted-input note), docs/validation.md
  • No internal story, epic or requirement identifier anywhere
  • Not drive-by: only the files the fixes need; formatter changes outside them were reverted

Anything else a reviewer should know

  • Verified locally (Windows, .NET 11): 1,297 tests pass and 1 is skipped (the unrelated /ID[0]
    defect) on net10.0 and on net11.0. The three hostile documents in validation.yml are read under
    DOTNET_GCHeapHardLimit = 1 GiB, five runs out of five, and their --known flags are removed;
    jpeg-selector.pdf keeps its flag (a separate defect, next). The pdf.js corpus (987 files) reads exactly
    as before the change — 977 read, 4 refused, the 6 known faults — so the new limits refuse nothing
    legitimate. pyHanko, qpdf, the text extractors and the encryption matrix agree as before, and a genuine
    signature still validates Valid/Trusted.
  • The default of 128 MiB is a measured choice. At 512 MiB, and then 256 MiB, the bomb was still
    refused only intermittently inside a 1 GiB heap: the buffer doubles on the large-object heap, which the
    collector does not compact by default, and every read pass met the stream again. 128 MiB is far above
    any real content stream, object stream or font program and refused reliably. The source read keeps its
    pooled buffer, so the large-document numbers in docs/measurements.md are unaffected.
  • Mutation testing (Stryker 5.1.0, project mode, the ten changed files): 80.7 % on the changed lines
    (130 of 161). The survivors were read: the working-buffer arithmetic can only be pinned at a 2 GB
    budget, the pool flag is not observable from a test (it is covered by the heap-limit runs instead), the
    checked casts and the depth >/>= boundary are equivalent at reachable sizes, and the encrypted-
    document branch of the gap check is unreachable in-repo because the signer refuses encrypted documents.
  • Disclosure. These defects were described publicly in Validate the output with independent readers and the input with fuzzing and corpora #32 with reproducers, by the contributor's
    choice, on the basis that this fix follows immediately.

…ng and corpora

A writer that only reads its own output back cannot see the mistakes it shares with itself.
validation.yml now puts the example output and a validation corpus (written by
tools/SharpCorePDF.Validation, with a manifest of the text, passwords, permissions, revisions and
signature coverage each document must have) through qpdf, pypdf, pdfminer.six, MuPDF, pdf.js,
fontTools, OTS, PDFium and pyHanko, checks the incremental-update invariants, and replays the fuzz
seeds and four hostile documents. robustness.yml runs libFuzzer over the parser and the DCT, CCITT,
LZW, Flate and RunLength decoders, reads the veraPDF and pdf.js corpora one process per file, and
judges the PDF/A writer with veraPDF over the veraPDF corpus. compliance.yml cross-validates PDF/A-4.

The checks found defects that are not fixed here. Each is pinned by a skipped test
(PdfSignatureAttackTests, PdfIncrementalInvariantTests) or a strict --known-defect flag that fails
the run once the defect is fixed, and docs/validation.md lists them with their evidence.

No shipped package changes. The constitution's statement that no Python validator runs in CI is
corrected as a proposed PATCH amendment (2.0.1) for the maintainer to ratify. Local AI assistant
state (.claude/, CLAUDE.local.md) is ignored.
pdf.js 6 has no PDFDocumentProxy.destroy(), so the encryption check's pdf.js leg reported every document as unopenable: the call threw inside the try that was meant to catch a wrong password. The loading task owns the document and destroying it releases both. The fix was made before the previous commit but an editor saved a stale copy of the file over it.
Signature wrapping: the validator required only that the byte range start at 0 and end at the end of
the file, so bytes written into the unsigned gap could close the signature value early and add
entries while the digest still matched, and the signature was reported valid and covering the whole
document. The gap must now be exactly one hexadecimal string equal to the dictionary's /Contents;
anything else is MalformedByteRange, integrity Unknown and CoversWholeDocument false.

Decompression bombs: Flate, LZW and RunLength decoded without a limit, so a 1.1 MB document took the
process past 1 GiB. PdfEngineOptions.MaxDecodedStreamBytes (128 MiB) now stops them; decode buffers
above 16 MiB no longer stay in the shared array pool, and buffer growth no longer overflows.

Stack overflow: a chain of streams whose /Length is the next stream was resolved by recursion, and
60,000 links ended the process. The chain is followed no deeper than MaxNestingDepth, then the
stream is found by its endstream keyword, as for a /Length cycle.

Allocations sized by the file: an object stream's /N is bounded by what the stream can hold, and a
JPEG whose output is above the caller's budget is measured from its frame header instead of being
decoded first; the working sizes are computed in 64 bits.

Tests that fail without the fixes: PdfSignatureAttackTests (gap wrapping, gap shape),
PdfFilterTests (limits per filter, content stream past the limit), PdfReaderRecoveryTests (length
chain, indirect lengths, object stream /N), PdfJpegFilterTests (oversized frames, working
buffers). The three hostile documents of validation.yml are read under a 1 GiB heap, and their
--known flags are removed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant