fix: inspect escaped JSON through source-mapped security views

docs/ANALYSIS_RESOURCE_BOUNDS.md

@@ -103,6 +103,86 @@ charged for each projected occurrence, so a compact alias graph cannot amplify t
 manifest past these limits. A malformed or incomplete claimed frontmatter leaves the manifest
 empty, marks the primary artifact `partial`, and records an allowlisted parse error or limit reason.
 
+## JSON quote ownership
+
+Standalone `.json` artifacts also have a separate, **2,097,152-character** host-syntax
+prepass for command-parser completeness. It reuses the iterative grammar to prove the
+entire original document, then treats only JSON punctuation outside strings as word
+boundaries. The proof carries through bounded windows only with ordered source mappings
+and intact original quote delimiters. String contents and all finding views stay visible;
+individual command-word, reconstruction, archive, output, and runtime limits still apply.
+This allowance does not expand the Markdown/frontmatter quote-ownership limits below.
+Concrete destructive commands inside original JSON strings retain an explicit incomplete
+interpretation until their finding provenance is supported. Existing raw and reconstructed
+findings remain visible; structural syntax ownership cannot turn that gap into a clean scan.
+
+The deterministic instruction parser can distinguish structural JSON quotes from instruction
+delimiters only after validating a complete JSON value within **131,072 source characters**.
+This is a structural capacity limit, independent of file-size and scan-time allowances.
+Candidates through 65,536 characters keep the established bounded JSON decoder path. Larger
+candidates use an iterative syntax validator that does not allocate decoded strings, numbers,
+arrays, or objects. A failed small-input decode is never retried through the larger-input path.
+
+The inclusive ceiling counts characters, not UTF-8 bytes or decoded JSON string lengths:
+
+- Standalone JSON includes surrounding whitespace. After bounded frontmatter, only the body
+  following the closing delimiter is counted.
+- An explicitly labeled, closed JSON fence counts its body, including newlines and raw list or
+  blockquote prefixes. Opening and closing fence lines are excluded.
+- Unicode characters count once; each source character of an escape such as `\u0061` counts.
+  The frontmatter boundary search has its own 65,536-character prefix bound.
+
+The iterative validator reads one complete value and trailing JSON whitespace. It rejects
+invalid escapes, raw controls inside strings, malformed numbers, unmatched delimiters, missing
+separators, trailing commas, and additional values. No quotes acquire ownership before validation
+reaches the end. Validation retains a byte per open container in a non-recursive grammar stack;
+the source ceiling therefore bounds nesting storage, token count, and the number of quote spans.
+There is no separate decoded-value allocation or integer-conversion cost. Deadline/cancellation
+checks occur within whitespace, string, number, and container processing, with fewer than 256
+validation characters between checks. The normalized fence-validation copy has a separate
+524,288-character ceiling to account for tabs expanding into at most four columns. Quote offsets
+continue to refer to the original source. The later quote-span collection is also bounded by
+the source characters and retains its existing runtime checks.
+
+This validates structural quote ownership; it does not exempt string contents from security
+analysis. Real commands and reconstructed instructions still pass through the existing analyzers.
+The unchanged 256,000-character analysis windows and their bounded context remain separate limits.
+A JSON value that straddles a window or exceeds another parser/runtime allowance may remain
+incomplete, even when its total size is below the JSON ownership ceiling. No unvalidated window
+fragment is treated as a complete value.
+
+When marker reconstruction is already incomplete and its first unresolved directive lies in an
+oversized JSON candidate, the ledger reports `json_quote_ownership_limit`. Its message identifies
+the zero-based, end-exclusive source character span `[start, end)`, observed character count,
+limit, and next step; the ledger also retains `observed_characters` and `limit_characters`.
+The exception's `path` identifies the source artifact. It does not assert that the candidate is
+valid JSON or that capacity is the only unresolved instruction issue. Invalid, truncated, and
+oversized candidates never acquire quote ownership. Unrelated parser failures retain their
+existing reason codes; an unclosed fence cannot establish a complete JSON body boundary.
+
+To resolve a capacity limitation, split the input into smaller **complete** JSON values or
+documents, preserve all required content and references, and rescan. Do not truncate the input
+or remove a required reference. Raising the timeout cannot raise this structural ceiling.
+Findings remain visible, and unresolved coverage remains incomplete: CLI `--fail-on-incomplete`
+returns nonzero and the MCP installation gate rejects it, even after successful semantic analysis.
+JSON with no unresolved instruction parsing may still complete without needing quote ownership.
+
+This extends the supported candidate size from 65,536 to 131,072 characters with a different,
+bounded validation algorithm. Previously incomplete valid examples can now complete when the
+remaining analysis also succeeds. Consumers that enumerate ledger reasons must recognize
+`json_quote_ownership_limit` for unresolved candidates exceeding the current ceiling, while
+continuing to reject incomplete reports regardless of reason. Do not add the old generic
+`obfuscated_instruction_text` as a second exception merely to preserve an exact-string predicate.
+That generic reason remains available for separate unresolved instruction parsing.
+
+Historical expectations tied to the 65,536-character ceiling must be retained as versioned
+compatibility evidence. A proposed successor contract may require complete analysis for an
+otherwise supported 65,537-character benign value and move the size-boundary rejection control to
+131,073 characters. Such a dataset migration needs explicit acceptance; it does not retroactively
+turn an old failed contract into a pass or weaken an existing expected-complete requirement.
+Further increases require new resource measurements and review, including dense strings, deep
+nesting, malformed input, cancellation, and analysis-window interactions.
+
 ## Intra-bundle references
 
 Reference extraction from the primary instructions is independently bounded:
@@ -149,6 +229,7 @@ malicious evasion. Keep required references and use the reason to choose the fix
 |---|---|
 | `dependency_parse_error` | Correct the malformed dependency manifest or lockfile, then rerun the scan. |
 | `static_parse_limit` | Inspect the expression and analyzer. If valid source is misinterpreted, correct or update the scanner and rerun. |
+| `json_quote_ownership_limit` | Use the reported source span and 131,072-character bound to split complete JSON values while retaining required content, then rescan. |
 | `read_error`, `stat_error`, `file_disappeared`, `missing_file_cache` | Ensure the resolved target remains readable throughout the scan. |
 | `size_limit`, `runtime_limit` | Review the reported bounds and input size; distinguish a scanner performance problem from a legitimate resource ceiling. |
 | `binary_content`, `opaque_content` | Provide inspectable source or analysis support for the referenced format. |

docs/release/json-quote-ownership-migration.md

@@ -0,0 +1,29 @@
+# JSON quote ownership upgrade
+
+Status: unreleased change after the 2.12.0 baseline. The release version will be
+assigned during release preparation; this note does not describe shipped 2.12.0 behavior.
+
+JSON quote ownership supports complete candidates through 131,072 raw source
+characters. Values above the former 65,536-character ceiling use iterative syntax
+validation without decoded-object allocation. Otherwise supported valid examples
+can therefore change from incomplete analysis with strict CLI exit 1 to complete
+analysis with exit 0. All other analysis must still succeed.
+
+The smaller decoder path, frontmatter boundary, analysis windows, and runtime
+gates retain their existing constraints. Structural ownership does not exempt
+commands or instructions inside strings from analysis. Unsupported command
+parsing or instruction reconstruction remains explicitly incomplete.
+
+Remaining size-limited uncertainty reports `json_quote_ownership_limit` with
+source coordinates and the current ceiling. Consumers that enumerate reason
+codes must recognize that reason and reject incomplete reports regardless of its
+name. Do not add a duplicate generic reason merely to satisfy an old predicate.
+
+Retain historical size-bound expectations and failures as versioned compatibility
+evidence. A successor contract can require completion for the previously rejected
+65,537-character benign case and move the rejection control to 131,073 characters.
+This is an explicit supported-input migration, not a retrospective pass for an old
+oracle. Existing expected-complete requirements must not be weakened.
+
+See [JSON quote ownership bounds](../ANALYSIS_RESOURCE_BOUNDS.md#json-quote-ownership)
+for counting rules, validation budgets, remaining limitations, and remediation.

src/skillspector/artifacts.py

@@ -106,6 +106,7 @@ class SecurityTextView:
     reconstructions: tuple[SecurityTextReconstruction, ...] = ()
     right_boundary_is_fixed: bool = False
     right_boundary_recovery_start: int | None = None
+    source_end_offsets: array[int] | None = None
 
     def source_offset(self, derived_offset: int) -> int:
         """Map a derived character offset to the corresponding source offset."""
@@ -116,6 +117,17 @@ def source_offset(self, derived_offset: int) -> int:
         index = min(max(derived_offset, 0), len(self.source_offsets) - 1)
         return self.source_offsets[index]
 
+    def source_end_offset(self, derived_offset: int) -> int:
+        """Return the exclusive original end of one projected character."""
+        if not self.text:
+            return 0
+        if self.source_end_offsets is None:
+            return self.source_offset(derived_offset) + 1
+        if not self.source_end_offsets:
+            return 0
+        index = min(max(derived_offset, 0), len(self.source_end_offsets) - 1)
+        return self.source_end_offsets[index]
+
     def reconstructed_source_spans(
         self,
         derived_start: int,

src/skillspector/inspection_ledger.py

@@ -155,6 +155,7 @@ class LedgerReason(StrEnum):
     OUTPUT_LIMIT = "output_limit"
     TRANSITIVE_CHILD_SCAN_FAILED = "transitive_child_scan_failed"
     STATIC_PARSE_LIMIT = "static_parse_limit"
+    JSON_QUOTE_OWNERSHIP_LIMIT = "json_quote_ownership_limit"
     PYTHON_SOURCE_AMBIGUOUS = "python_source_ambiguous"
     PYTHON_SOURCE_DECODE_ERROR = "python_source_decode_error"
     OBFUSCATED_INSTRUCTION_TEXT = "obfuscated_instruction_text"
@@ -560,6 +561,12 @@ def format_recovery_policy_accepted(completeness: Mapping[str, object]) -> bool:
     LedgerReason.PYTHON_SOURCE_DECODE_ERROR: (
         "Python source bytes could not be decoded under their declared encoding."
     ),
+    LedgerReason.JSON_QUOTE_OWNERSHIP_LIMIT: (
+        "JSON quote ownership was not validated because a candidate source span exceeds "
+        "the character limit. JSON validity remains unverified; other instruction uncertainty "
+        "may remain. Split the document into smaller complete JSON values without dropping "
+        "required content, then rescan. Increasing the scan timeout does not raise this limit."
+    ),
     LedgerReason.OBFUSCATED_INSTRUCTION_TEXT: (
         "Obfuscated instruction text could not be fully evaluated by the deterministic layer."
     ),
@@ -641,6 +648,7 @@ class ParserCheck(StrEnum):
     COMMAND_STRING_DEPTH = "command_string_depth"
     COMMAND_STRING_RUNTIME = "command_string_runtime"
     COMMAND_STRING_EFFECTS = "command_string_effects"
+    JSON_STRING_DECODING = "json_string_decoding"
     STARTUP_PROOF_CHARACTERS = "startup_proof_characters"
     BRACE_CHARACTERS = "brace_characters"
     BRACE_ALTERNATIVES = "brace_alternatives"
@@ -783,6 +791,8 @@ class InspectionLedgerEvent(ResourceMeasurements):
     message: NotRequired[str]
     error_class: NotRequired[str]
     stage: NotRequired[str]
+    source_start_offset: NotRequired[int]
+    source_end_offset: NotRequired[int]
     parser_diagnostic: NotRequired[ParserDiagnostic]
 
 
@@ -812,6 +822,8 @@ class InspectionLedgerException(ResourceMeasurements):
     error_class: NotRequired[str]
     analyzers: NotRequired[list[str]]
     fatal: NotRequired[bool]
+    source_start_offset: NotRequired[int]
+    source_end_offset: NotRequired[int]
     scope: NotRequired[str]
     source_identity: NotRequired[str]
     parser_diagnostic: NotRequired[ParserDiagnostic]
@@ -1175,6 +1187,31 @@ def _exception_from_event(
     parser_diagnostic = sanitize_parser_diagnostic(event.get("parser_diagnostic"))
     if parser_diagnostic is not None:
         exception["parser_diagnostic"] = parser_diagnostic
+    if exception["reason_code"] is LedgerReason.JSON_QUOTE_OWNERSHIP_LIMIT:
+        # Copy only consistent numeric evidence, never an arbitrary message or
+        # source payload from graph state. Keep distinct spans distinct when
+        # the public projection groups contributing analyzers below.
+        start = event.get("source_start_offset")
+        end = event.get("source_end_offset")
+        observed = event.get("observed_characters")
+        limit = event.get("limit_characters")
+        if (
+            type(start) is int
+            and type(end) is int
+            and type(observed) is int
+            and type(limit) is int
+            and 0 <= start < end <= (1 << 63) - 1
+            and observed == end - start
+            and observed > limit > 0
+        ):
+            exception["source_start_offset"] = start
+            exception["source_end_offset"] = end
+            exception["observed_characters"] = observed
+            exception["limit_characters"] = limit
+            exception["message"] += (
+                f" Observed source character span [{start}, {end}) "
+                f"({observed} characters; limit {limit})."
+            )
     return exception