Skip to content

fix(parser): read a backquoted substitution after a quoted segment - #2661

Merged
chaliy merged 8 commits into
everruns:mainfrom
xmakro:fix/backtick-after-quote
Oct 11, 2026
Merged

chaliy merged 8 commits into
everruns:mainfrom
xmakro:fix/backtick-after-quote

Conversation

@xmakro

@xmakro xmakro commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

What changed

Backquoted substitutions immediately after single- or double-quoted text now form one word and execute correctly. Their unquoted output still splits and globs, matching Bash.

Words beginning with quoted text also preserve literal glob characters when an adjacent substitution or variable expands to nothing. Quoted expansion output stays literal; unquoted expansion output remains active in patterns. Empty quoted fields and escaped internal marker bytes retain their meaning.

Why

The quote-continuation reader treated an opening backquote as ordinary text. Reuse the existing backquote reader so delimiter handling, runtime syntax errors, analysis visibility, dispatch hooks, and execution limits follow the established path.

Review also exposed a shared quote/glob bug affecting both backticks and $(). Preserve the existing per-part quote metadata through glob escaping and use the same per-part decision for command-substitution byte charges. Cache quote-metadata presence once per word to avoid repeated scans on long mixed words.

Before / After

printf '[%s]\n' 'x'`printf '1 2'`; echo tail

Before: quoted-adjacent backticks were misread; the contributor's spec cases and focused regressions failed.
After, identical to Bash:

[x1]
[2]
tail

With virtual files /tmp/p-a and /tmp/p-b:

printf '[%s]\n' '/tmp/p-*'$(printf '')

Before: [/tmp/p-a] and [/tmp/p-b].
After: [/tmp/p-*]. The same guarantee holds for backticks and "$prefix"$empty.

Risk

  • Medium: shared quote/glob encoding and expansion paths are affected.
  • No new dependencies, host I/O, unsafe code, workflows, or permissions. Existing sandbox dispatch and budgets remain in force.

Validation

  • Nine focused integration tests, including 21 Bash comparisons (three compound-array cases), malformed delimiters, runtime-only body errors, single execution, splitting/globbing, static-analysis visibility, dispatch vetoes, and shared execution budgets.
  • 188 parser tests, all four spec-suite tests, 21 script-analysis tests, and escaped-output byte-charge tests pass.
  • All-target/all-feature Clippy, repository script tests, knowledge/doc/workflow/capability checks, and dependency vetting pass.
  • Local just pre-pr reaches the existing macOS stack overflow in blackbox_security_tests::finding_nested_cmd_subst_stack_overflow::depth_50_is_bounded; reproduced again on a clean source snapshot of current main c6fd211920d534ba0e4ca2c642e65747879efbbc. An earlier workspace library run also hit the unchanged macOS CPython alignment test; it was reproduced on its then-current clean main be8ec33c. Full local gates are not green.
  • All 3,741 Bashkit library tests, seven arithmetic-budget regressions, and 19 single-threaded security fail-point tests pass. Final CLI smoke output matches Bash.
  • just bench: 96 cases, zero errors, 100% Bash output match. Parallel-execution Criterion smoke and measured runs complete on the conflict-resolution revision. No performance claim inferred from a busy local machine.
  • Conflict resolution integrates main c6fd2119: per-part quoting and byte-charge decisions now use the budgeted buffer introduced by the pending-array security fix. Its allocation guards, borrowed scalar expansion, capacity leases, boxed future, and consumed-output lease release are preserved. All 25 execution-budget, seven prompt-resource, seven arithmetic-resource, and 19 fail-point tests pass.
  • Final CI: all 42 current checks pass at a8bcbe730c968b5c45c660fb4a0518ed7d707d81, including the required aggregate Check, Linux tests, strict Bash/sed/grep parity, CPython native/interpreted tests, filesystem and terminal tests, fail-point tests, and property-based security tests. GitHub reports the PR clean and mergeable.
  • CodeQL limitation: scans succeeded for the previous PR head 1ec7edcd and current main c6fd2119. No fresh combined scan started for a8bcbe73; GitHub default setup excludes fork PRs (GitHub documentation). The exact conflict-resolution diff was reviewed for preserving per-part escaping, allocation-before-growth checks, shared budget leases, and boxed recursive futures. No workflows or security configuration were changed.

Checklist

  • Tests added or updated, with failures reproduced before fixes
  • Backward compatibility considered: Bash-compatible behavior; public API unchanged
  • Parser knowledge and public compatibility guide updated
  • Security review performed against parser, analysis, dispatch, and budget contracts

xmakro and others added 7 commits October 9, 2026 09:35
`'b='`cmd`` and `"b="`cmd`` lexed the backquote as a literal word
character, so the substitution swallowed the script up to the next
backquote: `echo 'b='`echo X`` failed with "unterminated backtick
substitution". A quoted segment followed by `$(cmd)` already worked;
the backquoted form is now read the same way.

@chaliy chaliy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved for shipment at 1ec7edcd38de13b7307d013d507cf8019fda8e26.

The contribution reuses the existing backquote reader. Follow-up fixes preserve quoted glob literals and empty fields, keep unquoted expansion output active, and charge escaped substitution bytes consistently. Quote metadata is checked once per word rather than repeatedly scanning long mixed words.

No introduced sandbox escape, permission bypass, unsafe code, dependency, or host-I/O change found. Tests verify malformed delimiters, runtime-only body errors, analysis visibility, dynamic command detection, dispatch vetoes, and shared execution budgets.

All 48 current checks pass, including required Check and Rust CodeQL; no new CodeQL alerts. Local validation includes 3,737 Bashkit library tests, 188 parser tests, 21 analysis tests, nine focused regressions with 18 Bash comparisons, seven arithmetic-budget tests, 19 fail-point tests, and 96 benchmark cases with zero errors and 100% Bash output match.

The full local macOS gates retain two upstream failures: nested-substitution stack overflow and CPython static alignment. Both reproduce on clean main and are documented in the PR body. GitHub CI and merge status are green/clean.

@chaliy chaliy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the conflict resolution against main c6fd211. The pending-array allocation guards, capacity leases, consumed-substitution lease release, and boxed recursive future are preserved alongside per-part quote escaping and matching byte charges. No security or design-alignment blockers found in this resolution.

All 42 current CI checks pass, including required Check and security suites. Local quoting tests include 21 Bash comparisons and compound-array cases; all 25 execution-budget regressions pass. CLI smoke and 96 benchmark cases match Bash.

Limitations recorded in the PR body: the local macOS stack-overflow gate reproduces on clean current main; no fresh combined CodeQL scan started for this fork revision (previous PR head and current main scans passed). Approve for shipment.

@chaliy
chaliy merged commit 94e24ae into everruns:main Oct 11, 2026
42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants