Skip to content

feat: ship the literal scanning header for derived grammars - #163

Merged
exKAZUu merged 1 commit into
mainfrom
feat/ship-literal-header
Nov 2, 2026
Merged

exKAZUu merged 1 commit into
mainfrom
feat/ship-literal-header

Conversation

@exKAZUu

@exKAZUu exKAZUu commented Nov 2, 2026

Copy link
Copy Markdown
Member

Why

@willbooster/tree-sitter-typescript builds its grammar from this package's grammar.js, but its external scanner keeps its own copies of the template-character and JSX-text scanning. JavaScript 21's grammar adds an _template_escape_sequence external token that validates untagged template escapes in src/literal.h; shipping that header lets the TypeScript scanner include it instead of copying it again, so later fixes to literal scanning reach TypeScript through a dependency update.

Requirements

  • required: TypeScript derives from the current JavaScript grammar (and scanner, where shared), preferably via the package dependency so future JavaScript fixes propagate.
  • chosen: ship only src/literal.h, the self-contained part of the scanner that TypeScript can include unchanged; the rest of src/scanner.c is interleaved with TypeScript-specific logic in TypeScript's own scanner.

Customer Summary

The npm package now also contains src/literal.h. Parsing is unchanged.

Technical Summary

  • package.json: adds src/literal.h to files.
  • README.md: notes that extending grammars' scanners can include it.

Testing

bun run verify; npm pack --dry-run lists src/literal.h. The TypeScript package's existing check that its vendored JavaScript files match the installed package will cover the header once it is vendored there.

Co-authored-by: WillBooster (Claude Code) <agent@willbooster.com>
@exKAZUu
exKAZUu merged commit 78d5bbe into main Nov 2, 2026
10 checks passed
@exKAZUu
exKAZUu deleted the feat/ship-literal-header branch November 2, 2026 11:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant