gh-153569: consolidate tokenizer input, layout, and diagnostics - #157055

Merged

pablogsal merged 6 commits into

python:mainfrom

pablogsal:gh-153569-tokenizer-input-diagnostics

Sep 24, 2026

Commits on Sep 24, 2026

gh-153569: move tokenizer input state and relocation into the reader
pablogsal
committed
9ad9e67
gh-153569: borrow diagnostic lines through the source API
pablogsal
committed
11e1334
gh-153569: group indentation and logical-line state
pablogsal
committed
2311ddb
gh-153569: remove unused tokenizer cursor and source lookup APIs
pablogsal
committed
68f376e
gh-153569: report tokenizer diagnostics without rewinding the scanner
pablogsal
committed
28e15b4
gh-153569: use tokenizer views and remove obsolete API remnants
pablogsal
committed
9016f1d