Skip to content

Fix delayed query cleanup during file metadata scans - #18679

Merged
JackieTien97 merged 1 commit into
apache:masterfrom
JackieTien97:codex/fix-series-scan-cancellation
Sep 20, 2026
Merged

JackieTien97 merged 1 commit into
apache:masterfrom
JackieTien97:codex/fix-series-scan-cancellation

Conversation

@JackieTien97

Copy link
Copy Markdown
Contributor

Description

A cancelled or timed-out fragment can keep loading overlapping file metadata inside a single operator call. Its Driver retains its execution lock, so the fragment cleanup callback waits for Driver destruction and delays other callbacks sharing the notification executor.

Check the fragment's terminal state at file-scan entry and immediately before and after loading each sequential or unsequential file's metadata. The scan exits on the Driver thread and preserves the original failure cause. The check is shared by non-aligned and aligned scans, skips compaction contexts without a state machine, and uses matching English/Chinese message constants.

This provides cooperative cancellation between metadata reads. An in-flight I/O operation or monitor acquisition must still return before cancellation is observed; the reader locking and notification executor lifecycle are unchanged.

Validation

  • Clean DataNode reactor build and 59 targeted tests passed, including 24 new parameterized cases for non-aligned, aligned tree, and aligned table scans. Coverage includes terminal states, ascending/descending seq/unseq scans, cancellation during the final or missing metadata read, cached metadata, active and compaction contexts, and concurrent Driver close followed by the memory cleanup callbacks.
  • Full reactor English and Chinese test compilation passed (50 modules each); the compiled Chinese message and its use in the scanner bytecode were also verified.
  • Checkstyle, Spotless, diff whitespace checks, and message-key/hash/placeholder parity checks passed.
mvn clean test -pl iotdb-core/datanode -am \
  -Dtest=SeriesScanUtilCancellationTest,DataDriverTest,FragmentInstanceExecutionTest,SeriesScanLimitOffsetPushDownTest,AlignedSeriesScanLimitOffsetPushDownTest,SeriesScanPredicatePushDownTest,AlignedSeriesScanPredicatePushDownTest \
  -Dsurefire.failIfNoSpecifiedTests=false
mvn test-compile -DskipTests
mvn test-compile -DskipTests -P with-zh-locale
  • Self-reviewed, including concurrent scan and close behavior.
  • Added regression tests and comments explaining the cancellation boundary.
  • Included the Apache license header in the new test file.

@JackieTien97
JackieTien97 merged commit d2fcb53 into apache:master Sep 20, 2026
39 checks passed
@JackieTien97
JackieTien97 deleted the codex/fix-series-scan-cancellation branch September 20, 2026 09:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant