Repository navigation
fix: transaction commit failures, PageBased DELETE durability, UPDATE RETURNING and CHECK IN/BETWEEN/GLOB - #519
Open
CastleSoft wants to merge 1 commit into
Conversation
… RETURNING and CHECK IN/BETWEEN/GLOB - EndBatchUpdate (every ADO.NET/linq2db/EF Core commit): a buffered whole-file write was re-buffered during the flush, so the commit threw "Collection was modified" and the write was lost; buffered bytes now go straight to the file in their stored form. The commit's primary-key index rebuild searched the index it had just cleared, leaving columnar tables reading as empty until reopen; positions are captured before clearing. - PageBased DELETE was undone by every reopen (slot flags were never persisted); a deleted slot is now stored with length 0 and read back as deleted. Older files read as before. - UPDATE ... RETURNING re-ran the WHERE after the update and missed rows whose WHERE column the SET changed; the matched rows are read first and the SET list applied to them. - CHECK with IN / NOT IN / BETWEEN / NOT BETWEEN / GLOB accepted every row; the SQLite-semantics evaluator now reads them (also for computed operands in WHERE). - Known issues 12 (columnar scan inside a transaction misses its own UPDATE/DELETE) and 13 (PageBased rollback is a stub) recorded with cause and fix outline.
|
This was referenced Nov 16, 2026
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Summary
Fixes items 1 and 2 from the follow-up list and covers the investigation of item 3. Investigating item 3 turned up three defects that cause failed commits or data loss; all three are fixed here. Two more gaps need design-level work, so they're recorded with cause and a fix outline instead of being rushed. Every fix has a regression test that was run red on the unfixed source and green after.
1.
UPDATE … RETURNINGreturned the wrong rows — fixedThe returned rows were found by running the WHERE clause again after the update. So when the SET changed a column the WHERE tests, nothing came back. Both
SET status = 'done' WHERE status = 'open' RETURNING idandSET n = n + 10 WHERE n < 5 RETURNING nreturned 0 rows. The matched rows are now read first, and the SET list is applied to them the same way the update applies it.UpdateReturningTests.2. CHECK constraints using IN / BETWEEN / GLOB — fixed
This was the opposite of the suspicion, and worse: they accepted every row.
CHECK (status IN ('a','b'))let'z'in, andCHECK (n BETWEEN 1 AND 10)let50in. The SQLite-semantics evaluator now handles IN, NOT IN, BETWEEN, NOT BETWEEN and GLOB:*,?and[…]character classes.length(name) BETWEEN 3 AND 6.CheckConstraintOperatorTests.3. Reads inside a transaction — investigated
The transaction under investigation is
BeginBatchUpdate()/EndBatchUpdate(), which every ADO.NETDbTransactionopens, so linq2db and EF Core transactions too.Storage.WriteBytessaw the transaction still open and buffered the write again into the list being looped over. The write was lost; had the loop not thrown, the already-encrypted bytes would have been encrypted a second time.PageBasedEngine.Rollback()is a stub ("we don't support rollback in this POC"). Fixing it needs per-page undo images or shadow paging, which is a change to the engine's durability path.BatchUpdateTransactionTestspins the fixed rows.Tests
The 24 new test cases are in
UpdateReturningTests,CheckConstraintOperatorTestsandBatchUpdateTransactionTests. Twenty fail on the unfixed source. The four that already pass there are expected:RETURNING *where the SET doesn't touch a WHERE column, which already worked.ReopenRoundTripMatrixTestsandFormatCompatPolicyTestsSharpCoreDB.CI.slnfbuilds with 0 errors. The documentation consistency tests pass.Performance
Write-path gate: GATE PASSED, with nothing slower than baseline × 1.50. Three metrics landed in the watch band:
The run wasn't on a quieted machine: a permission rule blocked the quiet script's temp-directory cleanup. The only hot-path change in this PR is one flag test per slot in
PageManager.WritePage. The columnar READ and DELETE paths aren't touched.