From 3733d5c53611e10f91b0c5800e769e13f39b8d41 Mon Sep 17 00:00:00 2001 From: ideoma <2159629+ideoma@users.noreply.github.com> Date: Fri, 23 Oct 2026 16:36:16 +0100 Subject: [PATCH 1/3] docs(wal): document adaptive WAL apply block size --- documentation/concepts/out-of-order-data.md | 5 +++ documentation/concepts/write-ahead-log.md | 6 +++ documentation/configuration/wal.md | 46 ++++++++++++++++++--- 3 files changed, 51 insertions(+), 6 deletions(-) diff --git a/documentation/concepts/out-of-order-data.md b/documentation/concepts/out-of-order-data.md index c8458c280..23c4cf97c 100644 --- a/documentation/concepts/out-of-order-data.md +++ b/documentation/concepts/out-of-order-data.md @@ -153,6 +153,11 @@ filesystem overhead. The target remains 30 to 80 million rows per partition. See [Choosing a partition interval](/docs/concepts/partitions/#choosing-a-partition-interval). +### WAL apply block size + +Larger [WAL apply blocks](/docs/configuration/wal/#adaptive-wal-apply-blocks) +reduce the number of O3 merges. + ### O3 memory page size The `cairo.o3.column.memory.size` configuration controls how much memory the diff --git a/documentation/concepts/write-ahead-log.md b/documentation/concepts/write-ahead-log.md index 10b0e1302..99cfe36a1 100644 --- a/documentation/concepts/write-ahead-log.md +++ b/documentation/concepts/write-ahead-log.md @@ -105,6 +105,11 @@ When data is written to a WAL table: This decouples the commit (fast) from storage application (background), enabling high write throughput. +WAL apply sorts and commits many WAL transactions together as one block; +larger blocks reduce out-of-order merges but use more memory per block. +[Adaptive blocks](/docs/configuration/wal/#adaptive-wal-apply-blocks) are enabled +by default and grow within a memory budget. + Date: Fri, 23 Oct 2026 16:53:39 +0100 Subject: [PATCH 2/3] docs(wal): clarify block size uses bytes --- documentation/configuration/wal.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/documentation/configuration/wal.md b/documentation/configuration/wal.md index a84217af7..f06ec45bb 100644 --- a/documentation/configuration/wal.md +++ b/documentation/configuration/wal.md @@ -12,10 +12,10 @@ To cap the native memory a single WAL apply batch may allocate, see ## Adaptive WAL apply blocks -With adaptive blocks enabled, the row limit starts at -`cairo.wal.apply.block.size` worth of rows and doubles whenever the row limit -cuts a block short, while the block fits its memory budget. The grown limit -resets when the table writer reopens. The look-ahead transaction count starts +With adaptive blocks enabled, QuestDB derives the initial row limit from the +byte size configured by `cairo.wal.apply.block.size`. This row limit doubles +whenever it cuts a block short, while the block fits its memory budget. The +grown limit resets when the table writer reopens. The look-ahead transaction count starts at `cairo.wal.apply.look.ahead.txn.count` and adapts to one block's worth of transactions. From 3def0010b9236a2682a73f43b7ac8aedcf4486ae Mon Sep 17 00:00:00 2001 From: ideoma <2159629+ideoma@users.noreply.github.com> Date: Fri, 23 Oct 2026 16:54:56 +0100 Subject: [PATCH 3/3] docs(wal): order block settings by explanation flow --- documentation/configuration/wal.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/documentation/configuration/wal.md b/documentation/configuration/wal.md index f06ec45bb..a4aee7cf0 100644 --- a/documentation/configuration/wal.md +++ b/documentation/configuration/wal.md @@ -22,13 +22,13 @@ transactions. When adaptive blocks are disabled, blocks are sized by `cairo.wal.max.lag.size` and `cairo.wal.squash.uncommitted.rows.multiplier`. -## cairo.wal.apply.block.max.rows +## cairo.wal.apply.block.size -- **Default**: `100000000` +- **Default**: `75M` - **Reloadable**: no -Hard row cap for both the starting block size and its growth, from 1 to 2,000,000,000. -Has no effect when adaptive blocks are disabled. +Starting data size in bytes for an adaptive WAL apply block. +Set to `0` to disable adaptive blocks. ## cairo.wal.apply.block.memory.percent @@ -39,13 +39,13 @@ Percentage of the RSS limit (`ram.usage.limit.bytes` / `ram.usage.limit.percent` as the budget for growth beyond the starting block size. With no RSS limit, or with this set to `0`, the row limit does not grow. -## cairo.wal.apply.block.size +## cairo.wal.apply.block.max.rows -- **Default**: `75M` +- **Default**: `100000000` - **Reloadable**: no -Starting data size in bytes for an adaptive WAL apply block. -Set to `0` to disable adaptive blocks. +Hard row cap for both the starting block size and its growth, from 1 to 2,000,000,000. +Has no effect when adaptive blocks are disabled. ## cairo.wal.apply.parallel.sql.enabled