From 541fc4c19540eafc446858480fb803807400e6ac Mon Sep 17 00:00:00 2001 From: Nick Woolmer <29717167+nwoolmer@users.noreply.github.com> Date: Thu, 22 Oct 2026 18:53:19 +0100 Subject: [PATCH 1/2] docs(sql): document percentile_cont and percentile_disc Document the exact percentile functions added in questdb/questdb#6680: percentile_cont and percentile_disc, their quantile_cont and quantile_disc aliases, the DOUBLE[] multi-percentile form, and the whole-partition window functions. Add the array form of approx_percentile. Co-Authored-By: Claude Opus 5.5 (1M context) --- documentation/query/functions/aggregation.md | 227 +++++++++++++++++- .../functions/window-functions/overview.md | 1 + .../functions/window-functions/reference.md | 76 +++++- 3 files changed, 298 insertions(+), 6 deletions(-) diff --git a/documentation/query/functions/aggregation.md b/documentation/query/functions/aggregation.md index 31baf203e0..4944741bc6 100644 --- a/documentation/query/functions/aggregation.md +++ b/documentation/query/functions/aggregation.md @@ -41,6 +41,8 @@ calculations. Functions are organized by category below. | [kurtosis / kurtosis_samp](#kurtosis--kurtosis_samp) | Sample excess kurtosis | | [kurtosis_pop](#kurtosis_pop) | Population excess kurtosis | | [mode](#mode) | Most frequent value | +| [percentile_cont / quantile_cont](#percentile_cont--quantile_cont) | Exact percentile, interpolated between values | +| [percentile_disc / quantile_disc](#percentile_disc--quantile_disc) | Exact percentile, returns an input value | | [skewness / skewness_samp](#skewness--skewness_samp) | Sample skewness | | [skewness_pop](#skewness_pop) | Population skewness | | [stddev / stddev_samp](#stddev--stddev_samp) | Sample standard deviation | @@ -264,12 +266,25 @@ GROUP BY symbol; `approx_percentile(value, percentile, precision)` calculates the approximate value for the given non-negative column and percentile using the -[HdrHistogram](http://hdrhistogram.org/) algorithm. +[HdrHistogram](http://hdrhistogram.org/) algorithm. Pass an array of +percentiles to read several of them from the same histogram. + +```questdb-sql +approx_percentile(value, percentile [, precision]) +approx_percentile(value, ARRAY[percentile1, percentile2, ...] [, precision]) +``` + +The histogram uses a bounded amount of memory per group regardless of how many +rows the group holds. When you need exact results, or the input contains +negative values, use [percentile_cont](#percentile_cont--quantile_cont) or +[percentile_disc](#percentile_disc--quantile_disc) instead. #### Parameters - `value` is any numeric non-negative value. -- `percentile` is a `double` value between 0.0 and 1.0, inclusive. +- `percentile` is a `double` value between 0.0 and 1.0, inclusive, or a + one-dimensional `double[]` of such values. It must be a constant or a bind + variable. - `precision` is an optional `int` value between 0 and 5, inclusive. This is the number of significant decimal digits to which the histogram will maintain value resolution and separation. For example, when the input column contains @@ -284,7 +299,8 @@ value for the given non-negative column and percentile using the #### Return value -Return value type is `double`. +Return value type is `double`, or `double[]` when `percentile` is an array. The +array holds the results in the same order as the requested percentiles. #### Examples @@ -297,9 +313,21 @@ WHERE timestamp IN '$today'; | :---------------- | | 211.24 | +```questdb-sql demo title="Several approximate quantity percentiles per FX pair" +SELECT + symbol, + approx_percentile(quantity, ARRAY[0.5, 0.9, 0.99], 3) AS qty_pcts +FROM fx_trades +WHERE timestamp IN '$today' +ORDER BY symbol; +``` + #### See also - [approx_median](#approx_median) - Shorthand for 50th percentile +- [percentile_cont](#percentile_cont--quantile_cont) and + [percentile_disc](#percentile_disc--quantile_disc) calculate exact + percentiles ## arg_max @@ -1643,6 +1671,199 @@ SELECT nsum(a) FROM (SELECT rnd_double() a FROM long_sequence(100)); ``` +## percentile_cont / quantile_cont + +`percentile_cont(value, percentile)` calculates the exact continuous percentile +of a group, interpolating linearly between the two input values closest to the +requested position. Use it for latency SLOs, fill-size distributions, or price +bands where an interpolated value is acceptable and an exact answer is needed. +Pass an array of percentiles to compute several of them over the same values in +one pass. `quantile_cont` is an alias. + +The function follows the SQL standard and PostgreSQL definition. For `n` +non-NULL values sorted in ascending order, the result sits at the 0-based +position `percentile * (n - 1)`. When that position falls between two values, +the result is interpolated between them. + +```questdb-sql +percentile_cont(value, percentile) +percentile_cont(value, ARRAY[percentile1, percentile2, ...]) +``` + +Unlike [approx_percentile](#approx_percentile), `percentile_cont` keeps every +non-NULL value of a group in memory, so its memory use grows with the size of +the group. It accepts negative values and supports parallel execution. + +#### Parameters + +- `value` (numeric): the values to calculate the percentile of. NULL values are + ignored. +- `percentile` (`double` or `double[]`): a constant or bind variable between + 0.0 and 1.0, inclusive. A `double[]` must be one-dimensional and returns one + result per element. A value that varies per row is rejected. + +#### Return value + +Return value type is `double`, or `double[]` when `percentile` is an array. The +array holds the results in the same order as the requested percentiles. + +Returns `NULL` when the group has no non-NULL values. + +#### Examples + +Given a `fills` table with execution latency per trading desk: + +| ts | desk | latency_us | +| :-------------------------- | :--- | :--------- | +| 2026-10-01T09:00:00.000000Z | eq | 120 | +| 2026-10-01T09:00:01.000000Z | eq | 150 | +| 2026-10-01T09:00:02.000000Z | eq | 180 | +| 2026-10-01T09:00:03.000000Z | eq | 240 | +| 2026-10-01T09:00:04.000000Z | eq | 900 | +| 2026-10-01T09:00:05.000000Z | fi | 300 | +| 2026-10-01T09:00:06.000000Z | fi | 310 | +| 2026-10-01T09:00:07.000000Z | fi | 2500 | +| 2026-10-01T09:00:08.000000Z | fi | NULL | + +```questdb-sql title="Latency percentiles per desk" +SELECT + desk, + percentile_cont(latency_us, 0.5) AS p50, + percentile_cont(latency_us, ARRAY[0.5, 0.9, 0.99]) AS pcts +FROM fills +ORDER BY desk; +``` + +| desk | p50 | pcts | +| :--- | :---- | :--------------------- | +| eq | 180.0 | [180.0,636.0,873.6] | +| fi | 310.0 | [310.0,2062.0,2456.2] | + +For the `eq` desk, the 90th percentile sits at position `0.9 * 4 = 3.6`, so the +result is 60% of the way from 240 to 900, which is 636. The NULL latency on the +`fi` desk is ignored. + +```questdb-sql demo title="Quantity percentiles per FX pair in 15-minute buckets" +SELECT + timestamp, + symbol, + percentile_cont(quantity, ARRAY[0.5, 0.9, 0.99]) AS qty_pcts +FROM fx_trades +WHERE timestamp IN '$now-1h..$now' +SAMPLE BY 15m; +``` + +Access individual elements of the result with array indexing, which is 1-based: + +```questdb-sql demo title="Read individual percentiles from the array" +SELECT symbol, pcts[1] AS p50, pcts[3] AS p99 +FROM ( + SELECT symbol, percentile_cont(quantity, ARRAY[0.5, 0.9, 0.99]) AS pcts + FROM fx_trades + WHERE timestamp IN '$today' +); +``` + +:::note + +- `SAMPLE BY` with `FILL(LINEAR)` is not supported. The array form also rejects + a constant fill value such as `FILL(0)`. `FILL(NULL)` and `FILL(PREV)` work + with both forms. +- A long-running sort or selection on a very large group does not stop + immediately on a query timeout or cancellation. It finishes the current + group first. + +::: + +#### See also + +- [percentile_disc](#percentile_disc--quantile_disc) returns an actual value + from the input instead of an interpolated one +- [approx_percentile](#approx_percentile) uses a fixed amount of memory per + group, at the cost of precision +- [percentile_cont window function](/docs/query/functions/window-functions/reference/#percentile) + attaches the percentile of a partition to every row + +## percentile_disc / quantile_disc + +`percentile_disc(value, percentile)` calculates the exact discrete percentile +of a group. The result is always one of the input values, which makes it the +right choice when an interpolated value would be meaningless, such as a price +on a tick grid, an integer count, or a value that must exist in the data. +`quantile_disc` is an alias. + +The function follows the SQL standard and PostgreSQL definition: it returns the +first value, in ascending order, whose cumulative distribution reaches the +percentile. For `n` non-NULL values, that is the `ceil(percentile * n)`-th +smallest value, or the smallest value when `percentile` is 0. + +```questdb-sql +percentile_disc(value, percentile) +percentile_disc(value, ARRAY[percentile1, percentile2, ...]) +``` + +`percentile_disc` shares the parameter rules, memory use, and limitations of +[percentile_cont](#percentile_cont--quantile_cont). + +#### Parameters + +- `value` (numeric): the values to calculate the percentile of. NULL values are + ignored. +- `percentile` (`double` or `double[]`): a constant or bind variable between + 0.0 and 1.0, inclusive. A `double[]` must be one-dimensional and returns one + result per element. + +#### Return value + +- `long` for integer input (`byte`, `short`, `int`, `long`), so the result + keeps the exact input value +- `double` for floating-point input +- `double[]` when `percentile` is an array, for any input type + +Returns `NULL` when the group has no non-NULL values. + +#### Examples + +Using the `fills` table from the [percentile_cont](#percentile_cont--quantile_cont) +examples: + +```questdb-sql title="Discrete latency percentiles per desk" +SELECT + desk, + percentile_disc(latency_us, 0.5) AS p50, + percentile_disc(latency_us, ARRAY[0.5, 0.9, 0.99]) AS pcts +FROM fills +ORDER BY desk; +``` + +| desk | p50 | pcts | +| :--- | :-- | :-------------------- | +| eq | 180 | [180.0,900.0,900.0] | +| fi | 310 | [310.0,2500.0,2500.0] | + +`p50` is a `long` because `latency_us` is a `long` column. The `eq` desk has +five values, so the 90th percentile is the `ceil(0.9 * 5) = 5`th smallest value, +900. + +```questdb-sql demo title="Median trade price per FX pair, discrete and interpolated" +SELECT + symbol, + percentile_disc(price, 0.5) AS median_traded, + percentile_cont(price, 0.5) AS median_interpolated +FROM fx_trades +WHERE timestamp IN '$today' +ORDER BY symbol; +``` + +#### See also + +- [percentile_cont](#percentile_cont--quantile_cont) interpolates between + input values +- [approx_percentile](#approx_percentile) uses a fixed amount of memory per + group, at the cost of precision +- [percentile_disc window function](/docs/query/functions/window-functions/reference/#percentile) + attaches the percentile of a partition to every row + ## skewness / skewness_samp `skewness_samp(value)` - Calculates the sample skewness of a set of values, diff --git a/documentation/query/functions/window-functions/overview.md b/documentation/query/functions/window-functions/overview.md index 1b4fcb98dc..ddebfc2c25 100644 --- a/documentation/query/functions/window-functions/overview.md +++ b/documentation/query/functions/window-functions/overview.md @@ -63,6 +63,7 @@ Arithmetic operations on window functions (e.g., `sum(...) OVER (...) / sum(...) | [`nth_value()`](reference.md#nth_value) | N-th value in window | Yes | | [`ntile()`](reference.md#ntile) | Bucket number from 1 to N | No | | [`percent_rank()`](reference.md#percent_rank) | Relative rank (0 to 1) | No | +| [`percentile_cont()` / `percentile_disc()`](reference.md#percentile) | Exact percentile of the whole partition | No (frame clause rejected) | | [`rank()`](reference.md#rank) | Rank with gaps for ties | No | | [`row_number()`](reference.md#row_number) | Sequential row number | No | | [`sdt()`](reference.md#sdt) | SUBSAMPLE keep flag: Swinging Door Trending, error-bounded | No (frame clause rejected) | diff --git a/documentation/query/functions/window-functions/reference.md b/documentation/query/functions/window-functions/reference.md index 0007a889ad..46f9f78189 100644 --- a/documentation/query/functions/window-functions/reference.md +++ b/documentation/query/functions/window-functions/reference.md @@ -1,15 +1,15 @@ --- title: Window Functions Reference sidebar_label: Function Reference -description: Complete reference for all window functions in QuestDB including avg, sum, ksum, count, stddev, variance, covariance, correlation, rank, dense_rank, percent_rank, ntile, cume_dist, row_number, lag, lead, nth_value, EMA, VWEMA, the SUBSAMPLE keep-flag functions lttb, m4, minmax, uniform, cadence, sdt, and more. -keywords: [subsample, lttb, m4, minmax, uniform, cadence, sdt, downsampling, window functions, avg, sum, ksum, count, stddev, stddev_pop, stddev_samp, var_pop, var_samp, variance, covar_pop, covar_samp, corr, correlation, rank, dense_rank, percent_rank, ntile, cume_dist, row_number, lag, lead, first_value, last_value, nth_value, min, max, ema, vwema, exponential moving average] +description: Complete reference for all window functions in QuestDB including avg, sum, ksum, count, stddev, variance, covariance, correlation, rank, dense_rank, percent_rank, ntile, cume_dist, row_number, lag, lead, nth_value, percentile_cont, percentile_disc, EMA, VWEMA, the SUBSAMPLE keep-flag functions lttb, m4, minmax, uniform, cadence, sdt, and more. +keywords: [subsample, lttb, m4, minmax, uniform, cadence, sdt, downsampling, window functions, avg, sum, ksum, count, stddev, stddev_pop, stddev_samp, var_pop, var_samp, variance, covar_pop, covar_samp, corr, correlation, rank, dense_rank, percent_rank, ntile, cume_dist, row_number, lag, lead, first_value, last_value, nth_value, percentile_cont, percentile_disc, quantile_cont, quantile_disc, percentile, median, min, max, ema, vwema, exponential moving average] --- This page provides detailed documentation for each window function. For an introduction to window functions and how they work, see the [Overview](overview.md). For syntax details on the `OVER` clause, see [OVER Clause Syntax](syntax.md). ## Aggregate window functions -These functions respect the frame clause and calculate values over the specified window frame. +These functions respect the frame clause and calculate values over the specified window frame. The exception is [percentile_cont() / percentile_disc()](#percentile), which always use the whole partition. ### avg() / EMA / VWEMA {#avg} @@ -634,6 +634,76 @@ With a 3-row frame, `nth_3` always equals the current row's `price` because it i --- +### percentile_cont() / percentile_disc() {#percentile} + +Calculates the exact percentile of a whole partition and attaches it to every row of that partition. This lets you compare each row with its group's distribution without a self-join, for example to flag trades above the 99th percentile of size for their symbol. `quantile_cont()` and `quantile_disc()` are aliases. + +**Syntax:** +```questdb-sql +percentile_cont(value, percentile) OVER ([PARTITION BY columnName]) +percentile_disc(value, percentile) OVER ([PARTITION BY columnName]) +percentile_cont(value, ARRAY[p1, p2, ...]) OVER ([PARTITION BY columnName]) +percentile_disc(value, ARRAY[p1, p2, ...]) OVER ([PARTITION BY columnName]) +``` + +**Arguments:** +- `value`: Numeric column. NULL values are ignored +- `percentile`: Constant `double` between 0.0 and 1.0, inclusive, or a one-dimensional `double[]` constant of such values + +**Return value:** +- `percentile_cont()`: `double`, interpolated between the two input values closest to the requested position +- `percentile_disc()`: an actual input value. `long` for integer input, `double` for floating-point input +- `double[]` for either function when `percentile` is an array, with results in the order requested +- `NULL` when the partition has no non-NULL values + +**Description:** + +These are the window forms of the [percentile_cont](/docs/query/functions/aggregation/#percentile_cont--quantile_cont) and [percentile_disc](/docs/query/functions/aggregation/#percentile_disc--quantile_disc) aggregate functions, and compute the same values. Every row in a partition gets the same result. + +**Behavior:** +- Only whole-partition windows are supported. `ORDER BY` and frame clauses (`ROWS`, `RANGE`) inside `OVER` are rejected, including `ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING` +- `percentile` must be a constant. Bind variables are rejected +- The function buffers every value of the partition, so memory use grows with partition size +- Not supported in [live views](/docs/query/sql/create-live-view/) + +**Examples:** +```questdb-sql title="Median quantity per symbol next to each trade" demo +SELECT + timestamp, + symbol, + quantity, + percentile_cont(quantity, 0.5) OVER (PARTITION BY symbol) AS median_qty +FROM fx_trades +WHERE timestamp IN '$now-1h..$now'; +``` + +```questdb-sql title="Trades at or above the 99th percentile of size for their symbol" demo +SELECT timestamp, symbol, ecn, quantity, p99_qty +FROM ( + SELECT + timestamp, + symbol, + ecn, + quantity, + percentile_disc(quantity, 0.99) OVER (PARTITION BY symbol) AS p99_qty + FROM fx_trades + WHERE timestamp IN '$today' +) +WHERE quantity >= p99_qty; +``` + +```questdb-sql title="Interquartile range per symbol" demo +SELECT + timestamp, + symbol, + price, + percentile_cont(price, ARRAY[0.25, 0.75]) OVER (PARTITION BY symbol) AS iqr +FROM fx_trades +WHERE timestamp IN '$now-1h..$now'; +``` + +--- + ### stddev_pop() / stddev_samp() / stddev() {#stddev} Calculates the standard deviation of values over the window frame. `stddev_pop()` computes population standard deviation (divides by N), `stddev_samp()` computes sample standard deviation (divides by N-1). `stddev()` is an alias for `stddev_samp()`. From b48196da5a997987142253281d96b9796d7a1591 Mon Sep 17 00:00:00 2001 From: Nick Woolmer <29717167+nwoolmer@users.noreply.github.com> Date: Thu, 22 Oct 2026 18:53:19 +0100 Subject: [PATCH 2/2] docs(cookbook): build query time histogram from a percentile array Replace ten approx_percentile calls and ten UNION ALL branches with one percentile_disc call over an array of percentiles, expanded with UNNEST WITH ORDINALITY. Add a non-cumulative variant using lag(). Co-Authored-By: Claude Opus 5.5 (1M context) --- .../operations/query-times-histogram.md | 165 ++++++++---------- 1 file changed, 77 insertions(+), 88 deletions(-) diff --git a/documentation/cookbook/operations/query-times-histogram.md b/documentation/cookbook/operations/query-times-histogram.md index 0ecffa0a4c..39351cae8a 100644 --- a/documentation/cookbook/operations/query-times-histogram.md +++ b/documentation/cookbook/operations/query-times-histogram.md @@ -11,105 +11,94 @@ Create a histogram of query execution times using the `_query_trace` system tabl [Query tracing](/docs/concepts/deep-dive/query-tracing/) needs to be enabled for the `_query_trace` table to be populated. ::: -## Solution: Percentile-based histogram - -We can create a subquery that first calculates the percentiles for each bucket, in this case at 10% intervals. Then on a second query we can do a `UNION` of 10 subqueries where each is doing a `CROSS JOIN` against the calculated percentiles and finding how many queries are below the threshold for the bucket. - -Note in this case the histogram is cumulative, and each bucket includes the results from the smaller buckets as well. If we prefer non-cumulative, the condition would change from less than to `BETWEEN`. - -```sql -WITH quantiles AS ( - SELECT - approx_percentile(execution_micros, 0.10, 5) AS p10, - approx_percentile(execution_micros, 0.20, 5) AS p20, - approx_percentile(execution_micros, 0.30, 5) AS p30, - approx_percentile(execution_micros, 0.40, 5) AS p40, - approx_percentile(execution_micros, 0.50, 5) AS p50, - approx_percentile(execution_micros, 0.60, 5) AS p60, - approx_percentile(execution_micros, 0.70, 5) AS p70, - approx_percentile(execution_micros, 0.80, 5) AS p80, - approx_percentile(execution_micros, 0.90, 5) AS p90, - approx_percentile(execution_micros, 1.0, 5) AS p100 - FROM _query_trace -), cumulative_hist AS ( -SELECT '10' AS bucket, p10 as micros_threshold, count(*) AS frequency -FROM _query_trace CROSS JOIN quantiles -WHERE execution_micros < p10 - -UNION ALL - -SELECT '20', p20 as micros_threshold, count(*) -FROM _query_trace CROSS JOIN quantiles -WHERE execution_micros < p20 - -UNION ALL - -SELECT '30', p30 as micros_threshold, count(*) -FROM _query_trace CROSS JOIN quantiles -WHERE execution_micros < p30 - -UNION ALL - -SELECT '40', p40 as micros_threshold, count(*) -FROM _query_trace CROSS JOIN quantiles -WHERE execution_micros < p40 +## Problem -UNION ALL +You want to see how query execution times are distributed, for example how +many queries finish within the fastest 10%, 20%, and so on up to 100%. -SELECT '50', p50 as micros_threshold, count(*) -FROM _query_trace CROSS JOIN quantiles -WHERE execution_micros < p50 - -UNION ALL - -SELECT '60', p60 as micros_threshold, count(*) -FROM _query_trace CROSS JOIN quantiles -WHERE execution_micros < p60 - -UNION ALL - -SELECT '70', p70 as micros_threshold, count(*) -FROM _query_trace CROSS JOIN quantiles -WHERE execution_micros < p70 - -UNION ALL - -SELECT '80', p80 as micros_threshold, count(*) -FROM _query_trace CROSS JOIN quantiles -WHERE execution_micros < p80 - -UNION ALL - -SELECT '90', p90 as micros_threshold, count(*) -FROM _query_trace CROSS JOIN quantiles -WHERE execution_micros < p90 - -UNION ALL +## Solution: Percentile-based histogram -SELECT '100', p100 as micros_threshold, count(*) -FROM _query_trace CROSS JOIN quantiles - ) - SELECT * FROM cumulative_hist; +Compute every bucket threshold in one pass with +[`percentile_disc`](/docs/query/functions/aggregation/#percentile_disc--quantile_disc) +and an array of percentiles. [`UNNEST ... WITH ORDINALITY`](/docs/query/sql/unnest/) +turns the resulting array into one row per bucket, and a `CROSS JOIN` against +`_query_trace` counts the queries at or below each threshold. + +```questdb-sql title="Cumulative histogram of query execution times" +WITH thresholds AS ( + SELECT u.bucket * 10 AS bucket, u.micros_threshold + FROM ( + SELECT percentile_disc( + execution_micros, + ARRAY[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0] + ) AS p + FROM _query_trace + ) q, UNNEST(q.p) WITH ORDINALITY u(micros_threshold, bucket) +) +SELECT t.bucket, t.micros_threshold, count() AS frequency +FROM thresholds t +CROSS JOIN _query_trace qt +WHERE qt.execution_micros <= t.micros_threshold +ORDER BY t.bucket; ``` **Output:** -```csv -"bucket","micros_threshold","frequency" -"10",215.0,26 -"20",348.0,53 -"30",591.0,80 -"40",819.0,106 -"50",1088.0,133 -"60",1527.0,160 -"70",2293.0,186 -"80",4788.0,213 -"90",23016.0,240 -"100",1078759.0,267 +| bucket | micros_threshold | frequency | +| :----- | :--------------- | :-------- | +| 10 | 524.0 | 9 | +| 20 | 567.0 | 17 | +| 30 | 822.0 | 26 | +| 40 | 1588.0 | 34 | +| 50 | 1909.0 | 43 | +| 60 | 4421.0 | 51 | +| 70 | 7419.0 | 60 | +| 80 | 11096.0 | 68 | +| 90 | 15860.0 | 77 | +| 100 | 20787.0 | 85 | + +The histogram is cumulative: each bucket includes the queries from the smaller +buckets. `percentile_disc` returns an execution time that actually occurred, so +the `<=` comparison always includes the query at the threshold, and the 100% +bucket counts every query. + +For a non-cumulative histogram, subtract the previous bucket's count with +[`lag()`](/docs/query/functions/window-functions/reference/#lag): + +```questdb-sql title="Non-cumulative histogram of query execution times" +WITH thresholds AS ( + SELECT u.bucket * 10 AS bucket, u.micros_threshold + FROM ( + SELECT percentile_disc( + execution_micros, + ARRAY[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0] + ) AS p + FROM _query_trace + ) q, UNNEST(q.p) WITH ORDINALITY u(micros_threshold, bucket) +), cumulative_hist AS ( + SELECT t.bucket, t.micros_threshold, count() AS frequency + FROM thresholds t + CROSS JOIN _query_trace qt + WHERE qt.execution_micros <= t.micros_threshold + ORDER BY t.bucket +) +SELECT + bucket, + micros_threshold, + frequency - coalesce(lag(frequency) OVER (ORDER BY bucket), 0) AS frequency +FROM cumulative_hist; ``` +`percentile_disc` keeps every execution time in memory while it computes the +thresholds. On a very large `_query_trace` table, use +[`approx_percentile`](/docs/query/functions/aggregation/#approx_percentile) +with the same array instead, for example +`approx_percentile(execution_micros, ARRAY[0.1, ..., 1.0], 5)`, which uses a +bounded amount of memory. :::info Related Documentation - [Query tracing](/docs/concepts/deep-dive/query-tracing/) +- [percentile_disc() function](/docs/query/functions/aggregation/#percentile_disc--quantile_disc) - [approx_percentile() function](/docs/query/functions/aggregation/#approx_percentile) +- [UNNEST](/docs/query/sql/unnest/) :::