Skip to content

docs(sql): document percentile_cont and percentile_disc #599

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
javier merged 2 commits into from
Oct 23, 2026
Merged

docs(sql): document percentile_cont and percentile_disc #599

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
541fc4c
docs(sql): document percentile_cont and percentile_disc
nwoolmer Oct 22, 2026
b48196d
docs(cookbook): build query time histogram from a percentile array
nwoolmer Oct 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
165 changes: 77 additions & 88 deletions documentation/cookbook/operations/query-times-histogram.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,105 +11,94 @@ Create a histogram of query execution times using the `_query_trace` system tabl
[Query tracing](/docs/concepts/deep-dive/query-tracing/) needs to be enabled for the `_query_trace` table to be populated.
:::

## Solution: Percentile-based histogram

We can create a subquery that first calculates the percentiles for each bucket, in this case at 10% intervals. Then on a second query we can do a `UNION` of 10 subqueries where each is doing a `CROSS JOIN` against the calculated percentiles and finding how many queries are below the threshold for the bucket.

Note in this case the histogram is cumulative, and each bucket includes the results from the smaller buckets as well. If we prefer non-cumulative, the condition would change from less than to `BETWEEN`.

```sql
WITH quantiles AS (
SELECT
approx_percentile(execution_micros, 0.10, 5) AS p10,
approx_percentile(execution_micros, 0.20, 5) AS p20,
approx_percentile(execution_micros, 0.30, 5) AS p30,
approx_percentile(execution_micros, 0.40, 5) AS p40,
approx_percentile(execution_micros, 0.50, 5) AS p50,
approx_percentile(execution_micros, 0.60, 5) AS p60,
approx_percentile(execution_micros, 0.70, 5) AS p70,
approx_percentile(execution_micros, 0.80, 5) AS p80,
approx_percentile(execution_micros, 0.90, 5) AS p90,
approx_percentile(execution_micros, 1.0, 5) AS p100
FROM _query_trace
), cumulative_hist AS (
SELECT '10' AS bucket, p10 as micros_threshold, count(*) AS frequency
FROM _query_trace CROSS JOIN quantiles
WHERE execution_micros < p10

UNION ALL

SELECT '20', p20 as micros_threshold, count(*)
FROM _query_trace CROSS JOIN quantiles
WHERE execution_micros < p20

UNION ALL

SELECT '30', p30 as micros_threshold, count(*)
FROM _query_trace CROSS JOIN quantiles
WHERE execution_micros < p30

UNION ALL

SELECT '40', p40 as micros_threshold, count(*)
FROM _query_trace CROSS JOIN quantiles
WHERE execution_micros < p40
## Problem

UNION ALL
You want to see how query execution times are distributed, for example how
many queries finish within the fastest 10%, 20%, and so on up to 100%.

SELECT '50', p50 as micros_threshold, count(*)
FROM _query_trace CROSS JOIN quantiles
WHERE execution_micros < p50

UNION ALL

SELECT '60', p60 as micros_threshold, count(*)
FROM _query_trace CROSS JOIN quantiles
WHERE execution_micros < p60

UNION ALL

SELECT '70', p70 as micros_threshold, count(*)
FROM _query_trace CROSS JOIN quantiles
WHERE execution_micros < p70

UNION ALL

SELECT '80', p80 as micros_threshold, count(*)
FROM _query_trace CROSS JOIN quantiles
WHERE execution_micros < p80

UNION ALL

SELECT '90', p90 as micros_threshold, count(*)
FROM _query_trace CROSS JOIN quantiles
WHERE execution_micros < p90

UNION ALL
## Solution: Percentile-based histogram

SELECT '100', p100 as micros_threshold, count(*)
FROM _query_trace CROSS JOIN quantiles
)
SELECT * FROM cumulative_hist;
Compute every bucket threshold in one pass with
[`percentile_disc`](/docs/query/functions/aggregation/#percentile_disc--quantile_disc)
and an array of percentiles. [`UNNEST ... WITH ORDINALITY`](/docs/query/sql/unnest/)
turns the resulting array into one row per bucket, and a `CROSS JOIN` against
`_query_trace` counts the queries at or below each threshold.

```questdb-sql title="Cumulative histogram of query execution times"
WITH thresholds AS (
SELECT u.bucket * 10 AS bucket, u.micros_threshold
FROM (
SELECT percentile_disc(
execution_micros,
ARRAY[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]
) AS p
FROM _query_trace
) q, UNNEST(q.p) WITH ORDINALITY u(micros_threshold, bucket)
)
SELECT t.bucket, t.micros_threshold, count() AS frequency
FROM thresholds t
CROSS JOIN _query_trace qt
WHERE qt.execution_micros <= t.micros_threshold
ORDER BY t.bucket;
```

**Output:**

```csv
"bucket","micros_threshold","frequency"
"10",215.0,26
"20",348.0,53
"30",591.0,80
"40",819.0,106
"50",1088.0,133
"60",1527.0,160
"70",2293.0,186
"80",4788.0,213
"90",23016.0,240
"100",1078759.0,267
| bucket | micros_threshold | frequency |
| :----- | :--------------- | :-------- |
| 10 | 524.0 | 9 |
| 20 | 567.0 | 17 |
| 30 | 822.0 | 26 |
| 40 | 1588.0 | 34 |
| 50 | 1909.0 | 43 |
| 60 | 4421.0 | 51 |
| 70 | 7419.0 | 60 |
| 80 | 11096.0 | 68 |
| 90 | 15860.0 | 77 |
| 100 | 20787.0 | 85 |

The histogram is cumulative: each bucket includes the queries from the smaller
buckets. `percentile_disc` returns an execution time that actually occurred, so
the `<=` comparison always includes the query at the threshold, and the 100%
bucket counts every query.

For a non-cumulative histogram, subtract the previous bucket's count with
[`lag()`](/docs/query/functions/window-functions/reference/#lag):

```questdb-sql title="Non-cumulative histogram of query execution times"
WITH thresholds AS (
SELECT u.bucket * 10 AS bucket, u.micros_threshold
FROM (
SELECT percentile_disc(
execution_micros,
ARRAY[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]
) AS p
FROM _query_trace
) q, UNNEST(q.p) WITH ORDINALITY u(micros_threshold, bucket)
), cumulative_hist AS (
SELECT t.bucket, t.micros_threshold, count() AS frequency
FROM thresholds t
CROSS JOIN _query_trace qt
WHERE qt.execution_micros <= t.micros_threshold
ORDER BY t.bucket
)
SELECT
bucket,
micros_threshold,
frequency - coalesce(lag(frequency) OVER (ORDER BY bucket), 0) AS frequency
FROM cumulative_hist;
```

`percentile_disc` keeps every execution time in memory while it computes the
thresholds. On a very large `_query_trace` table, use
[`approx_percentile`](/docs/query/functions/aggregation/#approx_percentile)
with the same array instead, for example
`approx_percentile(execution_micros, ARRAY[0.1, ..., 1.0], 5)`, which uses a
bounded amount of memory.

:::info Related Documentation
- [Query tracing](/docs/concepts/deep-dive/query-tracing/)
- [percentile_disc() function](/docs/query/functions/aggregation/#percentile_disc--quantile_disc)
- [approx_percentile() function](/docs/query/functions/aggregation/#approx_percentile)
- [UNNEST](/docs/query/sql/unnest/)
:::
Loading
Loading