Refuse invalid queries before SQL is sent: typed params, GROUP BY and dialect checks

CHANGELOG.md

@@ -2,6 +2,35 @@
 
 ## Unreleased
 
+- **Breaking:** invalid queries are refused before any SQL is sent: as type errors where the
+  type can see them, otherwise as a `QueryBuilderError` / `QueryBuilderDefect` from `compile`.
+  - Params are in the query's type. `compile`, `compileUnion` and `Database.run` require every
+    `param.*` the query uses, with a value of its type (`CHQuery`, `CHUnionQuery`, `CHInsert`,
+    `CHUpdate` and `CHDelete` gain a `Params` type parameter; `Expr` and `Condition` gain `P`).
+  - A second `where()` / `having()` ANDs with the first instead of replacing it, on queries and
+    on writes.
+  - Comparisons refuse `null` (use `isNull()`); an empty `in_()` / `notIn()` is `1 = 0` / `1 = 1`.
+    `like` / `ilike` accept a nullable string.
+  - `limit` / `offset` refuse negative, fractional or non-finite values instead of rounding them.
+  - A query with no `select()` cannot be compiled, run, joined, used in `FROM`, a CTE, `EXISTS`
+    or `INSERT ... SELECT`. `unionAll` branches must agree on aliases and column types.
+    `inSubquery` / `notInSubquery` need exactly one column of a comparable type.
+  - Join aliases must be unique and must not shadow a FROM column or the FROM alias; CTE names
+    must be unique.
+  - `update().set({})` and a SET or insert row naming a column the table cannot write are type
+    errors; an UPDATE or DELETE without `where()` or `allRows()` cannot be compiled or run.
+  - An aggregate in WHERE or a join's ON, a column that is neither grouped nor aggregated, and
+    grouping by an aggregate fail to compile. SQL the builder did not write (`rawExpr`,
+    `CH.sql`, windows, `makeExpr`) is not looked inside.
+  - Built-in functions belong to a dialect: a ClickHouse function (such as `count()`) in a
+    Postgres compile fails, and the reverse. `coalesce`, `nullIf` and `lower` are portable.
+    `Dialect.functions` names a dialect's function set.
+  - `makeExpr`, `makeUntypedExpr` and `makeCond` take the expressions they interpolate as
+    `uses`, whose params the result carries; a param in the SQL that no `uses` entry carries
+    fails to compile. Their value type comes from the schema: explicit type arguments
+    (`makeExpr<T>`, `subqueryExpr<T>`, `compileTypedFnCall<R>`) are errors, so they cannot
+    silently drop params. `untypedSubqueryExpr` returns `Expr<unknown>`.
+  - `inSubquery` / `notInSubquery` check at compile time that the subquery selects one column.
 - Add `CH.sql`: SQL templates inside expressions and conditions. `CH.sql(type)\`…\`` is a typed
   `Expr`, ``CH.sql`…` `` an untyped one, `CH.sql.cond` a `Condition`; with `sql.ident`, `sql.raw`
   and `sql.join`. Interpolated columns and params render as SQL and placeholders, a builder

docs/expressions.md

@@ -19,8 +19,10 @@ $.Timestamp.gte(new Date(...))            // Timestamp >= '2026-01-01 00:00:00'
 
 ### Testing for NULL
 
-`.eq(null)` emits `= NULL`; it does not test whether a value is missing. Use `.isNull()` (or
-`.isNotNull()` for present values), which write `IS NULL` and work on every dialect:
+`= NULL` is never true in SQL, so a comparison does not take `null`: `.eq(null)`,
+`.in_(null)` and the like are type errors, and a `null` that arrives at runtime fails
+compilation with a `QueryBuilderError`. Use `.isNull()` (or `.isNotNull()` for present
+values), which write `IS NULL` and work on every dialect:
 
 ```ts title="null-filter.ts"
 import * as CH from "@maple-dev/effect-orm"
@@ -62,7 +64,9 @@ Every `Expr<T>` carries:
 Each accepts a raw value or another `Expr<T>`. String literals are escaped; booleans emit as
 `1` / `0`.
 
-`in_` carries a trailing underscore because `in` is a reserved word in JavaScript.
+`in_` carries a trailing underscore because `in` is a reserved word in JavaScript. An empty
+list is written as the constant it means, `1 = 0` for `in_()` and `1 = 1` for `notIn()`, since
+`IN ()` is not SQL.
 
 ### String-only
 

docs/extending.md

@@ -78,7 +78,7 @@ import * as CH from "@maple-dev/effect-orm"
 import * as T from "@maple-dev/effect-orm/types"
 
 const greatestOf = (first: CH.Expr<number>, ...rest: CH.Expr<number>[]) =>
-	CH.compileTypedFnCall<number>("greatest", T.float64.schema, first, ...rest)
+	CH.compileTypedFnCall("greatest", T.float64.schema, first, ...rest)
 
 const Events = CH.table("events", { Name: T.string, DurationMs: T.uint64 })
 export const compiled = CH.compileUnsafe(
@@ -104,11 +104,16 @@ anything bespoke:
 import { makeExpr } from "@maple-dev/effect-orm"
 import { raw, compile } from "@maple-dev/effect-orm/sql"
 
-const quantileExact = (q: number) => (expr: CH.Expr<number>) =>
-	makeExpr<number>(raw(`quantileExact(${q})(${compile(expr.toFragment())})`), T.float64.schema)
+const quantileExact =
+	(q: number) =>
+	<Q = never>(expr: CH.Expr<number, Q>) =>
+		makeExpr(raw(`quantileExact(${q})(${compile(expr.toFragment())})`), T.float64.schema, undefined, [expr])
 ```
 
-This is how the bundled `quantile` is built. Note the second argument: `makeExpr` requires a
+This is how the bundled `quantile` is built. The last argument, `uses`, lists the expressions
+the fragment interpolates (see [below](#params-and-checks-on-a-custom-function)). The value
+type comes from the schema; `makeExpr<number>(…)` with an explicit type argument does not
+type-check. Note the second argument: `makeExpr` requires a
 schema — passing `undefined` is how a wrapper _forwards_ the untypedness of its own argument
 (`schemaOf(arg)`), not something to write. For an expression that genuinely has no type, use
 `makeUntypedExpr`, which says so and costs the query its row schema knowingly.
@@ -128,6 +133,21 @@ console.log(predicate) // Name = 'O\'Reilly'
 Use this for string literals only. Keep SQL structure and identifiers under application
 control, and validate numeric inputs such as the quantile level separately.
 
+### Params and checks on a custom function
+
+`Expr<T, P>` carries the `param.*` placeholders inside an expression, so `compile` can require
+them. `defineFn`, `defineCondFn` and `compileTypedFnCall` pass their arguments' params on by
+themselves. `makeExpr`, `makeUntypedExpr` and `makeCond` cannot see inside the SQL you build,
+so they take the expressions you interpolate as `uses` (the last argument): the result carries
+their params, and compiling fails with a `QueryBuilderDefect` if the SQL holds a param that no
+`uses` entry carries. A param can therefore not reach a query without being in its type.
+
+Generic functions take their params as one type parameter per argument (`Expr<number, Q>`
+above); a parameter written as a plain `Expr<T>` accepts any expression but drops its params
+from the type, so they are then checked only when compiling. SQL built with `makeExpr`, `defineFn` or `CH.sql` is also opaque to the GROUP
+BY checks (see [Queries](./queries.md#groupby)): a mistake inside it reaches the database, but
+it never makes a valid query fail.
+
 ## A column type of your own
 
 `T.custom(sql, schema)` is the extension point the built-in types are built from — `T.uint64` is

docs/params-and-compilation.md

@@ -29,6 +29,31 @@ Param names must be alphanumeric, optionally separated by single underscores —
 through the placeholder that `compile` later matches, and `__` would make its boundary
 ambiguous. A name that cannot round-trip is a `QueryBuilderDefect` at declaration.
 
+## Params are in the query's type
+
+A query remembers the params it uses, with their types, and `compile`, `compileUnion` and
+`Database.run` require them:
+
+```ts
+const byOrg = CH.from(Events)
+	.select("Name")
+	.where(($) => [$.OrgId.eq(CH.param.string("orgId")), $.Ms.gt(CH.param.int("minMs"))])
+
+CH.compile(byOrg, { orgId: "org_1", minMs: 100 }) // ok
+CH.compile(byOrg, { orgId: "org_1" }) // type error: paramsRequired { orgId: string; minMs: number }
+CH.compile(byOrg, { orgId: 1, minMs: 100 }) // type error: orgId is a string
+```
+
+Params are collected from `where`, `having`, `select`, join `on` callbacks, subqueries in
+`FROM`, joins, CTEs and `EXISTS`/`IN`, union branches, and insert rows, `SET` records and
+write `where`s. Extra keys are allowed, so one params object can serve several queries. A
+query without params takes none.
+
+A function the builder does not know passes its arguments' params on only if its signature
+says so: `defineFn`, `defineCondFn` and `compileTypedFnCall` from the extending API do, a
+hand-written `makeExpr` does not. A param the type does not see is still checked when
+compiling, as below.
+
 ## What each kind accepts
 
 The declared kind is checked when the value arrives, so a value of the wrong shape is a
@@ -280,6 +305,7 @@ const query = CH.from(Events)
 	.where(($) => [$.Name.eq(CH.param.string("name"))])
 
 export const outcome = await Effect.runPromise(
+	// @ts-expect-error -- a missing param is a type error too; this shows the runtime failure
 	CH.compile(query, {}).pipe(
 		Effect.map((compiled) => ({ ok: true as const, sql: compiled.sql })),
 		Effect.catchTag("@maple-dev/effect-orm/QueryBuilderError", (error) =>

docs/postgres.md

@@ -109,7 +109,11 @@ which no session time zone can reinterpret; a zoneless string is read as UTC.
 
 The shared operators (`eq`, `in_`, `like`, `ilike`, `and`, `or`, `not`, arithmetic, `lit`) work
 unchanged. The ClickHouse function catalog on the root entry (`quantile`, `toStartOfInterval`,
-map subscripts, …) writes ClickHouse SQL and will not run on Postgres.
+`count()`, …) writes ClickHouse SQL, so compiling a query that uses one for Postgres is a
+`QueryBuilderDefect` naming the function; the Postgres functions above fail the same way on
+ClickHouse. `coalesce`, `nullIf` and `lower` from the root entry render the same on both and
+are allowed on either. A custom `Dialect` opts in with `functions: "clickhouse"` or
+`"postgres"`; without it, nothing is checked.
 
 ## Known differences
 

docs/queries.md

@@ -58,13 +58,13 @@ _(Backed by `docs/queries.md > select by column name`.)_
 Entries may be `undefined`, which drops them — that is what makes optional filters clean. See
 [`when` / `whenTrue`](./expressions.md#optional-predicates).
 
-**Calling `where` again replaces the previous callback.** It does not append predicates.
-Put the complete filter set in one callback, including tenant and time bounds. Both flat
+**Calling `where` again adds conditions**, ANDed with the earlier ones, as in Kysely. A shared
+base that filters by tenant keeps that filter however many `.where(...)` calls follow. Both flat
 conditions and `.and()` preserve [tenant scoping](./tenant-scoping.md); `.or()` does not.
+`having` accumulates the same way.
 
-The same replacement rule applies to `select`, `groupBy`, `having`, `orderBy`, `limit`,
-`offset`, and `format`. Joins and CTEs accumulate. Immutable does not mean additive:
-a second `.where(...)` on a shared base can remove its tenant filter.
+`select`, `groupBy`, `orderBy`, `limit`, `offset`, and `format` replace the previous value. Joins
+and CTEs accumulate.
 
 ## `groupBy`
 
@@ -75,6 +75,19 @@ Takes **output keys** (the aliases from `select`), not raw column names:
 .groupBy("name")
 ```
 
+Once a query groups or aggregates, every column it reads outside an aggregate must be a
+`groupBy` key, as both databases require. Compiling one that breaks the rule is a
+`QueryBuilderDefect` naming the alias and column, instead of a server error:
+
+- `select(($) => ({ name: $.Name, n: CH.count() }))` with no `groupBy("name")`;
+- an aggregate in `where` or a join's `on` (filter on it in `having`);
+- `groupBy` naming an aggregate alias.
+
+An expression over a grouped column (`CH.lower($.Name)` with `Name` grouped) and a repeat of a
+grouped expression are fine. Only SQL the builder writes is checked: a window (`CH.over`), a
+`CH.sql` template, `rawExpr` and functions declared with `defineFn` / `makeExpr` are not looked
+inside, so they can hide a mistake from this check but never trigger a false one.
+
 ## `having`
 
 Filter groups after aggregation. The callback has the input-column accessor, so either repeat
@@ -127,9 +140,10 @@ Postgres wants those keys to lead the ORDER BY. Both ClickHouse and Postgres sup
 .limit(50).offset(100)
 ```
 
-Both take numbers, not `param.*` expressions, and are rounded with `Math.round` before
-emission. That is not input validation: reject non-finite, negative, or fractional values at
-your request boundary, and enforce an application maximum. Use a stable `orderBy` when paging;
+Both take non-negative integers, not `param.*` expressions. A negative or fractional literal is
+a type error; a value that arrives at runtime (`NaN`, `-1`, `1.5`) fails compilation with a
+`QueryBuilderError` instead of being rounded. Still enforce an application maximum at your
+request boundary. Use a stable `orderBy` when paging;
 [Recipes](./recipes.md#paginate-a-grouped-result) shows where an offset is appropriate.
 
 ## `format`

docs/reference.md

@@ -152,7 +152,7 @@ outer set and its failures land in the outer error channel. See
 | Export                             | Purpose                                   |
 | ---------------------------------- | ----------------------------------------- |
 | `subqueryExpr(q, type, wrap?)`     | Inner SQL as an `Expr` of a declared type |
-| `untypedSubqueryExpr<T>(q, wrap?)` | Same with no type — costs the row schema  |
+| `untypedSubqueryExpr(q, wrap?)`    | Same with no type — costs the row schema  |
 | `subqueryCond(q, wrap)`            | Inner SQL as a `Condition`                |
 
 `wrap` receives the inner SQL and returns the text to emit. It defaults to wrapping the SQL in
@@ -177,9 +177,9 @@ parentheses, which is the plain "this value is a sub-SELECT" case.
 | `compileFnCall<R>(name, ...args)`      | Variadic/generic wrapper (untyped result)          |
 | `compileTypedFnCall<R>(name, schema,)` | Same, with the result codec                        |
 | `compileFnCallCond(name, ...args)`     | Same, returning `Condition`                        |
-| `makeExpr<T>(fragment, schema)`        | Build an `Expr` from a fragment and its codec      |
-| `makeUntypedExpr<T>(fragment)`         | Same with no codec — costs the row schema          |
-| `makeCond(fragment)`                   | Build a `Condition` from a fragment                |
+| `makeExpr(fragment, schema, literal?, uses?)` | Build an `Expr` from a fragment and its codec; `uses` carries params |
+| `makeUntypedExpr(fragment, literal?, uses?)`  | Same with no codec — costs the row schema          |
+| `makeCond(fragment, uses?)`            | Build a `Condition` from a fragment                |
 | `schemaOf(expr)`                       | An expression's codec, or `undefined`              |
 | `schemaOfAny(...exprs)`                | The first codec among several                      |
 | `elementSchema(expr)`                  | The element codec of an array expression           |

docs/tenant-scoping.md

@@ -180,5 +180,5 @@ inspected. Whatever you pass is taken at face value — which is why it also req
 `"single-tenant"` proves a structural restriction to one value, not that the requester is allowed
 to access that value. Resolve tenant IDs from trusted context. Treat `"untenanted"` as acceptable
 only for tables your application intentionally models as shared; omitting `tenantColumn` from
-a real tenant table bypasses that evidence. Repeated `.where()` calls replace the earlier
-filter, so assemble tenant and optional predicates in the same callback.
+a real tenant table bypasses that evidence. Repeated `.where()` calls AND with the earlier
+ones, so a tenant filter on a shared base query survives later filters.

docs/updates-and-deletes.md

@@ -40,7 +40,9 @@ names the record type.
 `where` works as in a query: a list of conditions, AND-joined, with an `undefined` one skipped,
 so optional filters compose. A write with no `where` would change every row, so:
 
-- compiling an UPDATE or DELETE with no `where()` is a `QueryBuilderDefect`;
+- an UPDATE or DELETE with no `where()` or `allRows()` is a type error, and compiling one
+  that slipped past the types is a `QueryBuilderDefect`;
+- calling `where` again ANDs the new conditions with the earlier ones;
 - a `where()` whose conditions all came out `undefined` (or render to nothing) is a
   `QueryBuilderError`, because that happens with data (every optional filter absent) and would
   otherwise widen a filtered write to the whole table;

scripts/check-doc-examples.mjs

@@ -76,11 +76,12 @@ assert.match(sql(page.compiled), /ORDER BY count DESC, name ASC LIMIT 25 OFFSET
 const ids = await import("./large-ids")
 assert.match(sql(ids.compiled), /toString\\(records.Id\\) AS id/)
 assert.equal(ids.rows[0]?.id, "18446744073709551615")
-const replaced = CH.compileUnsafe(CH.from(Events).select("Name")
+const anded = CH.compileUnsafe(CH.from(Events).select("Name")
   .where(($) => [$.OrgId.eq("org_123")])
   .where(($) => [$.Name.eq("checkout")]), {})
-assert.equal(replaced.tenantScope, "cross-tenant")
-assert.doesNotMatch(replaced.sql, /OrgId =/)
+assert.equal(anded.tenantScope, "single-tenant")
+assert.match(anded.sql, /OrgId = 'org_123'/)
+assert.match(anded.sql, /Name = 'checkout'/)
 const benchmark = await import("./benchmark-suite")
 const suite = await Effect.runPromise(benchmark.default)
 assert.equal(suite.source, "events")

src/ch/compilation-regressions.test.ts

@@ -82,6 +82,7 @@ describe("subquery source scope", () => {
 	it.effect("keeps deferred failures typed and restores compilation context", () => Effect.gen(function* () {
 		const predicate = CH.subqueryExpr(scopedCount, T.uint64).gt(0)
 		const query = outer.select("Id").having(() => [predicate])
+		// @ts-expect-error -- a missing param is a type error too
 		const result = yield* CH.compile(query, { outer: "a" }).pipe(Effect.result)
 		expect(result._tag).toBe("Failure")
 		if (result._tag === "Failure") expect(result.failure.code).toBe("UnresolvedParam")

src/ch/compile.test.ts

@@ -337,7 +337,7 @@ describe("CompiledQuery.tenantScope", () => {
 		// The shape that satisfied the old `sql.includes("OrgId")` guard.
 		const compiled = compileCHUnsafe(
 			CH.from(events)
-				.select(($) => ({ OrgId: $.OrgId, count: $.Count }))
+				.select(($) => ({ OrgId: $.OrgId, count: CH.sum($.Count) }))
 				.groupBy("OrgId"),
 			{},
 		)
@@ -513,6 +513,7 @@ describe("compile puts failures in the error channel", () => {
 	// than a typed failure anyone could map to a 400.
 	it.effect("a missing param value is a typed failure", () =>
 		Effect.gen(function* () {
+			// @ts-expect-error -- a missing param is a type error too
 			const error = yield* Effect.flip(CH.compile(query, {}))
 			expect(error._tag).toBe("@maple-dev/effect-orm/QueryBuilderError")
 			expect(error.code).toBe("UnresolvedParam")