Skip to content

Repository files navigation

chtypes

ClickHouse's own type system, as a library you can call.

Validate, coerce and preview rows before they reach a server —
answered by ClickHouse's real C++, not a model of it.

CI License: Apache 2.0 Go PyPI npm crates.io

Docs · Install · Quickstart · Supported versions · How it compares · Examples


"What does ClickHouse do with 256 into a UInt8?" has exactly one correct answer, and it is whatever ClickHouse's code does — which changes between releases. chtypes compiles ClickHouse's real C++ (DataTypeFactory, ISerialization, ReadHelpers, evaluateMissingDefaults, the MergeTree insert-time merge) per release into a native artifact behind a small frozen C ABI, and hands it to Go, Python, TypeScript and Rust. Nothing semantic is reimplemented in any binding.

row + schema + ClickHouse version  ─▶  accepted / rejected / poisoned
                                       the stored value, byte for byte
                                       every silent change, named

That last line is the point. ClickHouse will accept 256 into a UInt8, store 0, and never mention it. chtypes names it.

Install

Two things, always: the binding for your language, and at least one artifact — the per-version native library it loads. The binding is small; the artifact is real ClickHouse, compiled.

go get github.com/wave-rf/chtypes/go     # Go
uv add chtypes                           # Python  (or: pip install chtypes)
pnpm add @wavehouse/chtypes              # TypeScript
cargo add chtypes                        # Rust

Then fetch an artifact — verified, into the per-user cache every binding reads by default. Each binding ships the same command, so you need nothing from this repository:

go run github.com/wave-rf/chtypes/go/cmd/chtypes@latest fetch 25.8
python -m chtypes fetch 25.8
npx @wavehouse/chtypes fetch 25.8
cargo install chtypes && chtypes fetch 25.8

An ed25519 signature over the release and the sha256 of every byte are checked before anything lands.

⚠️ From 0.3.0 the rolling channel serves ABI revision 5. A 0.2.x binding will download these artifacts and refuse them at load. Pinning an exact ClickHouse version does not help — a relink republishes the same version and the newest row wins. Point --url / --tag at a source still serving revision 4, or upgrade to 0.3.0. fetch --all takes every published line. Full details: docs/install.md.

Quickstart

One artifact, one schema, one row — the same program in each language. The row is accepted, and 256 is silently stored as 0.

Go
reg, _ := chtypes.NewRegistry(chtypes.DefaultRegistryDir())
lib, _ := reg.For("25.8")                      // a line or an exact patch; never a nearest match
cs, _ := lib.CompileDDL("x UInt8, ts DateTime DEFAULT now()")
defer cs.Close()

r, _ := cs.Row(chtypes.JSONEachRow, []byte(`{"x":256}`))
fmt.Println(r.Outcome)                // accepted
fmt.Println(r.Transformed[0].Reason)  // overflow_wrap — 256 stored as 0, silently
fmt.Println(r.Substituted)            // ts: send it explicitly, or preview != stored
Python
from chtypes import Format, Registry

registry = Registry()
library = registry.for_version("25.8")

with library.compile_ddl("x UInt8, ts DateTime DEFAULT now()") as schema:
    r = schema.rows(Format.JSON_EACH_ROW, b'{"x":256}\n')

r.outcome                     # Outcome.ACCEPTED
r.rows[0].value("x").text     # '0'             — what would actually be stored
r.transformed[0].reason       # 'overflow_wrap' — which is the product
r.rows[0].substituted         # ts: send it explicitly, or preview != stored
TypeScript
import { Format, Registry } from '@wavehouse/chtypes';

const registry = new Registry();
const lib = registry.for('25.8');
const schema = lib.compileDdl('x UInt8, ts DateTime DEFAULT now()');

const r = schema.row(Format.JSONEachRow, Buffer.from('{"x":256}'));
console.log(r.outcome);                 // accepted
console.log(r.transformed[0]?.reason);  // overflow_wrap — 256 stored as 0, silently
console.log(r.substituted[0]?.column);  // ts — send it explicitly in the real INSERT
schema.close();                         // or `using schema = …` on Node >= 24
Rust
use chtypes::{Format, Registry, NO_SETTINGS};

let registry = Registry::from_search_path();
let lib = registry.for_version("25.8")?;
let schema = lib.compile("x UInt8, ts DateTime DEFAULT now()").compile()?;

let r = schema.rows(Format::JsonEachRow, br#"{"x":256}"#, NO_SETTINGS)?;
assert_eq!(r.outcome, chtypes::Outcome::Accepted);
assert_eq!(r.rows[0].values[0].text, "0");                      // what would be stored
assert_eq!(r.transformed[0].reason, chtypes::reason::OVERFLOW_WRAP);

A bad row is a verdict, not an error: outcome becomes rejected, carrying ClickHouse's own error code and message. Exceptions (or the Err arm) are for the machinery — a missing artifact, an unreadable document. Full quickstart · the same tour, runnable, in all four languages

How it compares

chtypes hand-rolled validation round-trip to a real server
Exact ClickHouse semantics ClickHouse's own C++ an approximation that drifts yes
Answers before the insert yes yes no — the row has already gone
Names what was silently changed yes, every coercion no no
Per-release answers one artifact per ClickHouse line no only that one server's version
Cost per row in-process call in-process call a network round trip
Needs a running ClickHouse no no yes

The guarantees every binding is held to

  • Transformed is the product. ClickHouse never says "I changed your value"; chtypes derives that report (overflow_wrap, date_clamp, poisoned, ttl_expired, …) and it is not optional.
  • Over-accepts and over-rejects have no budget — a non-zero count is refused unless a person has named that case and recorded why, with a tracking reference. Known cases exist and are registered individually rather than absorbed into an allowance; what that does and does not promise: docs/limitations.md.
  • unsupported is an answer, never a guess. A binding surfaces the library's decline; it never papers over one.
  • The four bindings give one answer. The golden set — a published file of expected answers every binding must reproduce — is run by all of them, and each is scored against real ClickHouse servers at the same agreement as the reference.

What is supported

Three axes — the language you call from, the platform you run on, and the ClickHouse line you want answers for. docs/support.md carries the full matrix, generated from this tree's manifests and the release's own index so it cannot drift from what actually ships.

The short version: Go, Python, TypeScript and Rust; linux-amd64, linux-arm64 and darwin-arm64 (Unix only — both loaders are dlopen); and every ClickHouse line that has passed the artifact producer's comparison against a real server, today spanning 24.8 through 26.8 and growing as new releases pass it.

Documentation

Install · Quickstart getting a binding and an artifact, and the first program
Guides artifacts, fetching, settings, batches, filters, discovery, multi-version, transformations
Reference per-language API, the C ABI contract, the binding contract
Supported versions languages, platforms, ClickHouse lines
Examples four side-by-side runnable tours, same sections in every language

Project status

Pre-1.0, and published to all four registries — the badges above read the live version from each. What is already frozen before 1.0 is in docs/support.md.

This repository is the SDK half of chtypes, Apache 2.0. The other half — the C++ wrapper, the per-version vendoring and build pipeline, the artifacts themselves, and the differential proof (tens of thousands of cases scored against real ClickHouse servers on every supported version) — belongs to the artifact producer, under its own license. The bindings here contain no ClickHouse code: they load an artifact and speak the ABI. Artifacts carry their own license; see the LICENSE inside each release.

Contributing

Issues and pull requests are welcome — start with CONTRIBUTING.md. A change to one binding's behavior lands in all four in the same cycle; that is the contract, not a preference. Report security issues privately: SECURITY.md.

AI-assisted development

Much of this repository was written with AI assistance and reviewed by a human before merge. The review gate is the same regardless of who or what authored a change, and the test suites are deliberately built to refuse a silent pass — every artifact-dependent test skips loudly by name, and a suite that ran nothing fails. If you find documentation that drifted from the code, please open an issue; that is the failure mode we most want reported.

License

Apache 2.0 — see LICENSE and NOTICE.

About

ClickHouse's own type system as a library: Go, Python, TypeScript and Rust bindings over the frozen chs_* C ABI (Apache 2.0)

Topics

Resources

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages