parqdb-io

Billion-scale embedded vector database built entirely on Parquet and Arrow.

ParqDB is an embedded vector database for larger-than-memory search and analytics on billion-scale multimodal data, with Parquet storage and Arrow-native execution.

• Billion-scale search in bounded memory. Search 1B vectors at 90.3% recall with 63.05 ms median latency using just 2 CPU cores and 4 GB of memory.
• Everything is Parquet. Source data and vector indexes use standard Parquet rather than proprietary binary formats.
• Multimodal data, SQL-native search. Combine vector search with filters, joins, and aggregations in one execution plan.
• Built for serving and analytics. Parallelize across queries for throughput, or within one query for low-latency analytical and large-k search.
• Scale from one core to thousands. Run embedded on one machine, then use the same Parquet index with Spark or StarRocks at cluster scale.

python -m pip install parqdb

Continue with the project repository, getting-started guide, or open index specification.

ParqDB is open source under the MIT and Apache-2.0 licenses.