Skip to content

bench: RAG cost at 10,000 files / 10M queries - #18

Merged
uurtech merged 1 commit into
masterfrom
bench/cost-10k
Oct 14, 2026
Merged

uurtech merged 1 commit into
masterfrom
bench/cost-10k

Conversation

@uurtech

@uurtech uurtech commented Oct 14, 2026

Copy link
Copy Markdown
Owner

Re-ran cost_bench.py --files 10000 --queries 10000000 (--verify green) and re-rendered the site + README.

  • Hero: $9,878 saved per 10M questions (Sonnet 5 input: $14,326 JDF vs $24,204 PDF)
  • Hero labels read the run's query volume instead of hard-coded 1M
  • Embedding time now measured on Apple MPS (was L40S); $/token rows are machine-independent

🤖 Generated with Claude Code

Re-run cost_bench.py --files 10000 --queries 10000000 (verify green),
re-render site/README. Hero labels now follow the run's query volume
instead of a hard-coded 1M; static copy says 10,000 files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@uurtech
uurtech merged commit 47d4d5e into master Oct 14, 2026
4 checks passed
@uurtech
uurtech deleted the bench/cost-10k branch October 14, 2026 11:24

This branch was successfully deployed

1 active deployment
github-pages — 47d4d5e6 Deployed Oct 14, 2026 by uurtech via deploy #147
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant