Skip to content

feat(llm): bundled AI key — hosted proxy + per-user daily quota - #2

Merged
Codehagen merged 5 commits into
mainfrom
feat/bundled-ai-key
Apr 17, 2026
Merged

Codehagen merged 5 commits into
mainfrom
feat/bundled-ai-key

Conversation

@Codehagen

@Codehagen Codehagen commented Apr 17, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • Removes the AI_GATEWAY_API_KEY signup prereq from getting-started. CLI routes LLM calls through helpbase.dev/api/v1/llm/* on the user's Supabase session token, gated by a 500k tokens/day free-tier cap plus a 10M tokens/day global circuit breaker. TTHW drops from ~10 min to ~3 min.
  • BYOK escape hatch preserved. AI_GATEWAY_API_KEY in env bypasses the proxy entirely — CI and power users keep working with zero helpbase.dev dependency.
  • Supabase migration applied (llm_proxy_quota_and_waitlist): llm_usage_events, global_daily_tokens, waitlist_signups + 2 RPCs. Types regenerated.
  • /waitlist page shipped so the 429 upgrade link isn't a 404.

Architecture

CLI ──► /api/v1/llm/generate-object
        └── parallel: verify JWT + SUM tokens today
            └── quota ≤ 500k? global < 10M? → Vercel AI Gateway
                └── log llm_usage_events + upsert global_daily_tokens
                    └── 200 {object, usage, quota}

BYOK path: AI_GATEWAY_API_KEY=... helpbase generate ... → direct Vercel AI SDK call, no helpbase round-trip.

Test plan

  • Shared: 126 tests pass (10 new for llm.ts BYOK + hosted branches, 5 for helpers/errors)
  • CLI: 333 tests pass (existing ai.unit.test.ts regression + new flow)
  • Web: 46 tests pass
  • Typecheck clean across shared, cli, create-helpbase, mcp, web
  • pnpm sync:templates clean — hosted-tier files correctly excluded from scaffolder + shadcn registry
  • pnpm registry:build clean
  • Before merge: set AI_GATEWAY_API_KEY + SUPABASE_SERVICE_ROLE_KEY in Vercel prod env (otherwise proxy returns 503)
  • After merge: run scripts/smoke-llm-proxy.sh https://helpbase.dev $HELPBASE_TOKEN to verify live
  • Manual: fresh-machine install, npm i -g helpbase@next → helpbase login → helpbase generate <url> works without any env vars

DX deltas

Before After
TTHW 10+ min ~3 min
Account signups 2 (helpbase + Vercel) 1 (helpbase)
Credit card Vercel required None
Getting-started lines 7 3

Error UX (429 example)

✖ You've used today's free allocation (500k tokens). Resets in 14h 23m.

  fix:
    • Join the waitlist for paid tier → https://helpbase.dev/waitlist
    • Or use your own Vercel AI Gateway key: export AI_GATEWAY_API_KEY=...
    • Docs: https://helpbase.dev/docs/byok

Follow-ups (not in this PR)

  • TODO-022: nightly reconciliation job for failed llm_usage_events writes (fail-open is the one silent failure mode).
  • TODO-019: hosted-parity eval via /api/v1 + HELPBASE_TOKEN.
  • TODO-023: better-auth migration (lands with Stripe PR; auth.ts:12 comment already anticipates it).
  • TODO-024: web /dashboard/usage page.

Plan: ~/.claude/plans/gentle-napping-lagoon.md
Reviews: ENG CLEAR (1 critical gap, mitigated via TODO-022), DX CLEAR (score 8/10).

🤖 Generated with Claude Code

Summary by CodeRabbit

Release Notes

  • New Features

    • Introduced helpbase login magic-link authentication for free-tier access (no card required)
    • Added hosted LLM proxy endpoints with quota tracking and daily usage limits
    • Implemented "Bring Your Own Key" (BYOK) mode to route requests through users' own Vercel AI Gateway accounts
    • Added paid-tier waitlist signup at /waitlist
    • Enhanced code block syntax highlighting with improved styling
    • Added usage today endpoint to check remaining quota
  • Documentation

    • New BYOK guide for users with existing Gateway accounts
    • Updated setup guides to use login flow instead of API key configuration
    • Simplified getting-started instructions

Remove the AI_GATEWAY_API_KEY signup prereq from getting-started. CLI now
routes LLM calls through helpbase.dev/api/v1/llm/* on the user's session
token, gated by a 500k tokens/day free-tier cap and a 10M tokens/day
global circuit breaker. BYOK stays as an env-var escape hatch for CI and
power users — AI_GATEWAY_API_KEY set bypasses the proxy entirely.

TTHW drops from ~10 min (two signups, one credit card) to ~3 min
(one magic-link login).

Server:
  - /api/v1/llm/{generate-object,generate-text} — RPC-style proxies over
    Vercel AI SDK; auth via bearer token (Supabase session); quota + cap
    pre-checked, usage + cost logged post-call.
  - /api/v1/usage/today — GET for `helpbase whoami`.
  - /waitlist — static page + server action → waitlist_signups table.

Schema (migration llm_proxy_quota_and_waitlist):
  - llm_usage_events (one row per call, RLS: owner-read only)
  - global_daily_tokens (circuit breaker counter)
  - waitlist_signups (email collection from 429 messages)
  - get_user_tokens_today(uuid) RPC
  - get_global_tokens_today() RPC

CLI:
  - packages/shared/src/{llm,llm-errors,llm-wire}.ts — one-entry-point
    client that branches on AI_GATEWAY_API_KEY.
  - ai.ts callGenerator delegates to callLlmObject, preserving
    MissingApiKeyError for existing callers.
  - context/generate/sync resolve auth inline: if no session + TTY,
    prompt for magic-link login and resume with the same args.
  - whoami shows "used X / 500k tokens (Y%)" + BYOK hint.
  - Threshold warnings at 80% and 95% for context --ask.
  - helpbase doctor checks /api/v1/llm reachable + /api/v1/usage/today.

Docs:
  - README Getting Started: 3 lines instead of 7 (drop Vercel signup step).
  - 3 copies of sync-in-5-minutes.mdx: drop AI_GATEWAY_API_KEY prereq.
  - New guides/byok.mdx (3 synced copies): when + how to bypass the
    proxy for CI, power users, privacy-sensitive workflows.
  - registry/helpbase-workflow README: AI_GATEWAY_API_KEY recommended
    for CI (BYOK), HELPBASE_TOKEN as alternative.

Tests: 10 cases for llm.ts (BYOK branch, 401/429/503/network/happy),
5 for llm-errors helpers + classes. Total suite 505 tests (was 494).

Follow-ups:
  - Vercel env: set AI_GATEWAY_API_KEY + SUPABASE_SERVICE_ROLE_KEY in
    production before merging (proxy returns 503 without them).
  - scripts/smoke-llm-proxy.sh for post-deploy verification.
  - TODO-022: nightly reconciliation for failed usage-event writes.
  - TODO-019: hosted-parity eval via /api/v1 + HELPBASE_TOKEN.

Plan: /Users/christer/.claude/plans/gentle-napping-lagoon.md
Reviews: ENG CLEAR (1 critical gap, mitigated via TODO-022), DX CLEAR
(score 8/10, TTHW 10+ min → ~3 min).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented Apr 17, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
helpbase Ready Ready Preview, Comment Apr 17, 2026 10:52am

Request Review

@coderabbitai

coderabbitai Bot commented Apr 17, 2026 •

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

Added a complete hosted LLM proxy system with authentication, per-user quota enforcement, and usage logging. Introduced CLI support for magic-link login, BYOK mode for direct Vercel AI Gateway access, and a waitlist signup page. Integrated auth tokens into existing generation functions. Updated documentation and templates to reflect new authentication flows.

Changes

Cohort / File(s) Summary
LLM Proxy API Routes
apps/web/app/api/v1/llm/_shared.ts, apps/web/app/api/v1/llm/generate-object/route.ts, apps/web/app/api/v1/llm/generate-text/route.ts, apps/web/app/api/v1/usage/today/route.ts
New authentication, quota gating, and usage logging infrastructure. Routes enforce bearer-token validation, apply per-user and global daily token limits with reset times, clamp maxOutputTokens, and log usage events to database. Shared module exports withAuthAndQuota gate, logUsageEvent, and error formatters.
Shared LLM Infrastructure
packages/shared/src/llm.ts, packages/shared/src/llm-wire.ts, packages/shared/src/llm-errors.ts
New LLM client and wire protocol layer. llm.ts provides unified callLlmObject/callLlmText dispatching between hosted proxy and BYOK (direct Gateway) paths, with isByokMode() detection and error translation. llm-wire.ts defines request/response envelopes and error codes. llm-errors.ts exports typed error classes (AuthRequiredError, QuotaExceededError, GlobalCapError, etc.) with human-readable formatting helpers (humanTokens, humanUntil, formatQuotaSuffix).
CLI Auth Integration
packages/cli/src/lib/inline-auth.ts, packages/cli/src/lib/llm-errors-cli.ts, packages/cli/src/commands/context.ts, packages/cli/src/commands/generate.ts, packages/cli/src/commands/sync.ts, packages/cli/src/commands/whoami.ts, packages/cli/src/commands/doctor.ts
Added inline authentication (resolveAuthOrPromptLogin) supporting magic-link signup and BYOK detection. Updated generation commands to resolve auth before LLM calls and thread authToken into shared generators. Enhanced error handling via toCliLlmError to translate LLM errors into user-facing guidance. Updated whoami to show usage quota and BYOK mode status. Added doctor diagnostics for LLM proxy and usage endpoints.
Shared Generation Functions
packages/shared/src/ai.ts, packages/shared/src/ai-context.ts, packages/shared/src/ai-sync.ts, packages/shared/src/ai-text.ts, packages/shared/src/ai-visual.ts
Extended GenerateHowtosInput, GenerateSyncProposalsOptions, GenerateOptions, and VisualGenerateOptions to accept optional authToken. Refactored callGenerator to dispatch via callLlmObject and return object (legacy) or CallGeneratorResult with meta containing usage/quota. Translated AuthRequiredError to MissingApiKeyError for backward compatibility.
Waitlist & Auth Helpers
apps/web/app/(main)/waitlist/page.tsx, apps/web/lib/waitlist.ts, apps/web/lib/supabase-admin.ts
New waitlist signup page and form handler. waitlist.ts exports server action validating email, inserting into waitlist_signups, and redirecting with status. supabase-admin.ts provides service-role client initialization and verifyBearerToken() for API authentication.
Database Types & Supabase Integration
apps/web/types/supabase.ts
Extended Database type with three new tables: global_daily_tokens (global quota tracking), llm_usage_events (request/token/cost logging), waitlist_signups (email + source). Added three RPC function signatures: get_user_tokens_today, get_global_tokens_today, increment_global_tokens.
MDX Rendering & Syntax Highlighting
apps/web/public/r/help-center.json, registry/helpbase/lib/mdx-config.ts, packages/create-helpbase/templates/lib/mdx-config.ts, registry/helpbase/lib/mdx-components.tsx, packages/create-helpbase/templates/lib/mdx-components.tsx, packages/create-helpbase/templates/globals.css
Centralized MDX pipeline configuration with shared Shiki-backed syntax highlighting and plugin arrays. Added h1 override to render as h2 in article bodies (preventing duplicate page-level headings). Added CSS rules for code block styling with light/dark theme variables. Extracted content directory resolution to support HELPBASE_CONTENT_DIR environment variable.
Documentation & Guides
README.md, apps/web/content/guides/byok.mdx, apps/web/content/guides/sync-in-5-minutes.mdx, packages/create-helpbase/templates/content/guides/byok.mdx, registry/helpbase/content/guides/byok.mdx, plus 14 MDX template/content files
Updated README to promote npx helpbase login magic-link flow instead of AI_GATEWAY_API_KEY as default. Added BYOK guide documenting direct Vercel AI Gateway routing via environment variable. Updated sync guide to reference login flow. Removed redundant H1 headings from MDX documents (relying on frontmatter titles).
Workflow & Config Updates
apps/web/public/r/helpbase-workflow.json, registry/helpbase-workflow/README.md, registry/helpbase-workflow/helpbase-sync.yml, scripts/sync-templates.mjs, apps/web/package.json
Updated workflow YAML to accept both AI_GATEWAY_API_KEY (BYOK) and HELPBASE_TOKEN (hosted) with precedence documentation. Added ai package (^5.0.0) to web app dependencies. Extended template sync script exclusions for hosted-only routes and libraries. Added rehype-pretty-code and shiki to template dependencies.
Error Handling & Lookup
packages/cli/src/lib/errors.ts, apps/web/app/(main)/errors/[code]/page.tsx
Extended ErrorCode union with E_AUTH_REQUIRED, E_QUOTA_EXCEEDED, E_GLOBAL_CAP, E_LLM_NETWORK, E_LLM_GATEWAY. Added error page documentation entries for new error codes.
Testing & Validation
packages/shared/src/__tests__/llm.test.ts, packages/shared/src/__tests__/llm-errors.test.ts, scripts/smoke-llm-proxy.sh, scripts/smoke-registry.sh, packages/cli/test/context.test.ts, packages/create-helpbase/templates/lib/schemas.ts, registry/helpbase/lib/schemas.ts
Added comprehensive Vitest suites for LLM client logic (hosted vs BYOK paths, error mapping), error formatting, and quota helpers. Added bash smoke test for proxy endpoint validation (auth gates, usage, happy path). Updated registry smoke test heading expectations. Added deployment/MCP schemas for hosted tier workflows.
UI & Styling
packages/cli/src/lib/ui.ts, registry/helpbase/components/tenant-search-trigger.tsx
Updated spinner fallback to use plain-text markers (>, x, ok) for non-TTY environments. Added new search trigger button component for help center.
Template Creation
packages/create-helpbase/src/index.ts, packages/create-helpbase/templates/lib/content.ts, packages/create-helpbase/templates/lib/schemas.ts
Updated interactive setup to prompt for BYOK key with guidance to use helpbase login instead. Integrated shared MDX plugin configuration. Added deployment/MCP schemas.

Sequence Diagrams

sequenceDiagram
    participant Client as CLI/Client
    participant Auth as Auth Service
    participant Gate as API Auth/Quota Gate
    participant DB as Database
    participant LLM as LLM Provider
    participant Log as Usage Logger

    rect rgba(100, 150, 200, 0.5)
    Note over Client,Log: Hosted LLM Proxy Flow
    Client->>Auth: Check session or magic-link login
    Auth-->>Client: authToken
    Client->>Gate: POST /api/v1/llm/generate-text<br/>Authorization: Bearer <authToken>
    Gate->>DB: verifyBearerToken(authToken)
    DB-->>Gate: userId
    Gate->>DB: get_user_tokens_today(userId)
    DB-->>Gate: usedToday
    alt User quota exhausted
        Gate-->>Client: 429 quota_exceeded
    else Global cap hit
        Gate-->>Client: 503 global_cap_hit
    else Auth failed
        Gate-->>Client: 401 auth_required
    else Proceed
        Gate->>LLM: generateText(model, prompt)
        LLM-->>Gate: text, promptTokens, completionTokens
        Gate->>Log: logUsageEvent(userId, tokens, cost)
        Log->>DB: INSERT llm_usage_events
        Gate->>DB: increment_global_tokens(today, tokens)
        Gate-->>Client: { text, usage, quota }
    end
    end
Loading
sequenceDiagram
    participant CLI as CLI
    participant Env as Environment
    participant Gateway as Vercel AI Gateway

    rect rgba(100, 150, 200, 0.5)
    Note over CLI,Gateway: BYOK (Direct Gateway) Flow
    CLI->>Env: Check AI_GATEWAY_API_KEY
    alt Key set
        CLI->>CLI: isByokMode() = true
        CLI->>Gateway: generateText/Object<br/>(SDK calls with AI_GATEWAY_API_KEY)
        Gateway-->>CLI: { object/text, usage }
        Note over CLI: No hosted proxy, no auth<br/>Direct billing to user's Gateway account
    else Key not set
        CLI->>CLI: isByokMode() = false
        Note over CLI: Use hosted proxy auth flow
    end
    end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~75 minutes

Poem

🐰 A hop through hosted gateways grand,
With tokens earned and quotas planned,
Magic links that sign you in,
BYOK routes where keys begin,
Syntax shines through Shiki's light—
LLM proxy, burning bright! ✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 43.21% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely summarizes the main feature: bundled AI key with hosted proxy and per-user daily quota. It is specific and directly reflects the core change.
Description check ✅ Passed The PR description covers Summary, Architecture, Test plan with specific test counts, DX improvements, error UX example, and follow-ups. All major template sections are present and substantively filled with implementation details and verification evidence.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/bundled-ai-key

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 18

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (4)
apps/web/public/r/helpbase-workflow.json (1)

9-15: ⚠️ Potential issue | 🟠 Major

Workflow YAML doesn't plumb HELPBASE_TOKEN — docs and workflow are out of sync.

The README content tells users they can add HELPBASE_TOKEN as their sole secret to use the hosted proxy, but the workflow YAML only wires AI_GATEWAY_API_KEY into the job env. A user who follows the HELPBASE_TOKEN path will end up with a workflow where the token isn't exposed to the step, and helpbase sync will fail at runtime since neither auth mechanism is available.

Add HELPBASE_TOKEN to the env section:

🔧 Suggested patch to the embedded YAML
       - name: Run helpbase sync
         env:
           AI_GATEWAY_API_KEY: ${{ secrets.AI_GATEWAY_API_KEY }}
+          HELPBASE_TOKEN: ${{ secrets.HELPBASE_TOKEN }}
         run: |
           npx -y helpbase sync \
             --since origin/main \
             --apply \
             --yes
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@apps/web/public/r/helpbase-workflow.json` around lines 9 - 15, The workflow's
"Run helpbase sync" step currently exports only AI_GATEWAY_API_KEY, so
HELPBASE_TOKEN from the README isn't available at runtime; update the step that
sets env for the "Run helpbase sync" command to also export HELPBASE_TOKEN
(environment variable HELPBASE_TOKEN) so the helpbase CLI can use the
hosted-proxy auth path when that secret is provided; keep AI_GATEWAY_API_KEY
as-is (it continues to take precedence if both are present).
packages/cli/src/commands/sync.ts (1)

57-68: ⚠️ Potential issue | 🟡 Minor

Stale help text — references the old BYOK-first flow.

The new default is helpbase login (hosted proxy, free tier). The trailing line still advertises AI_GATEWAY_API_KEY as the primary setup and will confuse users landing on helpbase sync --help.

📝 Proposed fix
 Examples:
   $ helpbase sync --demo                     # 30-second tour, no key required
   $ helpbase sync --since HEAD~5             # diff last 5 commits
   $ helpbase sync --since origin/main        # diff against tracking branch
   $ helpbase sync --apply                    # apply proposals in-place
 
-Set AI_GATEWAY_API_KEY first — https://vercel.com/ai-gateway
+Run `helpbase login` first (free 500k tokens/day), or export AI_GATEWAY_API_KEY for BYOK.
+Docs: https://helpbase.dev/docs/byok
 `,
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/cli/src/commands/sync.ts` around lines 57 - 68, The help text
appended via .addHelpText in the sync command is stale: it still instructs users
to set AI_GATEWAY_API_KEY first; update the trailing guidance in the help block
(the string passed to .addHelpText in packages/cli/src/commands/sync.ts) to
mention the new default onboarding step ("helpbase login" — hosted proxy, free
tier) and remove or rephrase the AI_GATEWAY_API_KEY-first wording; keep the
example snippets unchanged but replace the final line with a short pointer to
"helpbase login" and the hosted proxy documentation/link.
packages/cli/src/commands/context.ts (1)

77-91: ⚠️ Potential issue | 🟡 Minor

Stale help text + error copy both assume BYOK-only.

Two related issues with outdated user-facing copy after the hosted proxy lands:

  1. Line 88–89 (addHelpText): Pick one: set ANTHROPIC_API_KEY, OPENAI_API_KEY, or AI_GATEWAY_API_KEY. BYOK — helpbase calls the provider directly. First key found wins; --model overrides. — this now misrepresents the default flow (signed-in hosted proxy).
  2. Line 123 (GatewayError branch fix): "Check your AI_GATEWAY_API_KEY is valid." — for hosted users without BYOK, there is no key to check; they hit this path when the proxy's upstream gateway fails. Misleading.
✏️ Suggested copy
-Pick one: set ${pc.cyan("ANTHROPIC_API_KEY")}, ${pc.cyan("OPENAI_API_KEY")}, or ${pc.cyan("AI_GATEWAY_API_KEY")}.
-BYOK — helpbase calls the provider directly. First key found wins; --model overrides.
+Sign in with `helpbase login` (free, no credit card, 500k tokens/day).
+Or BYOK: set AI_GATEWAY_API_KEY (or ANTHROPIC_API_KEY / OPENAI_API_KEY) to call providers directly.
+First key found wins; --model overrides.

And for the GatewayError fix copy at line 121–125, consider branching on whether AI_GATEWAY_API_KEY is set so you don't tell hosted users to validate a key they don't have.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/cli/src/commands/context.ts` around lines 77 - 91, Update the stale
user-facing copy in the addHelpText call in context.ts so it reflects the new
default hosted-proxy flow (mention that signed-in hosted proxy is the default
and BYOK is optional/override via AI_GATEWAY_API_KEY or --model) instead of
implying BYOK-only; also change the GatewayError branch message (the code path
that checks/throws GatewayError) to branch based on whether
process.env.AI_GATEWAY_API_KEY is present and, if present, advise validating
that key, otherwise give a hosted-proxy-specific message (e.g., upstream gateway
failure or contact support) so users without a key aren’t told to “check your
AI_GATEWAY_API_KEY.”
packages/cli/src/commands/generate.ts (1)

59-73: ⚠️ Potential issue | 🟡 Minor

Stale help text contradicts the new hosted-default onboarding.

The addHelpText still reads "Set AI_GATEWAY_API_KEY first — get a key at https://vercel.com/ai-gateway." After this PR, helpbase login is the default path (free tier, no card), and AI_GATEWAY_API_KEY is the BYOK opt-in — not a prerequisite. Users running helpbase generate --help will be directed through an unnecessary signup step, which is exactly what this PR removes.

✏️ Suggested replacement
-Set AI_GATEWAY_API_KEY first — get a key at https://vercel.com/ai-gateway.
+Sign in with `helpbase login` (free, no credit card, 500k tokens/day).
+Or BYOK: set AI_GATEWAY_API_KEY to bypass the hosted proxy (docs: helpbase.dev/docs/byok).
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/cli/src/commands/generate.ts` around lines 59 - 73, The help copy in
the addHelpText call still instructs users to "Set AI_GATEWAY_API_KEY first",
which is stale; update the message in packages/cli/src/commands/generate.ts (the
addHelpText block) to instead instruct users to run "helpbase login" as the
default onboarding path and mention that AI_GATEWAY_API_KEY is optional (BYOK
only) — e.g., replace the current line about AI_GATEWAY_API_KEY with a short
sentence like "Run `helpbase login` to set up a free account (no card required).
If you have your own API key, set AI_GATEWAY_API_KEY to opt into BYOK." and keep
the rest of the examples intact so the help text reflects hosted-default
onboarding.
🧹 Nitpick comments (19)
packages/create-helpbase/templates/lib/schemas.ts (1)

252-306: Optional: tighten ran_at and tenant_id with Zod 4 format validators.

Two small hardening wins in the new hosted-tier schemas:

  • ran_at: z.string() accepts any string; z.iso.datetime() rejects malformed timestamps at the RPC boundary before they land in tenant_deploys.validation_report JSONB.
  • tenant_id: z.string().uuid() — in Zod 4, z.uuid() validates strictly to RFC and is the idiomatic replacement.
♻️ Proposed tweak
 export const deployReportSchema = z.object({
   kept_count: z.number().int().nonnegative(),
   dropped_count: z.number().int().nonnegative(),
   dropped: z.array(z.object({
     slug: z.string(),
     reason: z.string(),
   })).default([]),
-  ran_at: z.string(),
+  ran_at: z.iso.datetime(),
 })
 export const tenantMcpQuerySchema = z.object({
-  tenant_id: z.string().uuid(),
+  tenant_id: z.uuid(),
   tool_name: z.enum(["search_docs", "get_doc", "list_docs"]),
   query: z.string().default(""),
   result_count: z.number().int().nonnegative(),
   matched: z.boolean(),
 })

Apply the same edit to registry/helpbase/lib/schemas.ts — it carries an identical copy.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/create-helpbase/templates/lib/schemas.ts` around lines 252 - 306,
Replace the permissive string validators with Zod 4 format validators: in
deployReportSchema change ran_at from z.string() to z.iso.datetime() and in
tenantMcpQuerySchema change tenant_id from z.string().uuid() to z.uuid(); update
the corresponding type usage if any relies on the old shape and run tests/TS
checks to ensure no downstream breakage, and apply the identical edits to the
copy in registry/helpbase/lib/schemas.ts.
registry/helpbase/lib/content.ts (1)

146-149: Consider pushing the type assertion into mdx-config.ts.

The as any at the call site leaks a typing concern into every consumer (this file and the template copy). If the version mismatch is between unified/next-mdx-remote plugin types, consider exporting rehypePlugins from ./mdx-config already typed as NonNullable<Parameters<typeof compileMDX>[0]['options']>['mdxOptions']['rehypePlugins'] (or a local alias), so the cast lives in one place.

Not a blocker — the eslint-disable comment keeps the current form honest.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@registry/helpbase/lib/content.ts` around lines 146 - 149, The call site in
content.ts is using "rehypePlugins as any" which leaks typing concerns; instead,
move the cast into mdx-config.ts by exporting rehypePlugins already typed as
NonNullable<Parameters<typeof
compileMDX>[0]['options']>['mdxOptions']['rehypePlugins'] (or a local type
alias) and update content.ts to import the strongly-typed rehypePlugins without
any "as any" or eslint-disable comments; change the export in ./mdx-config to
perform the single type assertion so all consumers (including content.ts and the
template copy) receive the correctly typed rehypePlugins.
scripts/sync-templates.mjs (1)

64-74: Inconsistent matching semantics between templates and registry exclude logic.

The templates loop (line 528) matches HOSTED_TIER_EXCLUDES entries with startsWith, but the registry lib walk (line 944) uses .includes(\lib/${f}`)(exact match). The new entrieslib/supabase-admin.tsandlib/waitlist.tshappen to be files, so both paths work today — but if the waitlist grows into a directory (e.g.lib/waitlist/index.ts), only the templates target will exclude it and the registry will silently start shipping Supabase/waitlist code. Consider normalizing both sites to startsWith`.

♻️ Proposed unification
-  const libFiles = walkFiles(join(APPS_WEB, "lib")).filter(
-    (f) => !f.endsWith(".gitkeep") && !HOSTED_TIER_EXCLUDES.includes(`lib/${f}`),
-  )
+  const libFiles = walkFiles(join(APPS_WEB, "lib")).filter(
+    (f) => !f.endsWith(".gitkeep") &&
+      !HOSTED_TIER_EXCLUDES.some((p) => `lib/${f}`.startsWith(p)),
+  )

Also applies to: 943-945

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@scripts/sync-templates.mjs` around lines 64 - 74, The registry exclusion
logic is using includes(`lib/${f}`) which mismatches the templates loop that
uses HOSTED_TIER_EXCLUDES with startsWith, causing directory-shape paths (e.g.,
lib/waitlist/index.ts) to slip through; update the registry walk check (the code
that iterates registry libs around the walk at ~944 and the related checks at
943-945) to use startsWith against the same HOSTED_TIER_EXCLUDES entries (or
normalize both sides to use startsWith) so exclusions behave consistently for
both file and directory paths; ensure you reference HOSTED_TIER_EXCLUDES when
performing the startsWith checks.
apps/web/lib/waitlist.ts (1)

14-32: Consider rate limiting this endpoint.

Server actions are publicly callable; without a rate limit an attacker can flood waitlist_signups with unique addresses (inflating list, enabling enumeration via 23505 timing). Even a simple IP-based throttle at the edge would help.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@apps/web/lib/waitlist.ts` around lines 14 - 32, Add a simple rate limit check
at the top of joinWaitlist to prevent automated floods: extract the caller IP
(e.g. from X-Forwarded-For or the platform request context) and consult a
rate-limiter store (in-memory LRU or Redis) to allow only N submissions per IP
per time window; if the IP is over the limit, log the event and redirect to
"/waitlist?status=throttled". Optionally also add a per-email throttle and/or
CAPTCHA fallback before calling getServiceRoleClient().from("waitlist_signups")
so the waitlist_signups inserts are protected from high‑volume or enumeration
attacks.
registry/helpbase/lib/mdx-config.ts (1)

56-65: Nit: highlighter cache isn't error-resilient.

If createHighlighter rejects once, the failed promise sticks in highlighterPromise and every subsequent MDX compile will see the same rejection with no retry. Low probability (Shiki init is deterministic), but a two-line guard would make cold-start flakiness self-healing:

Proposed fix
 function getSharedHighlighter(): Promise<Highlighter> {
   if (!highlighterPromise) {
-    highlighterPromise = createHighlighter({
+    const p = createHighlighter({
       themes: [...SHIKI_THEMES],
       langs: [...SHIKI_LANGS],
     })
+    p.catch(() => {
+      highlighterPromise = null
+    })
+    highlighterPromise = p
   }
   return highlighterPromise
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@registry/helpbase/lib/mdx-config.ts` around lines 56 - 65, The
getSharedHighlighter() caches a failing Promise in highlighterPromise so a
rejected createHighlighter(...) prevents future retries; update
getSharedHighlighter to reset highlighterPromise to null on error (e.g., catch
the rejection from createHighlighter or await inside try/catch) and rethrow the
error so transient failures can be retried; reference highlighterPromise,
getSharedHighlighter, and createHighlighter when applying the change.
apps/web/app/(main)/waitlist/page.tsx (1)

10-25: Simplify: make the page itself async instead of the sync-wrapper + async-child split.

Next.js 15 supports async server page components directly, so the indirection through WaitlistContent isn't needed:

Proposed simplification
-export default function WaitlistPage({
-  searchParams,
-}: {
-  searchParams?: Promise<{ status?: string; from?: string }>
-}) {
-  return <WaitlistContent searchParamsP={searchParams} />
-}
-
-async function WaitlistContent({
-  searchParamsP,
-}: {
-  searchParamsP?: Promise<{ status?: string; from?: string }>
-}) {
-  const sp = (await searchParamsP) ?? {}
+export default async function WaitlistPage({
+  searchParams,
+}: {
+  searchParams?: Promise<{ status?: string; from?: string }>
+}) {
+  const sp = (await searchParams) ?? {}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@apps/web/app/`(main)/waitlist/page.tsx around lines 10 - 25, The page
component uses a synchronous WaitlistPage that forwards a Promise prop to an
async WaitlistContent wrapper — simplify by making WaitlistPage itself async and
removing the WaitlistContent indirection: change export default function
WaitlistPage to an async function, accept searchParams?: { status?: string;
from?: string } (not Promise), await or directly use searchParams (depending on
runtime), compute sp/status/from as currently done (sp = searchParams ?? {},
status = sp.status, from = sp.from ?? "waitlist"), and delete the
WaitlistContent function so rendering happens directly in the async
WaitlistPage.
packages/cli/src/commands/doctor.ts (1)

351-392: Consider treating non-2xx/401 as info rather than warn to reduce noise.

warn on a transient 5xx from /api/v1/usage/today will fire any time the backend blips — users running doctor when things are actually fine will see yellow. A narrower classification (e.g., ok on 2xx, error only on 401 with re-login fix, warn otherwise) would match the tone of the existing checkApiReachable helper. Optional.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/cli/src/commands/doctor.ts` around lines 351 - 392, In
checkUsageEndpoint, change the severity classification for non-2xx responses:
keep severity "ok" when res.ok, make status === 401 return severity "error" (and
keep the re-login fix message), and change all other non-2xx responses from
"warn" to "info" so transient 5xxs are less noisy; update the returned object in
the else branch of checkUsageEndpoint (the block that currently returns label
"usage read", category "network", severity "warn" / fix logic) to implement this
new mapping.
packages/cli/src/commands/whoami.ts (1)

102-115: Defensive guard against dailyLimit === 0.

Math.round((usedToday / dailyLimit) * 100) produces NaN if dailyLimit is ever 0 (e.g., a future quota-pause state from the server), and the text rendering would print NaN%. DAILY_USER_LIMIT is a positive constant today so it's not an active bug, but since the value arrives off-the-wire via fetchUsageToday, a cheap guard avoids a future surprise.

♻️ Suggested guard
-      const { usedToday, dailyLimit, resetAt } = usage.quota
-      const pct = Math.round((usedToday / dailyLimit) * 100)
+      const { usedToday, dailyLimit, resetAt } = usage.quota
+      const pct = dailyLimit > 0 ? Math.round((usedToday / dailyLimit) * 100) : 0
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/cli/src/commands/whoami.ts` around lines 102 - 115, The pct
calculation in whoami.ts can produce NaN when dailyLimit is 0; update the block
that reads const { usedToday, dailyLimit, resetAt } = usage.quota and the pct
computation to defensively handle dailyLimit === 0 (e.g., set pct = dailyLimit >
0 ? Math.round((usedToday / dailyLimit) * 100) : 0 or display “—”/“0%”) so the
console output never prints NaN%; ensure the human-readable line that uses pct
still formats correctly when you choose the fallback.
apps/web/app/api/v1/llm/generate-text/route.ts (2)

80-99: Happy-path usage logging + quota snapshot LGTM.

Ordering (log after SDK call, compute quota with usedToday + totalTokens) matches the stated architecture and tolerates the log failing without 500ing the user. One observability nit: consider logging latency/tokens/cost to your metrics pipeline in addition to llm_usage_events so you can alert on gateway-error rate without scanning the DB — but that's a follow-up, not a blocker.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@apps/web/app/api/v1/llm/generate-text/route.ts` around lines 80 - 99, Add
emitting structured metrics to the metrics pipeline for latency, promptTokens,
completionTokens, total cost, and status immediately after computing latency and
usage (after wireUsageFromSdk and before/after logUsageEvent) so observability
doesn't rely solely on llm_usage_events; update the generate-text flow (look for
functions/vars: wireUsageFromSdk, latency, usage, logUsageEvent,
GenerateTextResponse, quotaSnapshot) to call your metrics client (e.g.,
metrics.increment / metrics.histogram) with tags for route="generate-text",
model=body.model, userId and status="ok" and include latencyMs, promptTokens,
completionTokens, and costUsd. Ensure metric emission is non-blocking and
failures are caught/logged but do not change the existing DB logging or response
path.

62-78: Gateway-error path is well structured — but the raw message may include upstream PII/secrets.

Truncating to 500 chars is good hygiene, but provider error bodies occasionally echo back request fragments (prompt excerpts, model params, and in rare cases API-key tails in "invalid key 'sk-...'" strings). Since rawPreview flows back to the CLI and is user-visible, consider stripping obvious patterns (bearer tokens, sk_, sk-, email addresses) before truncation, or sending a generic "check the gateway dashboard" message and logging the raw one server-side only. Not a current bug — defensive for the hosted-proxy's posture.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@apps/web/app/api/v1/llm/generate-text/route.ts` around lines 62 - 78, The
gateway-error handler in route.ts currently returns rawPreview derived from the
provider error message which may leak upstream PII or API keys; update the catch
block (the code that calls logUsageEvent and wireError) to sanitize the error
message before slicing and returning by stripping or redacting common
secrets/patterns (e.g., Bearer tokens, "sk_", "sk-", email addresses, long
hex/API-key-like strings) or replace with a generic user-facing string like "See
server logs for details", and ensure the full raw error is still recorded
server-side (e.g., in process logs or the existing logUsageEvent context) for
debugging; reference the existing identifiers logUsageEvent and wireError to
locate and change the returned rawPreview value.
scripts/smoke-llm-proxy.sh (1)

66-72: curl -f on the usage endpoint loses the response body on error.

With -f, curl exits non-zero on HTTP ≥ 400 and discards the response body, so the fail "usage endpoint error: $USAGE_JSON" message will only ever contain curl's stderr (e.g., "The requested URL returned error: 503"), not the server's error JSON. For a smoke script that's fine, but if you want the server's wireError body surfaced when diagnostics are needed, capture both body and status separately (e.g., write body to a tempfile and HTTP code via -w "%{http_code}").

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@scripts/smoke-llm-proxy.sh` around lines 66 - 72, The current USAGE_JSON
assignment uses curl -f which causes curl to exit non-zero and discard the
response body on HTTP errors; change the call that sets USAGE_JSON so you
capture both the response body and the HTTP status code (e.g., write the body to
a temp file and use curl -w "%{http_code}" to capture the status) rather than
relying on -f, then check the status and include the saved body in the fail
"usage endpoint error: $USAGE_JSON" message; update references to USAGE_JSON,
the fail invocation, and the subsequent grep checks to read from the saved body
when the status indicates an error and only proceed to grep the JSON fields if
the HTTP status is 2xx.
packages/cli/src/lib/inline-auth.ts (1)

78-99: Inline OTP errors bubble raw from sendLoginCode / verifyLoginCode.

If the user fat-fingers the code or the OTP expires, the thrown error from verifyLoginCode (and any network failure from sendLoginCode) escapes directly to the command's top-level handler rather than being wrapped into a HelpbaseError with a "try again" hint. Given this prompt is the critical first-run moment, a one-retry loop or a translated error would materially improve the TTHW story this PR targets. Not blocking.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/cli/src/lib/inline-auth.ts` around lines 78 - 99, Wrap network/OTP
errors from sendLoginCode and verifyLoginCode so they become user-friendly
HelpbaseError instances and add a single retry attempt for entering the 6-digit
code; specifically, catch errors thrown by sendLoginCode(email) and
verifyLoginCode(email, code) inside the inline-auth flow around the email/code
prompts, convert them to HelpbaseError (including a concise "try again" hint and
using authRequiredError(opts.retryCommand) where appropriate), and implement a
loop that on a verification failure prompts for the code one more time before
propagating the error. Ensure you reference and update the blocks around
sendLoginCode, verifyLoginCode, the code prompt, and the authRequiredError usage
so the user sees a friendly retry message instead of raw exceptions.
packages/cli/src/commands/generate.ts (1)

676-685: buildRetryCommand — double-quoting of --title may double-escape on re-run.

JSON.stringify(opts.title) emits double quotes, so the rebuilt command looks like helpbase generate --title "How to invite". If the user copies and pastes that string into their shell, it works; but if any code path passes it back into process.exec or echoes it through another JSON.stringify, you'll get nested escaping. Not currently a problem — just worth noting since this value shows up in user-facing error copy.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/cli/src/commands/generate.ts` around lines 676 - 685, The command
builder buildRetryCommand currently uses JSON.stringify(opts.title) which
injects quoted/json-escaped text and can cause double-escaping when the command
is re-used; change the behavior to push the raw title string
(parts.push("--title", opts.title)) so the title is not JSON-quoted, ensuring
the rebuilt command uses the plain user-provided title and avoiding nested
escaping when re-executed or re-serialized.
packages/shared/src/llm.ts (3)

132-190: maxOutputTokens truthy check drops legitimate zero — minor brittleness.

...(maxOutputTokens ? { maxOutputTokens } : {}) excludes 0, which in practice isn't a meaningful cap, but also excludes the more common typed-default case where callers pass 0 by mistake. Prefer an explicit != null check so only undefined/null skip the field:

-      ...(maxOutputTokens ? { maxOutputTokens } : {}),
+      ...(maxOutputTokens != null ? { maxOutputTokens } : {}),

Same pattern appears in byokGenerateText (line 201), hostedGenerateObject (line 237), and hostedGenerateText (line 263).

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/shared/src/llm.ts` around lines 132 - 190, The current truthy check
...(maxOutputTokens ? { maxOutputTokens } : {}) drops valid zero values; update
all such spread checks in byokGenerateObject, byokGenerateText,
hostedGenerateObject, and hostedGenerateText to only omit the field when
maxOutputTokens is null or undefined (use an explicit != null check) so a passed
0 is preserved when building the options for generateObject/generateText.

110-128: fetchUsageToday bypasses the wire error envelope.

Unlike /llm/* calls, this endpoint doesn't route through throwOnWireError, so a 429/503/5xx from /usage/today surfaces as a generic GatewayError("usage endpoint returned N") and loses the structured WireErrorBody. If /usage/today ever returns quota_exceeded, global_cap_hit, or any typed failure, the CLI can't distinguish it. Consider reusing throwOnWireError(res) here for consistency and forward-compat with new error codes.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/shared/src/llm.ts` around lines 110 - 128, fetchUsageToday currently
converts non-OK responses into a generic GatewayError and skips the structured
wire error parsing; update fetchUsageToday to call the shared
throwOnWireError(res) after receiving the Response (before throwing
AuthRequiredError/other errors) so quota_exceeded/global_cap_hit and other
WireErrorBody cases are preserved and surfaced consistently (ensure
throwOnWireError is imported/available in the module and remove or replace the
current generic GatewayError handling in fetchUsageToday accordingly).

340-348: zodToJsonSchema — simplify to direct Zod v4 API call.

The repo pins zod to 4.3.6, where z.toJSONSchema is the official first-party API. The defensive type cast and runtime check are unnecessary. Simplify to:

export function zodToJsonSchema(schema: z.ZodType): unknown {
-  const maybeFn = (z as unknown as { toJSONSchema?: (s: z.ZodType) => unknown }).toJSONSchema
-  if (typeof maybeFn === "function") return maybeFn(schema)
-  throw new Error(
-    "zod.toJSONSchema is not available — upgrade zod to v4+ or swap the shared llm module to accept pre-serialized JSON Schema.",
-  )
+  return z.toJSONSchema(schema)
}

The wrapper function still provides value as a stable abstraction point for future Zod migrations or swaps.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/shared/src/llm.ts` around lines 340 - 348, The zodToJsonSchema
function currently performs an unnecessary defensive cast and runtime check for
z.toJSONSchema; since the project pins Zod v4 (where z.toJSONSchema is stable),
replace the conditional wrapper with a direct call to z.toJSONSchema(schema)
inside zodToJsonSchema and remove the type-cast/maybeFn logic, keeping the
function as the single abstraction point (refer to zodToJsonSchema,
z.toJSONSchema, and z.ZodType to locate the code).
apps/web/app/api/v1/llm/generate-object/route.ts (1)

92-110: rawPreview type check is redundant.

message is already typed as string on line 94 (either err.message or the fallback string literal), so typeof message === "string" is always true. You can simplify to const rawPreview = message.slice(0, 500).

Proposed simplification
-    const message = err instanceof Error ? err.message : "Unknown gateway error"
-    const rawPreview = typeof message === "string" ? message.slice(0, 500) : undefined
+    const message = err instanceof Error ? err.message : "Unknown gateway error"
+    const rawPreview = message.slice(0, 500)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@apps/web/app/api/v1/llm/generate-object/route.ts` around lines 92 - 110, The
current catch block creates a redundant type check before slicing the error
message: replace the conditional assignment for rawPreview with a direct slice
since message is always a string (const rawPreview = message.slice(0, 500));
update the catch block around rawPreview in route.ts (the error handler that
calls logUsageEvent and wireError in generate-object) to use this simplified
expression and leave the rest (logUsageEvent call and wireError invocation)
unchanged.
apps/web/app/api/v1/llm/_shared.ts (2)

51-98: Quota TOCTOU: concurrent requests can each observe usedToday < limit and collectively blow past it.

The gate reads get_user_tokens_today and get_global_tokens_today separately from the event insert, so N parallel requests from the same user all see the pre-call counter and all pass the check. In the worst case, a user can issue ~parallelism × PER_CALL_CEILING tokens above the 500k/day cap before the first event lands. The header comment says "Overrun is accepted (same fire-and-forget pattern as apps/web/lib/rate-limit.ts)," which is fine as a design choice, but consider:

  • Bounding worst-case with a single-statement reservation RPC (reserve_tokens(p_user, p_request_cap)) that does the check + increment atomically and returns the clamped cap.
  • Or at minimum documenting the expected max overrun so operators know what to alert on.

Pairs with the critical global-counter bug above — once that's fixed, this per-user race becomes the dominant quota-bypass vector.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@apps/web/app/api/v1/llm/_shared.ts` around lines 51 - 98, The current TOCTOU
bug: reading get_user_tokens_today and get_global_tokens_today separately allows
N concurrent requests to each pass the check and exceed DAILY_USER_LIMIT; fix by
replacing the separate reads+checks with a single atomic reservation RPC call
(e.g., add and call a new reserve_tokens(p_user_id, p_request_cap) procedure via
getServiceRoleClient().rpc) that checks the current counter, increments it by
the requested amount (clamping to DAILY_USER_LIMIT and returning the allowed
cap), and use that returned clamped cap as maxOutputTokens (instead of computing
remaining and Math.min with PER_CALL_CEILING); if you cannot add the RPC
immediately, at minimum document the worst-case overrun in this function and
callers and limit PER_CALL_CEILING accordingly.

124-145: llm_usage_events insert failure is logged but not reconciled — confirm TODO-022 is tracked.

The fail-open logging path is reasonable for keeping the response alive, but with the current code a dropped insert means the user's usedToday snapshot returned in the response is still incremented (client-side) while the DB-side get_user_tokens_today is not. On the next call, the server re-reads a lower usedToday, so quota accounting silently drifts. Please make sure the referenced TODO-022 (nightly reconciliation from the PR follow-ups) is filed as a tracked issue, not just a comment — this matters for billing accuracy once the paid tier lands.

Want me to open a follow-up issue for the reconciliation job referencing TODO-022?

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@apps/web/app/api/v1/llm/_shared.ts` around lines 124 - 145, The insert
failure path for llm_usage_events (inside the insert block that logs insertErr)
can cause quota drift because get_user_tokens_today and the returned usedToday
diverge; open a tracked issue in your project tracker for the planned nightly
reconciliation (replace TODO-022 with that issue number/URL) and update the
comment to reference that issue ID; additionally, add a minimal durable fallback
by enqueuing the failed payload (same shape as the insert) to a retry queue or
table (e.g., llm_usage_events_retry) so failed inserts can be reprocessed by the
reconciliation job — update the code around the insertErr handling and document
the reconciliation job name (e.g., reconcile_llm_usage_events) and
get_user_tokens_today/getUserTokensToday usages so reviewers can locate the
logic.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@apps/web/app/`(main)/waitlist/page.tsx:
- Line 25: The hidden form value "from" (const from = sp.from ?? "waitlist") is
taken from the URL and passed into joinWaitlist where it is inserted into
waitlist_signups without checks; update joinWaitlist to validate and sanitize
the incoming source: enforce a max length (e.g. <= 100), verify allowed
characters with a regex or allowlist (e.g. alphanumerics, hyphen, underscore,
and dots), and reject or normalize values that fail (return an error or set to a
safe default like "waitlist"); ensure the validation runs before any database
insert into waitlist_signups and reference the same parameter name used in the
form/handler to locate where to add the checks.

In `@apps/web/app/api/v1/llm/_shared.ts`:
- Line 163: Remove the dead "noop" RPC call by deleting the await
client.rpc("get_global_tokens_today") line and its "noop to keep the client
warm" comment; do not replace it with any silent/error-swallowing call — if you
intended to check connectivity, perform an explicit, handled health/check call
instead and surface or log any errors, otherwise just remove the
client.rpc("get_global_tokens_today") invocation from this module.
- Around line 147-173: The upsert against global_daily_tokens currently
overwrites the daily counter (onConflict: "day" with payload), so concurrent
requests stomp the accumulated tokens and the global circuit breaker is
bypassed; replace this client-side upsert+fallback logic in the handler with an
atomic server-side increment RPC (create a SQL function
increment_global_tokens(p_day date, p_delta bigint) that INSERT ... ON CONFLICT
DO UPDATE SET tokens = global_daily_tokens.tokens + excluded.tokens, updated_at
= now()) and call that RPC (similar to get_global_tokens_today) instead of using
client.from("global_daily_tokens").upsert(...) or currentGlobalTokens(today),
and remove the read-modify-write fallback to ensure atomic increments used by
withAuthAndQuota.

In `@apps/web/app/api/v1/llm/generate-object/route.ts`:
- Around line 29-32: The comment claiming we “allow generous decoding on the
route” is misleading because exporting dynamic = "force-dynamic" only disables
static optimization and does not increase Vercel request body limits; update the
comment or code to reflect the truth: either remove or reword the comment next
to the exports (runtime and dynamic) to say it only forces dynamic rendering, or
implement an actual platform-level solution (e.g., move large base64 image
uploads to a signed-URL upload flow or adjust Vercel function limits in
vercel.json) and reference that change in the comment; look for the exports
named runtime and dynamic in route.ts when making the edit.
- Around line 84-89: The server currently converts body.schema via
jsonSchemaToZod which yields z.any() and loses model-side schema guidance;
update the generateObject calls (the one using schema:
jsonSchemaToZod(body.schema) and the similar block at lines ~134–147) to pass
the raw JSON Schema through the Vercel AI SDK helper instead by importing
jsonSchema from 'ai' and replacing schema: jsonSchemaToZod(body.schema) with
schema: jsonSchema(body.schema) so the model receives the original schema
constraints while leaving client-side re-parsing intact.

In `@apps/web/app/api/v1/llm/generate-text/route.ts`:
- Around line 40-48: Update input validation to reject an empty messages array
and defend against non-positive token budgets: change the check that uses
body.prompt/body.messages to explicitly require either a non-empty prompt or a
messages array with length > 0 (e.g., use Array.isArray(body.messages) and
messages.length > 0) so an empty [] is treated as missing; then compute
requested = body.maxOutputTokens ?? PER_CALL_CEILING and maxOutputTokens =
Math.min(requested, remainingBudget) as before but add a defensive guard that
returns wireError(429, "quota_exceeded", "...") if maxOutputTokens <= 0 before
calling the SDK to ensure quota cannot be bypassed by edge cases.

In `@apps/web/content/guides/sync-in-5-minutes.mdx`:
- Around line 46-47: The "10 full generates / 250 context --ask calls" breakdown
lacks provenance; verify whether these estimates are derived from measured
averages and either (A) add a brief calculation and citation next to the
statement in sync-in-5-minutes.mdx (show how 500,000 tokens -> 10 full generates
or 250 context --ask calls was computed), or (B) centralize the string and
calculation into a single source-of-truth used by sync-in-5-minutes.mdx,
byok.mdx, and waitlist/page.tsx (e.g., a shared constant or function), and
update those files to reference that source so the estimate stays consistent
when average tokens-per-call change. Include the unique tokens/strings ("500,000
tokens", "10 full generates", "250 context --ask calls") in the change to make
the mapping explicit.

In `@apps/web/lib/waitlist.ts`:
- Line 27: Remove the plain user email from logs in apps/web/lib/waitlist.ts:
update the console.error call (the one referencing email, source, error.message)
to drop the raw email (or replace it with a non-identifying hash if you must
correlate) and log only non-PII fields such as source plus error.code and
error.message; ensure the log message still includes context (e.g., "[waitlist
insert failed]") and use the existing error variable names (error.message /
error.code) so the change is minimal and localized to the console.error call.

In `@packages/cli/src/lib/errors.ts`:
- Around line 51-55: Add entries for the five new error codes to the ERRORS
object: add keys "e-auth-required", "e-quota-exceeded", "e-global-cap",
"e-llm-network", and "e-llm-gateway" with appropriate title/message/next-steps
text so the error docs resolve instead of 404; update any helper that maps the
TS enum members E_AUTH_REQUIRED, E_QUOTA_EXCEEDED, E_GLOBAL_CAP, E_LLM_NETWORK,
and E_LLM_GATEWAY to those string keys (ensure the mapping used by the errors
page uses these new entries). Also correct the docblock reference at the top of
errors.ts that currently points to the wrong errors page path so it reflects the
actual errors page location used by the site.

In `@packages/cli/src/lib/inline-auth.ts`:
- Around line 14-56: The code and JSDoc disagree: resolveAuthOrPromptLogin
currently falls through to an interactive TTY prompt when getCurrentSession()
returns null, but the JSDoc says a set-but-invalid HELPBASE_TOKEN should trigger
the CI error path; update resolveAuthOrPromptLogin to enforce that by adding a
check after the existing getCurrentSession() null check: if
process.env.HELPBASE_TOKEN is defined then throw
authRequiredError(opts.retryCommand) (same behavior as non-interactive),
referencing resolveAuthOrPromptLogin, getCurrentSession, HELPBASE_TOKEN, and
authRequiredError; alternatively, if the current TTY-first behavior is desired,
remove/update the JSDoc bullet about HELPBASE_TOKEN to match.

In `@packages/create-helpbase/src/index.ts`:
- Around line 323-326: The fallback hint string that currently suggests
"helpbase login" and "export AI_GATEWAY_API_KEY=..." is inconsistent with the
interactive scaffold prompt that still asks for an AI gateway key and shows a
Vercel signup URL; update the interactive prompt copy so both places match the
"no card, just helpbase login" flow by removing the Vercel signup reference and
replacing the AI_GATEWAY_API_KEY prompt text with guidance to run `helpbase
login` (or show the same BYOK wording used in the fallback hint). Locate the
interactive prompt block (the code that prompts for the AI gateway key / Vercel
signup) and the fallback template literal containing `helpbase login`, `helpbase
generate --url ${url}`, and `export AI_GATEWAY_API_KEY=...` and make their
messages consistent.

In `@packages/create-helpbase/templates/components/tenant-search-trigger.tsx`:
- Around line 20-28: Replace the synthetic KeyboardEvent dispatch in the onClick
handler with a direct imperative opener from the search dialog context:
introduce or import a hook such as useSearchDialog() (or openSearchDialog()) in
tenant-search-trigger.tsx, call its open() (or show()/openSearch()) method
inside the onClick instead of dispatching new KeyboardEvent, and remove the
KeyboardEvent creation; ensure the hook name you use matches the existing dialog
API and add the import at the top of the component.

In `@packages/create-helpbase/templates/lib/rate-limit.ts`:
- Around line 102-110: extractClientIp currently returns the string "unknown"
which collapses all unresolved clients into one rl:ip:unknown bucket; change
extractClientIp(Request) to return null (or undefined) when neither header is
present instead of "unknown", and update the rate-limit logic that calls
extractClientIp (the code building per-IP keys in rate limiter) to skip/omit the
per-IP key when the IP is null so the per-tenant bucket is used instead; apply
the same change in the other copy of rate-limit.ts referenced in the review.
- Around line 43-61: The enforceIpRateLimit function has a race where kv.incr
can create a key without TTL if kv.expire fails, causing permanent lockout;
change the logic to atomically create the key with TTL before incrementing (use
a SET key 0 EX IP_WINDOW_SECONDS NX or equivalent atomic API) then call kv.incr
so the TTL is guaranteed; also handle ttl responses explicitly (treat ttl <= 0
as missing TTL and set retryAfterSeconds to IP_WINDOW_SECONDS or reapply TTL)
instead of Math.max(1, ttl); apply the same fix to the corresponding
enforceIpRateLimit implementations in apps/web/lib/rate-limit.ts and
registry/helpbase/lib/rate-limit.ts and ensure any error paths still fail open
by returning { ok: true } only after attempting the safer atomic-set pattern.
- Around line 93-99: The unawaited Supabase update using
supabaseClient.from("tenants").update(...) can be dropped on Vercel serverless;
change the fire-and-forget call in the function that updates mcp_calls_today
(referencing supabaseClient and tenantId in rate-limit.ts) to either await the
update Promise so the invocation stays alive, or hand it off to waitUntil(...)
from `@vercel/functions` to run in the background; apply the same change to the
equivalent functions in registry/helpbase/lib/rate-limit.ts and
apps/web/lib/rate-limit.ts.

In `@packages/shared/src/llm-errors.ts`:
- Around line 89-109: formatQuotaSuffix and humanUntil need input validation to
avoid divide-by-zero/NaN outputs: in formatQuotaSuffix (function
formatQuotaSuffix, type WireQuotaStatus) guard q.dailyLimit <= 0 (or missing)
and q.usedToday negative by clamping them and compute pct only when dailyLimit >
0, then clamp pct to 0–100 and produce a sensible fallback string (e.g., "0%" or
"unknown") when division isn't possible; in humanUntil (function humanUntil)
validate the parsed date (check isNaN(ms) after new Date(iso).getTime()) and
return a safe token like "now" or "unknown" instead of formatting invalid
hours/minutes. Ensure callers still use humanTokens for numeric formatting but
do not rely on humanUntil or pct producing NaN/Infinity.

In `@registry/helpbase-workflow/README.md`:
- Around line 25-36: The README promises a HELPBASE_TOKEN path but the workflow
only forwards AI_GATEWAY_API_KEY; update the workflow (helpbase-sync.yml) to
also export HELPBASE_TOKEN into the run step's env (so HELPBASE_TOKEN is
available to the action) and correct the README instructions to show how to
retrieve the token after helpbase login (point to the stored session in
~/.helpbase/auth.json and show the JSON key to extract, rather than the
non-existent helpbase config get session.access_token), or alternately remove
the HELPBASE_TOKEN option from the README if you choose not to support it; key
symbols to edit: HELPBASE_TOKEN, AI_GATEWAY_API_KEY, helpbase-sync.yml, and the
README retrieval instructions referencing helpbase login and
~/.helpbase/auth.json.

In `@registry/helpbase/content/guides/byok.mdx`:
- Around line 89-98: The "Costs" section incorrectly implies two different
Gemini models; update the paragraph describing Gemini so it reflects that
DEFAULT_MODEL and TEST_MODEL are both "google/gemini-3.1-flash-lite-preview"
(referenced as DEFAULT_MODEL and TEST_MODEL in packages/shared/src/ai.ts) —
change "Gemini 3.1 Flash (the default)" to "Gemini 3.1 Flash Lite (the default)"
or explicitly state both default and --test use Gemini Flash Lite and that cost
variance is due to token usage; also adjust the Claude Sonnet 4.6 sentence to
note it is only available as a --model override and not a recommended default.

---

Outside diff comments:
In `@apps/web/public/r/helpbase-workflow.json`:
- Around line 9-15: The workflow's "Run helpbase sync" step currently exports
only AI_GATEWAY_API_KEY, so HELPBASE_TOKEN from the README isn't available at
runtime; update the step that sets env for the "Run helpbase sync" command to
also export HELPBASE_TOKEN (environment variable HELPBASE_TOKEN) so the helpbase
CLI can use the hosted-proxy auth path when that secret is provided; keep
AI_GATEWAY_API_KEY as-is (it continues to take precedence if both are present).

In `@packages/cli/src/commands/context.ts`:
- Around line 77-91: Update the stale user-facing copy in the addHelpText call
in context.ts so it reflects the new default hosted-proxy flow (mention that
signed-in hosted proxy is the default and BYOK is optional/override via
AI_GATEWAY_API_KEY or --model) instead of implying BYOK-only; also change the
GatewayError branch message (the code path that checks/throws GatewayError) to
branch based on whether process.env.AI_GATEWAY_API_KEY is present and, if
present, advise validating that key, otherwise give a hosted-proxy-specific
message (e.g., upstream gateway failure or contact support) so users without a
key aren’t told to “check your AI_GATEWAY_API_KEY.”

In `@packages/cli/src/commands/generate.ts`:
- Around line 59-73: The help copy in the addHelpText call still instructs users
to "Set AI_GATEWAY_API_KEY first", which is stale; update the message in
packages/cli/src/commands/generate.ts (the addHelpText block) to instead
instruct users to run "helpbase login" as the default onboarding path and
mention that AI_GATEWAY_API_KEY is optional (BYOK only) — e.g., replace the
current line about AI_GATEWAY_API_KEY with a short sentence like "Run `helpbase
login` to set up a free account (no card required). If you have your own API
key, set AI_GATEWAY_API_KEY to opt into BYOK." and keep the rest of the examples
intact so the help text reflects hosted-default onboarding.

In `@packages/cli/src/commands/sync.ts`:
- Around line 57-68: The help text appended via .addHelpText in the sync command
is stale: it still instructs users to set AI_GATEWAY_API_KEY first; update the
trailing guidance in the help block (the string passed to .addHelpText in
packages/cli/src/commands/sync.ts) to mention the new default onboarding step
("helpbase login" — hosted proxy, free tier) and remove or rephrase the
AI_GATEWAY_API_KEY-first wording; keep the example snippets unchanged but
replace the final line with a short pointer to "helpbase login" and the hosted
proxy documentation/link.

---

Nitpick comments:
In `@apps/web/app/`(main)/waitlist/page.tsx:
- Around line 10-25: The page component uses a synchronous WaitlistPage that
forwards a Promise prop to an async WaitlistContent wrapper — simplify by making
WaitlistPage itself async and removing the WaitlistContent indirection: change
export default function WaitlistPage to an async function, accept searchParams?:
{ status?: string; from?: string } (not Promise), await or directly use
searchParams (depending on runtime), compute sp/status/from as currently done
(sp = searchParams ?? {}, status = sp.status, from = sp.from ?? "waitlist"), and
delete the WaitlistContent function so rendering happens directly in the async
WaitlistPage.

In `@apps/web/app/api/v1/llm/_shared.ts`:
- Around line 51-98: The current TOCTOU bug: reading get_user_tokens_today and
get_global_tokens_today separately allows N concurrent requests to each pass the
check and exceed DAILY_USER_LIMIT; fix by replacing the separate reads+checks
with a single atomic reservation RPC call (e.g., add and call a new
reserve_tokens(p_user_id, p_request_cap) procedure via
getServiceRoleClient().rpc) that checks the current counter, increments it by
the requested amount (clamping to DAILY_USER_LIMIT and returning the allowed
cap), and use that returned clamped cap as maxOutputTokens (instead of computing
remaining and Math.min with PER_CALL_CEILING); if you cannot add the RPC
immediately, at minimum document the worst-case overrun in this function and
callers and limit PER_CALL_CEILING accordingly.
- Around line 124-145: The insert failure path for llm_usage_events (inside the
insert block that logs insertErr) can cause quota drift because
get_user_tokens_today and the returned usedToday diverge; open a tracked issue
in your project tracker for the planned nightly reconciliation (replace TODO-022
with that issue number/URL) and update the comment to reference that issue ID;
additionally, add a minimal durable fallback by enqueuing the failed payload
(same shape as the insert) to a retry queue or table (e.g.,
llm_usage_events_retry) so failed inserts can be reprocessed by the
reconciliation job — update the code around the insertErr handling and document
the reconciliation job name (e.g., reconcile_llm_usage_events) and
get_user_tokens_today/getUserTokensToday usages so reviewers can locate the
logic.

In `@apps/web/app/api/v1/llm/generate-object/route.ts`:
- Around line 92-110: The current catch block creates a redundant type check
before slicing the error message: replace the conditional assignment for
rawPreview with a direct slice since message is always a string (const
rawPreview = message.slice(0, 500)); update the catch block around rawPreview in
route.ts (the error handler that calls logUsageEvent and wireError in
generate-object) to use this simplified expression and leave the rest
(logUsageEvent call and wireError invocation) unchanged.

In `@apps/web/app/api/v1/llm/generate-text/route.ts`:
- Around line 80-99: Add emitting structured metrics to the metrics pipeline for
latency, promptTokens, completionTokens, total cost, and status immediately
after computing latency and usage (after wireUsageFromSdk and before/after
logUsageEvent) so observability doesn't rely solely on llm_usage_events; update
the generate-text flow (look for functions/vars: wireUsageFromSdk, latency,
usage, logUsageEvent, GenerateTextResponse, quotaSnapshot) to call your metrics
client (e.g., metrics.increment / metrics.histogram) with tags for
route="generate-text", model=body.model, userId and status="ok" and include
latencyMs, promptTokens, completionTokens, and costUsd. Ensure metric emission
is non-blocking and failures are caught/logged but do not change the existing DB
logging or response path.
- Around line 62-78: The gateway-error handler in route.ts currently returns
rawPreview derived from the provider error message which may leak upstream PII
or API keys; update the catch block (the code that calls logUsageEvent and
wireError) to sanitize the error message before slicing and returning by
stripping or redacting common secrets/patterns (e.g., Bearer tokens, "sk_",
"sk-", email addresses, long hex/API-key-like strings) or replace with a generic
user-facing string like "See server logs for details", and ensure the full raw
error is still recorded server-side (e.g., in process logs or the existing
logUsageEvent context) for debugging; reference the existing identifiers
logUsageEvent and wireError to locate and change the returned rawPreview value.

In `@apps/web/lib/waitlist.ts`:
- Around line 14-32: Add a simple rate limit check at the top of joinWaitlist to
prevent automated floods: extract the caller IP (e.g. from X-Forwarded-For or
the platform request context) and consult a rate-limiter store (in-memory LRU or
Redis) to allow only N submissions per IP per time window; if the IP is over the
limit, log the event and redirect to "/waitlist?status=throttled". Optionally
also add a per-email throttle and/or CAPTCHA fallback before calling
getServiceRoleClient().from("waitlist_signups") so the waitlist_signups inserts
are protected from high‑volume or enumeration attacks.

In `@packages/cli/src/commands/doctor.ts`:
- Around line 351-392: In checkUsageEndpoint, change the severity classification
for non-2xx responses: keep severity "ok" when res.ok, make status === 401
return severity "error" (and keep the re-login fix message), and change all
other non-2xx responses from "warn" to "info" so transient 5xxs are less noisy;
update the returned object in the else branch of checkUsageEndpoint (the block
that currently returns label "usage read", category "network", severity "warn" /
fix logic) to implement this new mapping.

In `@packages/cli/src/commands/generate.ts`:
- Around line 676-685: The command builder buildRetryCommand currently uses
JSON.stringify(opts.title) which injects quoted/json-escaped text and can cause
double-escaping when the command is re-used; change the behavior to push the raw
title string (parts.push("--title", opts.title)) so the title is not
JSON-quoted, ensuring the rebuilt command uses the plain user-provided title and
avoiding nested escaping when re-executed or re-serialized.

In `@packages/cli/src/commands/whoami.ts`:
- Around line 102-115: The pct calculation in whoami.ts can produce NaN when
dailyLimit is 0; update the block that reads const { usedToday, dailyLimit,
resetAt } = usage.quota and the pct computation to defensively handle dailyLimit
=== 0 (e.g., set pct = dailyLimit > 0 ? Math.round((usedToday / dailyLimit) *
100) : 0 or display “—”/“0%”) so the console output never prints NaN%; ensure
the human-readable line that uses pct still formats correctly when you choose
the fallback.

In `@packages/cli/src/lib/inline-auth.ts`:
- Around line 78-99: Wrap network/OTP errors from sendLoginCode and
verifyLoginCode so they become user-friendly HelpbaseError instances and add a
single retry attempt for entering the 6-digit code; specifically, catch errors
thrown by sendLoginCode(email) and verifyLoginCode(email, code) inside the
inline-auth flow around the email/code prompts, convert them to HelpbaseError
(including a concise "try again" hint and using
authRequiredError(opts.retryCommand) where appropriate), and implement a loop
that on a verification failure prompts for the code one more time before
propagating the error. Ensure you reference and update the blocks around
sendLoginCode, verifyLoginCode, the code prompt, and the authRequiredError usage
so the user sees a friendly retry message instead of raw exceptions.

In `@packages/create-helpbase/templates/lib/schemas.ts`:
- Around line 252-306: Replace the permissive string validators with Zod 4
format validators: in deployReportSchema change ran_at from z.string() to
z.iso.datetime() and in tenantMcpQuerySchema change tenant_id from
z.string().uuid() to z.uuid(); update the corresponding type usage if any relies
on the old shape and run tests/TS checks to ensure no downstream breakage, and
apply the identical edits to the copy in registry/helpbase/lib/schemas.ts.

In `@packages/shared/src/llm.ts`:
- Around line 132-190: The current truthy check ...(maxOutputTokens ? {
maxOutputTokens } : {}) drops valid zero values; update all such spread checks
in byokGenerateObject, byokGenerateText, hostedGenerateObject, and
hostedGenerateText to only omit the field when maxOutputTokens is null or
undefined (use an explicit != null check) so a passed 0 is preserved when
building the options for generateObject/generateText.
- Around line 110-128: fetchUsageToday currently converts non-OK responses into
a generic GatewayError and skips the structured wire error parsing; update
fetchUsageToday to call the shared throwOnWireError(res) after receiving the
Response (before throwing AuthRequiredError/other errors) so
quota_exceeded/global_cap_hit and other WireErrorBody cases are preserved and
surfaced consistently (ensure throwOnWireError is imported/available in the
module and remove or replace the current generic GatewayError handling in
fetchUsageToday accordingly).
- Around line 340-348: The zodToJsonSchema function currently performs an
unnecessary defensive cast and runtime check for z.toJSONSchema; since the
project pins Zod v4 (where z.toJSONSchema is stable), replace the conditional
wrapper with a direct call to z.toJSONSchema(schema) inside zodToJsonSchema and
remove the type-cast/maybeFn logic, keeping the function as the single
abstraction point (refer to zodToJsonSchema, z.toJSONSchema, and z.ZodType to
locate the code).

In `@registry/helpbase/lib/content.ts`:
- Around line 146-149: The call site in content.ts is using "rehypePlugins as
any" which leaks typing concerns; instead, move the cast into mdx-config.ts by
exporting rehypePlugins already typed as NonNullable<Parameters<typeof
compileMDX>[0]['options']>['mdxOptions']['rehypePlugins'] (or a local type
alias) and update content.ts to import the strongly-typed rehypePlugins without
any "as any" or eslint-disable comments; change the export in ./mdx-config to
perform the single type assertion so all consumers (including content.ts and the
template copy) receive the correctly typed rehypePlugins.

In `@registry/helpbase/lib/mdx-config.ts`:
- Around line 56-65: The getSharedHighlighter() caches a failing Promise in
highlighterPromise so a rejected createHighlighter(...) prevents future retries;
update getSharedHighlighter to reset highlighterPromise to null on error (e.g.,
catch the rejection from createHighlighter or await inside try/catch) and
rethrow the error so transient failures can be retried; reference
highlighterPromise, getSharedHighlighter, and createHighlighter when applying
the change.

In `@scripts/smoke-llm-proxy.sh`:
- Around line 66-72: The current USAGE_JSON assignment uses curl -f which causes
curl to exit non-zero and discard the response body on HTTP errors; change the
call that sets USAGE_JSON so you capture both the response body and the HTTP
status code (e.g., write the body to a temp file and use curl -w "%{http_code}"
to capture the status) rather than relying on -f, then check the status and
include the saved body in the fail "usage endpoint error: $USAGE_JSON" message;
update references to USAGE_JSON, the fail invocation, and the subsequent grep
checks to read from the saved body when the status indicates an error and only
proceed to grep the JSON fields if the HTTP status is 2xx.

In `@scripts/sync-templates.mjs`:
- Around line 64-74: The registry exclusion logic is using includes(`lib/${f}`)
which mismatches the templates loop that uses HOSTED_TIER_EXCLUDES with
startsWith, causing directory-shape paths (e.g., lib/waitlist/index.ts) to slip
through; update the registry walk check (the code that iterates registry libs
around the walk at ~944 and the related checks at 943-945) to use startsWith
against the same HOSTED_TIER_EXCLUDES entries (or normalize both sides to use
startsWith) so exclusions behave consistently for both file and directory paths;
ensure you reference HOSTED_TIER_EXCLUDES when performing the startsWith checks.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

Comment thread apps/web/app/(main)/waitlist/page.tsx Outdated
}) {
const sp = (await searchParamsP) ?? {}
const status = sp.status
const from = sp.from ?? "waitlist"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
fd -t f 'waitlist.ts' apps/web/lib | xargs -I{} sh -c 'echo "=== {} ==="; cat "{}"'

Repository: Codehagen/helpbase

Length of output: 1276


🏁 Script executed:

fd -t f -path '*waitlist/page.tsx' apps/web

Repository: Codehagen/helpbase

Length of output: 230


🏁 Script executed:

find apps/web -name "page.tsx" -path "*waitlist*" | head -20

Repository: Codehagen/helpbase

Length of output: 99


🏁 Script executed:

cat -n apps/web/app/\(main\)/waitlist/page.tsx

Repository: Codehagen/helpbase

Length of output: 3465


Add server-side validation for source parameter in joinWaitlist.

from flows from the URL (line 25) through a hidden form field (line 58) into joinWaitlist, where it's directly inserted into the database without validation. Enforce a max length and restrict characters (allowlist or regex) for source in the server-side joinWaitlist function before inserting into waitlist_signups.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@apps/web/app/`(main)/waitlist/page.tsx at line 25, The hidden form value
"from" (const from = sp.from ?? "waitlist") is taken from the URL and passed
into joinWaitlist where it is inserted into waitlist_signups without checks;
update joinWaitlist to validate and sanitize the incoming source: enforce a max
length (e.g. <= 100), verify allowed characters with a regex or allowlist (e.g.
alphanumerics, hyphen, underscore, and dots), and reject or normalize values
that fail (return an error or set to a safe default like "waitlist"); ensure the
validation runs before any database insert into waitlist_signups and reference
the same parameter name used in the form/handler to locate where to add the
checks.

Comment thread apps/web/app/api/v1/llm/_shared.ts Resolved
Comment thread apps/web/app/api/v1/llm/_shared.ts Resolved
Comment on lines +93 to +99
// Async increment; not awaited.
void supabaseClient
.from("tenants")
.update({ mcp_calls_today: current + 1 })
.eq("id", tenantId)
.then(() => undefined)
return { ok: true }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

find . -type f -name "rate-limit.ts" | head -20

Repository: Codehagen/helpbase

Length of output: 183


🏁 Script executed:

git ls-files | grep -E "(rate-limit|vercel)" | head -30

Repository: Codehagen/helpbase

Length of output: 177


🏁 Script executed:

cat -n packages/create-helpbase/templates/lib/rate-limit.ts 2>/dev/null | head -110

Repository: Codehagen/helpbase

Length of output: 4637


🏁 Script executed:

cat -n registry/helpbase/lib/rate-limit.ts 2>/dev/null | head -110

Repository: Codehagen/helpbase

Length of output: 4637


🏁 Script executed:

cat -n apps/web/lib/rate-limit.ts 2>/dev/null | head -110

Repository: Codehagen/helpbase

Length of output: 4637


🏁 Script executed:

rg "waitUntil" --type ts --type tsx

Repository: Codehagen/helpbase

Length of output: 89


🏁 Script executed:

cat package.json | grep -A 50 '"dependencies"' | head -60

Repository: Codehagen/helpbase

Length of output: 44


🏁 Script executed:

find . -name "package.json" -exec grep -l "@vercel/functions" {} \;

Repository: Codehagen/helpbase

Length of output: 44


🏁 Script executed:

rg "waitUntil" --type ts -g "*.ts" -g "*.js"

Repository: Codehagen/helpbase

Length of output: 44


🏁 Script executed:

find . -name "package.json" | head -5 | xargs cat

Repository: Codehagen/helpbase

Length of output: 6677


🏁 Script executed:

cd apps/web && cat package.json | grep -E '"dependencies"|@vercel' -A 20 | head -30

Repository: Codehagen/helpbase

Length of output: 797


🏁 Script executed:

grep -r "waitUntil" . --include="*.ts" --include="*.js" 2>/dev/null || echo "No waitUntil usage found"

Repository: Codehagen/helpbase

Length of output: 86


🏁 Script executed:

rg "enforceTenantDailyCap" --type ts -g "*.ts" -B 2 -A 2

Repository: Codehagen/helpbase

Length of output: 1832


🏁 Script executed:

cat -n apps/web/app/\(tenant\)/t/\[tenant\]/mcp/route.ts | head -100

Repository: Codehagen/helpbase

Length of output: 3929


🏁 Script executed:

cat -n apps/web/app/\(tenant\)/t/\[tenant\]/mcp/route.ts | sed -n '100,200p'

Repository: Codehagen/helpbase

Length of output: 5010


🏁 Script executed:

cat -n apps/web/app/\(tenant\)/t/\[tenant\]/mcp/route.ts | sed -n '200,280p'

Repository: Codehagen/helpbase

Length of output: 3466


🏁 Script executed:

grep -r "maxDuration\|runtime" apps/web --include="*.ts" -B 2 -A 2 | head -40

Repository: Codehagen/helpbase

Length of output: 2481


🌐 Web query:

Vercel serverless functions unawaited promises dropped after response sent Node.js

💡 Result:

In Vercel serverless functions running Node.js, unawaited promises (fire-and-forget async operations) started after sending the HTTP response are terminated. Vercel halts all processing immediately upon response transmission to free resources, as confirmed in official discussions and documentation. This prevents background tasks like logging, analytics, or cache updates from completing if initiated post-response. To handle non-critical async work after responding: - Use waitUntil(promise) from @vercel/functions (for non-Next.js or older Next.js). This extends the function lifetime until the promise settles, without blocking the response. Example: import { waitUntil } from '@vercel/functions'; export default function handler(req, res) { res.status(200).json({ ok: true }); waitUntil(someAsyncTask); // Continues after response } - For Next.js 15.1+, use after(callback) from next/server, which internally uses waitUntil. - Promises passed to waitUntil share the function's timeout (default 5-10s, configurable up to 800s with Fluid Compute); they cancel if exceeded. Without waitUntil, unawaited promises may trigger unhandled rejection warnings or errors in logs but won't complete execution. Always await critical operations before responding, and use waitUntil for fire-and-forget tasks. Enable Fluid Compute in project settings for longer durations and better background support.

Citations:


Unawaited Supabase write may not flush on Vercel serverless.

void supabaseClient.from("tenants").update(...).then(...) starts the request but does not keep the serverless invocation alive. On Vercel Node.js runtime, the instance terminates as soon as the HTTP response is sent; any unawaited promises are dropped. This silently under-counts mcp_calls_today, the signal the daily cap depends on.

Either await the update (adds one RTT to a path that already did a .maybeSingle() read) or hand it off to waitUntil() from @vercel/functions if latency is critical:

🔒 Minimal fix — await the write
-  // Async increment; not awaited.
-  void supabaseClient
-    .from("tenants")
-    .update({ mcp_calls_today: current + 1 })
-    .eq("id", tenantId)
-    .then(() => undefined)
+  // Must await: unawaited promises are dropped when the serverless
+  // invocation ends, which silently under-counts the daily cap.
+  await supabaseClient
+    .from("tenants")
+    .update({ mcp_calls_today: current + 1 })
+    .eq("id", tenantId)
   return { ok: true }

If the added latency is measurable, use waitUntil instead:

import { waitUntil } from "@vercel/functions"
waitUntil(
  supabaseClient.from("tenants").update({ mcp_calls_today: current + 1 }).eq("id", tenantId),
)

The same fix is needed in registry/helpbase/lib/rate-limit.ts and apps/web/lib/rate-limit.ts.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// Async increment; not awaited.
void supabaseClient
.from("tenants")
.update({ mcp_calls_today: current + 1 })
.eq("id", tenantId)
.then(() => undefined)
return { ok: true }
// Must await: unawaited promises are dropped when the serverless
// invocation ends, which silently under-counts the daily cap.
await supabaseClient
.from("tenants")
.update({ mcp_calls_today: current + 1 })
.eq("id", tenantId)
return { ok: true }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/create-helpbase/templates/lib/rate-limit.ts` around lines 93 - 99,
The unawaited Supabase update using supabaseClient.from("tenants").update(...)
can be dropped on Vercel serverless; change the fire-and-forget call in the
function that updates mcp_calls_today (referencing supabaseClient and tenantId
in rate-limit.ts) to either await the update Promise so the invocation stays
alive, or hand it off to waitUntil(...) from `@vercel/functions` to run in the
background; apply the same change to the equivalent functions in
registry/helpbase/lib/rate-limit.ts and apps/web/lib/rate-limit.ts.

Comment on lines +102 to +110
export function extractClientIp(request: Request): string {
// Vercel sets x-forwarded-for. First entry is the client IP.
const fwd = request.headers.get("x-forwarded-for") ?? ""
const first = fwd.split(",")[0]?.trim()
if (first) return first
const real = request.headers.get("x-real-ip")
if (real) return real
return "unknown"
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

"unknown" fallback buckets all unknowable-IP traffic together.

When neither x-forwarded-for nor x-real-ip is present, every such request shares the single key rl:ip:unknown, so 100 legitimate-but-proxy-obscured clients collectively hit the per-IP cap in 5 minutes and block each other. In a Vercel deployment this should be rare (the platform sets x-forwarded-for), but MCP clients routed through corporate proxies or test harnesses can land here.

Options: bypass the IP check when the source is unresolved (return "unknown" → let the per-tenant cap do the work), or key by a stable request identity you already have (tenant id is already known at the caller, IP is just belt-and-suspenders).

Same applies to registry/helpbase/lib/rate-limit.ts.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/create-helpbase/templates/lib/rate-limit.ts` around lines 102 - 110,
extractClientIp currently returns the string "unknown" which collapses all
unresolved clients into one rl:ip:unknown bucket; change
extractClientIp(Request) to return null (or undefined) when neither header is
present instead of "unknown", and update the rate-limit logic that calls
extractClientIp (the code building per-IP keys in rate limiter) to skip/omit the
per-IP key when the IP is null so the per-tenant bucket is used instead; apply
the same change in the other copy of rate-limit.ts referenced in the review.

Comment thread packages/shared/src/llm-errors.ts Resolved
Comment thread registry/helpbase-workflow/README.md Resolved
- context.test.ts: `missing API key` test now asserts E_AUTH_REQUIRED
  (the new taxonomy) instead of E_CONTEXT_MISSING_KEY. In non-TTY CI,
  running context with no session + no BYOK hits the auth gate before
  the legacy missing-key path.
- ui.ts spinner fallback: drop pc.dim / pc.green / pc.red when we're
  already on the non-TTY fallback path. picocolors is memoized at
  import time so setting CI=1 inside a test doesn't retro-strip ANSI.
  Bare `> msg` / `ok msg` / `x msg` is friendlier to CI log scrapers
  and makes the pre-existing `not.toContain("\x1b[")` assertion pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…iles

Install-path smoke was failing because sync:templates shipped mdx-config.ts
(which imports shiki + rehype-pretty-code) without adding those to the
generated template package.json, so scaffolded projects couldn't build.

Also drop hosted-only files from scaffolder + shadcn registry outputs:
  - lib/rate-limit.ts (per-IP KV, hosted tier only)
  - lib/tenant-search.ts (hosted tenant full-text search)
  - components/tenant-search-trigger.tsx (hosted dashboard widget)

registry.json gets shiki + rehype-pretty-code under the help-center block's
`dependencies` so `shadcn add help-center` pulls them in.

Local smoke: pnpm smoke:install green.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…pdate smoke

shadcn add help-center was failing because registry.json listed lib/content.ts
but not its two local imports (lib/mdx-config.ts, lib/content-dir.ts), so
shadcn didn't copy them into user projects. Build crashed with:
  Module not found: Can't resolve './mdx-config'
  Module not found: Can't resolve './content-dir'

Also update the registry smoke to count id-anchored h2/h3 headings instead
of h1. Since QA ISSUE-001 (commit 5cfe697) removed duplicate `# Title` from
MDX bodies, the h1 now comes from the layout's frontmatter title and no
longer has an id attribute. The new assertion still proves the MDX pipeline
ran (rehype-slug, content rendering) without falsely failing on the layout
change.

Verified: pnpm smoke:registry green (4 h2/h3 headings in rendered article).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Response to CodeRabbit review on PR #2. One release-blocker + eight
correctness/copy fixes.

CRITICAL — global circuit breaker was broken
  _shared.ts logUsageEvent: the client-side `.upsert({day, tokens})` with
  `onConflict: "day"` OVERWROTE the daily counter on every call (PostgREST
  update-on-conflict uses the payload, not an increment). So `tokens`
  was set to whatever the single request's value was, stomping the
  running total and silently disabling the 10M/day global cap.

  Fix: new Postgres function `increment_global_tokens(p_day, p_delta)`
  that does an atomic INSERT ... ON CONFLICT DO UPDATE SET tokens =
  global_daily_tokens.tokens + excluded.tokens. Called from the proxy
  via supabase.rpc. Removes the read-modify-write fallback, the
  "noop to keep client warm" dead call, and the now-orphaned
  currentGlobalTokens helper.

MAJOR
  /api/v1/llm/generate-object/route.ts: was passing z.any() to
  generateObject via `jsonSchemaToZod()`, which throws away the
  model-side schema guidance. Switch to Vercel AI SDK's jsonSchema()
  helper to pass the client's JSON Schema through intact. CLI's
  downstream schema.parse() still provides defense-in-depth.

  apps/web/app/(main)/errors/[code]/page.tsx: add entries for the five
  new error codes (E_AUTH_REQUIRED, E_QUOTA_EXCEEDED, E_GLOBAL_CAP,
  E_LLM_NETWORK, E_LLM_GATEWAY). Without them the CLI's docs: links
  were 404s.

  apps/web/lib/waitlist.ts: stop logging raw user emails on insert
  failure (PII in server logs). source/code/message is enough.

  registry/helpbase-workflow: add HELPBASE_TOKEN to the workflow YAML
  env so the README's documented CI path actually works. Correct the
  README's token-retrieval instructions — `helpbase config get
  session.access_token` doesn't exist; point users at
  `jq -r .access_token ~/.helpbase/auth.json`.

MINOR
  - page.tsx + waitlist.ts: validate `source` URL param against
    ^[a-z0-9_-]{1,40}$ before insert and before echoing back.
  - generate-text + generate-object routes: `messages: []` no longer
    bypasses the prompt-or-messages check; defensive `maxOutputTokens
    <= 0` guard returns 429 quota_exceeded.
  - generate-object route: fix misleading `force-dynamic` body-size
    comment — dynamic only disables static optimization.
  - llm-errors.ts: formatQuotaSuffix guards divide-by-zero when
    dailyLimit is 0; humanTokens/humanUntil guard NaN/invalid ISO.
  - byok.mdx: default and --test both use Gemini Flash Lite in code;
    docs were claiming two different models.
  - inline-auth.ts: HELPBASE_TOKEN set-but-invalid now errors cleanly
    instead of silently falling into an interactive login prompt —
    matches the JSDoc contract and avoids masking broken CI tokens.
  - create-helpbase prompt copy now matches the "login first, BYOK
    optional" messaging used in error hints.
  - errors.ts docblock: fix path to `/app/(main)/errors/...`.

Verified:
  - `supabase` migration `atomic_increment_global_tokens` applied
  - `pnpm --filter @workspace/shared test` → 126 pass
  - `pnpm --filter helpbase test` → 333 pass (with clean HOME)
  - `pnpm --filter web test` → 46 pass
  - `pnpm --filter helpbase typecheck` + `pnpm --filter web typecheck`
    both clean
  - `pnpm sync:templates` + `pnpm registry:build` clean

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (3)
packages/cli/src/lib/ui.ts (1)

75-79: Docstring out of sync with new fallback prefix.

The JSDoc still describes the fallback as "→ message", but the implementation now emits "> message" (and x/ok on stop). Worth updating so the contract matches behavior.

📝 Proposed doc tweak
 /**
  * TTY-aware spinner. When spinners aren't safe (CI, pipes, --json, --quiet),
- * falls back to single-line "→ message" logs on stderr — no ANSI, no motion,
- * still informative.
+ * falls back to single-line "> message" logs on stderr (and "ok"/"x" on stop) —
+ * no ANSI, no motion, still informative.
  */
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/cli/src/lib/ui.ts` around lines 75 - 79, Update the JSDoc for the
TTY-aware spinner in ui.ts to match the implementation: change the described
fallback prefix from "→ message" to "> message" and mention that the stop state
uses "x" for failure and "ok" for success (no ANSI, no motion, still
informative). Locate the spinner implementation/comment block that describes the
fallback behavior and update those strings and wording so the docstring matches
the actual emitted output.
apps/web/app/(main)/errors/[code]/page.tsx (1)

307-316: Nit: prefer the relative /waitlist link in fixes for consistency.

The fixes list hardcodes https://helpbase.dev/waitlist while seeAlso uses the relative /waitlist. For in-app navigation consistency (and to avoid breakage if the apex domain ever changes), consider using the relative path in both places.

Proposed change
-      "Join the paid-tier waitlist: https://helpbase.dev/waitlist",
+      "Join the paid-tier waitlist: /waitlist",
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@apps/web/app/`(main)/errors/[code]/page.tsx around lines 307 - 316, The fixes
array currently contains an absolute URL "https://helpbase.dev/waitlist" while
seeAlso uses the relative "/waitlist"; update the fixes entry to use the
relative path "/waitlist" for consistency and in-app navigation. Locate the
fixes array (the object with keys fixes and seeAlso) and replace the hardcoded
absolute waitlist URL with the relative "/waitlist" string so both fixes and
seeAlso point to the same internal route.
packages/cli/src/lib/inline-auth.ts (1)

83-87: Optional: email validator is quite permissive.

v.includes("@") accepts inputs like @, a@, or @b. sendLoginCode will presumably reject these server-side, but you could tighten client-side with a simple regex (e.g. /^[^\s@]+@[^\s@]+\.[^\s@]+$/) to catch typos before an API round-trip. Non-blocking.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/cli/src/lib/inline-auth.ts` around lines 83 - 87, The current email
prompt uses text({ message: "Enter your email:", placeholder: "you@company.com",
validate: (v) => (v.includes("@") ? undefined : "Please enter a valid email") })
which is too permissive; update the validate handler in the email assignment to
check against a simple stricter regex (for example /^[^\s@]+@[^\s@]+\.[^\s@]+$/)
and return the same error string when it fails, so the client-side validation in
the email variable (and the text(...) call) blocks obvious malformed addresses
before calling sendLoginCode.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@apps/web/app/`(main)/waitlist/page.tsx:
- Around line 44-49: The anchor for the BYOK docs currently points to
"/docs/byok" but should point to "/guides/byok"; update the href on the anchor
element that renders the "Bring your own Vercel AI Gateway key" link in page.tsx
(the <a> element in the waitlist page) to use "/guides/byok" so it matches the
route group and the article at apps/web/content/guides/byok.mdx.

In `@apps/web/app/api/v1/llm/generate-text/route.ts`:
- Around line 40-55: Validate that body.prompt, if present, is a string and
reject non-string prompt values with wireError(400,...); similarly validate
body.maxOutputTokens is a finite non-NaN number (and optionally a non-negative
integer) before computing requested/maxOutputTokens — return wireError(400,
"bad_request", ...) for invalid types or negative values instead of letting NaN
or negatives flow into the Math.min/quota logic; update the logic around
requested, maxOutputTokens, PER_CALL_CEILING, remainingBudget and the quota
guard so requested is derived only from a validated numeric body.maxOutputTokens
(or default to PER_CALL_CEILING) and then clamp via Math.min to remainingBudget.

In `@scripts/smoke-registry.sh`:
- Line 141: The HEADING_COUNT assignment uses grep -cE which counts matching
lines, not occurrences, so minified SSR HTML can undercount; change the command
that sets HEADING_COUNT (the line referencing ARTICLE_HTML and pattern
'<h[2-3][^>]*id=') to count actual matches instead of lines — e.g., emit each
match with grep -o (or use perl/awk) and pipe to wc -l so HEADING_COUNT reflects
the number of heading occurrences rather than the number of lines containing
them.

---

Nitpick comments:
In `@apps/web/app/`(main)/errors/[code]/page.tsx:
- Around line 307-316: The fixes array currently contains an absolute URL
"https://helpbase.dev/waitlist" while seeAlso uses the relative "/waitlist";
update the fixes entry to use the relative path "/waitlist" for consistency and
in-app navigation. Locate the fixes array (the object with keys fixes and
seeAlso) and replace the hardcoded absolute waitlist URL with the relative
"/waitlist" string so both fixes and seeAlso point to the same internal route.

In `@packages/cli/src/lib/inline-auth.ts`:
- Around line 83-87: The current email prompt uses text({ message: "Enter your
email:", placeholder: "you@company.com", validate: (v) => (v.includes("@") ?
undefined : "Please enter a valid email") }) which is too permissive; update the
validate handler in the email assignment to check against a simple stricter
regex (for example /^[^\s@]+@[^\s@]+\.[^\s@]+$/) and return the same error
string when it fails, so the client-side validation in the email variable (and
the text(...) call) blocks obvious malformed addresses before calling
sendLoginCode.

In `@packages/cli/src/lib/ui.ts`:
- Around line 75-79: Update the JSDoc for the TTY-aware spinner in ui.ts to
match the implementation: change the described fallback prefix from "→ message"
to "> message" and mention that the stop state uses "x" for failure and "ok" for
success (no ANSI, no motion, still informative). Locate the spinner
implementation/comment block that describes the fallback behavior and update
those strings and wording so the docstring matches the actual emitted output.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 33d68f59-732e-421b-996d-c7484312beac

📥 Commits

Reviewing files that changed from the base of the PR and between 719a34f and 61739b3.

📒 Files selected for processing (25)
  • apps/web/app/(main)/errors/[code]/page.tsx
  • apps/web/app/(main)/waitlist/page.tsx
  • apps/web/app/api/v1/llm/_shared.ts
  • apps/web/app/api/v1/llm/generate-object/route.ts
  • apps/web/app/api/v1/llm/generate-text/route.ts
  • apps/web/content/guides/byok.mdx
  • apps/web/lib/waitlist.ts
  • apps/web/public/r/help-center.json
  • apps/web/public/r/helpbase-workflow.json
  • apps/web/public/r/registry.json
  • apps/web/types/supabase.ts
  • packages/cli/src/lib/errors.ts
  • packages/cli/src/lib/inline-auth.ts
  • packages/cli/src/lib/ui.ts
  • packages/cli/test/context.test.ts
  • packages/create-helpbase/src/index.ts
  • packages/create-helpbase/templates/content/guides/byok.mdx
  • packages/create-helpbase/templates/package.json
  • packages/shared/src/llm-errors.ts
  • registry.json
  • registry/helpbase-workflow/README.md
  • registry/helpbase-workflow/helpbase-sync.yml
  • registry/helpbase/content/guides/byok.mdx
  • scripts/smoke-registry.sh
  • scripts/sync-templates.mjs
✅ Files skipped from review due to trivial changes (7)
  • registry/helpbase-workflow/helpbase-sync.yml
  • packages/create-helpbase/templates/package.json
  • registry/helpbase-workflow/README.md
  • apps/web/content/guides/byok.mdx
  • packages/cli/src/lib/errors.ts
  • packages/create-helpbase/templates/content/guides/byok.mdx
  • registry/helpbase/content/guides/byok.mdx
🚧 Files skipped from review as they are similar to previous changes (6)
  • packages/create-helpbase/src/index.ts
  • scripts/sync-templates.mjs
  • apps/web/lib/waitlist.ts
  • apps/web/public/r/helpbase-workflow.json
  • apps/web/types/supabase.ts
  • apps/web/app/api/v1/llm/generate-object/route.ts

Comment on lines +44 to +49
Need more tokens right now?{" "}
<a href="/docs/byok" className="underline hover:text-foreground">
Bring your own Vercel AI Gateway key
</a>{" "}
— unlimited, your own cost.
</p>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
fd -t f 'page.(tsx|mdx|md)' apps/web/app | rg -i 'byok'
fd -t d 'byok' apps/web
rg -n --type=tsx --type=ts -C2 "'/docs/byok'|\"/docs/byok\""

Repository: Codehagen/helpbase

Length of output: 89


🏁 Script executed:

#!/bin/bash
# Search for files containing 'byok' reference
rg -i 'byok' apps/web --type ts --type tsx

# Look for docs directory structure
find apps/web -type d -name 'docs' | head -20

# Search for any page files with 'byok' in path or content
find apps/web -type f \( -name '*.tsx' -o -name '*.mdx' -o -name '*.md' \) | xargs grep -l -i 'byok' 2>/dev/null | head -20

Repository: Codehagen/helpbase

Length of output: 266


🏁 Script executed:

#!/bin/bash
# Look for docs routing configuration
find apps/web/app -type d -name 'docs' -o -name '[...slug]' | head -20

# Search for app directory structure related to docs
ls -la apps/web/app/

# Check if there's a docs folder or similar
find apps/web/app -maxdepth 3 -type f -name 'page.*' | grep -i doc

# Look at content directory structure
ls -la apps/web/content/
</script>

Repository: Codehagen/helpbase

Length of output: 1004


🏁 Script executed:

#!/bin/bash
# Look for routes in (main) directory
find apps/web/app/\(main\) -type d | sort

# Check if there's a docs folder in (main)
ls -la apps/web/app/\(main\)/ 2>/dev/null

# Search for any dynamic route segments
find apps/web/app -type d -name '\[*' -o -name '\.\.\.*'

# Look for files that handle /docs routing
rg 'docs.*byok|/docs/' apps/web/app --type ts --type tsx -A 2

Repository: Codehagen/helpbase

Length of output: 919


🏁 Script executed:

#!/bin/bash
# Check the page.tsx that handles [category]/[slug]
cat -n apps/web/app/\(main\)/\(docs\)/\[category\]/\[slug\]/page.tsx

# Check if there's middleware or redirects
cat -n apps/web/app/middleware.ts 2>/dev/null || echo "No middleware.ts"

# Check the layout in (docs)
cat -n apps/web/app/\(main\)/\(docs\)/layout.tsx 2>/dev/null || echo "No layout found"

# Verify exact path to content files
ls -la apps/web/content/guides/ | grep byok

Repository: Codehagen/helpbase

Length of output: 9157


🏁 Script executed:

#!/bin/bash
# Confirm the article metadata
cat -n apps/web/content/guides/byok.mdx | head -20

Repository: Codehagen/helpbase

Length of output: 1016


Fix broken BYOK documentation link.

The link should be /guides/byok, not /docs/byok. The docs are organized under route groups, so /docs doesn't appear in the URL. The route is /guides/byok based on the article location at apps/web/content/guides/byok.mdx.

Update line 46:

- <a href="/docs/byok" className="underline hover:text-foreground">
+ <a href="/guides/byok" className="underline hover:text-foreground">
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
Need more tokens right now?{" "}
<a href="/docs/byok" className="underline hover:text-foreground">
Bring your own Vercel AI Gateway key
</a>{" "}
— unlimited, your own cost.
</p>
Need more tokens right now?{" "}
<a href="/guides/byok" className="underline hover:text-foreground">
Bring your own Vercel AI Gateway key
</a>{" "}
— unlimited, your own cost.
</p>
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@apps/web/app/`(main)/waitlist/page.tsx around lines 44 - 49, The anchor for
the BYOK docs currently points to "/docs/byok" but should point to
"/guides/byok"; update the href on the anchor element that renders the "Bring
your own Vercel AI Gateway key" link in page.tsx (the <a> element in the
waitlist page) to use "/guides/byok" so it matches the route group and the
article at apps/web/content/guides/byok.mdx.

Comment on lines +40 to +55
if (!body.model || typeof body.model !== "string") {
return wireError(400, "bad_request", "Missing required field: `model` (string).")
}
const hasMessages = Array.isArray(body.messages) && body.messages.length > 0
if (!body.prompt && !hasMessages) {
return wireError(400, "bad_request", "Provide either `prompt` or a non-empty `messages` array.")
}

const requested = body.maxOutputTokens ?? PER_CALL_CEILING
const maxOutputTokens = Math.min(requested, remainingBudget)
// Defense in depth: `withAuthAndQuota` blocks at usedToday >= DAILY_USER_LIMIT,
// so remainingBudget should always be > 0 here. This guard protects against
// future refactors or edge-case drift in the quota math.
if (maxOutputTokens <= 0) {
return wireError(429, "quota_exceeded", "Daily free-tier token quota reached.")
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Tighten body validation for prompt and maxOutputTokens.

Two small gaps that survive the current checks:

  • Line 44: !body.prompt accepts only falsy, so a non-string prompt (number/object) slips through to generateText with a confusing SDK-level failure.
  • Line 48–54: body.maxOutputTokens isn't type-validated. NaN → Math.min(NaN, remainingBudget) === NaN → NaN <= 0 is false, so NaN is forwarded to the SDK. A negative number, conversely, trips the <= 0 branch and is surfaced as 429 quota_exceeded even though it's really a client bug (should be 400).
Proposed fix
-  const hasMessages = Array.isArray(body.messages) && body.messages.length > 0
-  if (!body.prompt && !hasMessages) {
-    return wireError(400, "bad_request", "Provide either `prompt` or a non-empty `messages` array.")
-  }
-
-  const requested = body.maxOutputTokens ?? PER_CALL_CEILING
-  const maxOutputTokens = Math.min(requested, remainingBudget)
-  // Defense in depth: `withAuthAndQuota` blocks at usedToday >= DAILY_USER_LIMIT,
-  // so remainingBudget should always be > 0 here. This guard protects against
-  // future refactors or edge-case drift in the quota math.
-  if (maxOutputTokens <= 0) {
+  const hasPrompt = typeof body.prompt === "string" && body.prompt.length > 0
+  const hasMessages = Array.isArray(body.messages) && body.messages.length > 0
+  if (!hasPrompt && !hasMessages) {
+    return wireError(400, "bad_request", "Provide either `prompt` (non-empty string) or a non-empty `messages` array.")
+  }
+
+  if (
+    body.maxOutputTokens !== undefined &&
+    (typeof body.maxOutputTokens !== "number" ||
+      !Number.isFinite(body.maxOutputTokens) ||
+      body.maxOutputTokens <= 0)
+  ) {
+    return wireError(400, "bad_request", "`maxOutputTokens` must be a positive finite number.")
+  }
+  const requested = body.maxOutputTokens ?? PER_CALL_CEILING
+  const maxOutputTokens = Math.min(requested, remainingBudget)
+  // Defense in depth: `withAuthAndQuota` blocks at usedToday >= DAILY_USER_LIMIT,
+  // so remainingBudget should always be > 0 here.
+  if (maxOutputTokens <= 0) {
     return wireError(429, "quota_exceeded", "Daily free-tier token quota reached.")
   }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@apps/web/app/api/v1/llm/generate-text/route.ts` around lines 40 - 55,
Validate that body.prompt, if present, is a string and reject non-string prompt
values with wireError(400,...); similarly validate body.maxOutputTokens is a
finite non-NaN number (and optionally a non-negative integer) before computing
requested/maxOutputTokens — return wireError(400, "bad_request", ...) for
invalid types or negative values instead of letting NaN or negatives flow into
the Math.min/quota logic; update the logic around requested, maxOutputTokens,
PER_CALL_CEILING, remainingBudget and the quota guard so requested is derived
only from a validated numeric body.maxOutputTokens (or default to
PER_CALL_CEILING) and then clamp via Math.min to remainingBudget.

Comment thread scripts/smoke-registry.sh
# frontmatter, so MDX bodies no longer include a leading `# Title`. A
# properly-rendered article still has 3+ h2/h3 headings with id= anchors
# from rehype-slug. Count those instead of demanding an h1 from MDX.
HEADING_COUNT=$(grep -cE '<h[2-3][^>]*id=' "$ARTICLE_HTML" || true)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

grep -c counts matching lines, not heading occurrences — can undercount on minified HTML.

Next.js SSR output is often emitted as a single (or few) long lines. grep -cE returns the number of lines that contain at least one match, so if all h2/h3 tags land on the same line the count will be 1 regardless of how many headings were rendered, causing a false failure (or masking a regression). Prefer counting actual matches.

🔧 Proposed fix
-HEADING_COUNT=$(grep -cE '<h[2-3][^>]*id=' "$ARTICLE_HTML" || true)
+HEADING_COUNT=$(grep -oE '<h[2-3][^>]*id=' "$ARTICLE_HTML" | wc -l | tr -d ' ')
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@scripts/smoke-registry.sh` at line 141, The HEADING_COUNT assignment uses
grep -cE which counts matching lines, not occurrences, so minified SSR HTML can
undercount; change the command that sets HEADING_COUNT (the line referencing
ARTICLE_HTML and pattern '<h[2-3][^>]*id=') to count actual matches instead of
lines — e.g., emit each match with grep -o (or use perl/awk) and pipe to wc -l
so HEADING_COUNT reflects the number of heading occurrences rather than the
number of lines containing them.

This branch was successfully deployed

1 active deployment
Preview — 61739b36 Deployed Apr 17, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant