Skip to content

feat(cli): agent ok reports whether Taskless is out of date - #606

Merged
theCodeDrift merged 9 commits into
mainfrom
agent-ok-environment-report
Nov 3, 2026
Merged

theCodeDrift merged 9 commits into
mainfrom
agent-ok-environment-report

Conversation

@theCodeDrift

@theCodeDrift theCodeDrift commented Nov 3, 2026 •

Copy link
Copy Markdown
Member

taskless agent ok is the first Taskless command of every session, so it is where an agent learns whether the environment needs updating. It is now a recipe, and a local check reports the same thing.

What changes

  • agent ok is a recipe (packages/cli/src/agent/ok.md). It renders the standard header and the one-command-per-call paragraph, but not the fetch-time directive, because it runs once per session. Its body is one of two passages, filled in from the state measured when it's fetched:
    • "Your Taskless environment is up to date and you can proceed."
    • "Your Taskless environment is out of date, the following steps are needed in order for commands to work reliably:", followed by numbered steps, each with its exact command. The recipe tells the agent to hand the steps to the user (with ! in Claude Code, or in another terminal), not to run them, and to run agent ok again once the user has.
  • Three things are measured, in src/install/freshness.ts, shared by agent ok and check:
    • cli: a newer published version of this build's own package (@taskless/cli or @taskless/cli-nightly).
    • skill: the installed skill doesn't match this CLI. This is the same comparison info reports.
    • rules: the rules owe a reconcile. This is the same status info and init report, with the same wording.
  • Latest-version lookup (src/install/latest-version.ts): it asks only the npm registry's dist-tags endpoint, with a 1s timeout. The answer is accepted only if it's a strict semantic version, because it ends up in a command the user runs. If the lookup fails it says nothing. It's skipped in CI, when NO_UPDATE_NOTIFIER is set, and on self builds. A jsDelivr race was measured and rejected, so no other host is involved.
  • check, outside CI, measures alongside the scan and adds one single-line notice, in text and in --json notices. It never changes the results or the exit code, and under NO_UPDATE_NOTIFIER it adds nothing.
  • Skill: the preflight section describes both outcomes and the hand-off to the user.
  • Tests: the test environment sets NO_UPDATE_NOTIFIER=1, so no test reaches the registry. The lookup's own tests clear it and stub fetch.

agent ok reads .taskless/ but never writes to it, and it always exits 0. The command line is unchanged, so the permission rules init writes still match.

Size

About 1600 lines added, roughly a third of them OpenSpec artifacts: the change itself plus the specs it merges on archive. The hand-written code and tests come in under the ~1200-line guideline.

Our own rules, reconciled and guarded

Running this change against our own repo showed the dogfood tree was behind. It recorded our rules against ast-grep 0.45.2 and Vale 3.18.0, and hadn't changed since 2026-08-29 (869dd49a), while the CLI bundles 0.51.0 and 3.24.0. Nothing refused, because pnpm cli sets TASKLESS_ALLOW_VERSION_MISMATCHES=1, which also lifts the engine gate. So pnpm lint had been running the house-style rules on engines they were never reconciled against.

  • Caught up: I walked the update recipe from 0.11.0 to 0.13.4. None of our 14 rules has a shape the ledger flags. validate and verify pass on the bundled engines, and update --rules stamped 0.13.4 with no rewrites needed. Only .taskless/taskless.json changed.
  • Guarded: dogfood-scaffold-current.test.ts now fails when our rules are recorded against engines the CLI doesn't bundle, or when an automatic rule rewrite is pending. The failure message names the fix. So the PR that bumps an engine has to reconcile our rules, the same way the existing check makes the PR that adds a migration apply it. I confirmed it fails on the old taskless.json with the actionable message.

This deliberately doesn't assert reconcile.required, because that also turns true on every plain version bump, which would make the automatic Version Packages PR red on every release.

Checks

  • pnpm typecheck and pnpm lint pass. The dogfood check that lint runs printed the new notice for this repo, which really is behind (it records CLI 0.11.2). That matches pnpm cli info.
  • Full CLI suite: 2895 passed, 10 skipped.
  • pnpm cli agent ok, run here and in an empty directory, renders the out-of-date and up-to-date passages respectively. Both exit 0, and the empty directory gets no .taskless/.

OpenSpec change agent-ok-update-notice is archived on this branch.

@theCodeDrift

Copy link
Copy Markdown
Member Author

@claude /review

@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @theCodeDrift's task in 4m 3s —— View job


Review complete

Review mode: incremental — read 0 prior review thread(s) before reviewing. .prior-review.json contained only the trigger comment itself (no prior reviews, review threads, or top-level comments), so this is the first substantive review of this PR — everything below is new.

  • Read .prior-review.json
  • Fetch PR diff and file list
  • Read changed files in full where needed
  • Review correctness, security, performance, tests, clarity
  • Post inline comments for concrete issues
  • Post final summary

Findings

[New] Real network call in a unit test (packages/cli/test/agent-ok.test.ts:78, inline comment posted) — beforeEach overrides the suite-wide NO_UPDATE_NOTIFIER=1 (set in vite.config.ts precisely to keep every test off the registry) back to "", so individual tests can drive stubRegistry(). The "captures cli_agent with the ok topic" test runs agent ok without stubbing fetch or the registry, so it makes a real fetch to https://registry.npmjs.org through the real measureEnvironment path. This directly contradicts the PR description's claim that no test reaches the registry, and is a flakiness/external-dependency risk in CI.

[New] Leftover stale doc comment (packages/cli/src/prompts/recipes.ts:835, inline comment posted) — a one-line JSDoc comment for resolveInvocation was left in place directly above the new multi-paragraph JSDoc block this PR adds for the same function. Cosmetic only; should be deleted.

What looked solid

  • freshness.ts's measureCli/measureSkill/measureRules/environmentSteps/environmentNotice are small, read-only, and well covered by freshness.test.ts, including CI, NO_UPDATE_NOTIFIER, no-.taskless/, and ordering cases.
  • latest-version.ts's strict-semver gate is correctly anchored (verified 01.0.0, a trailing newline, and a shell-operator string are all rejected) and the lookup never throws.
  • check.ts's outOfDateNotice is started after all early returns and awaited just before notices are assembled, so a project with no rules never opens a request, and a rejected measurement is swallowed without affecting the exit code or results — matches check-environment-notice.test.ts's "completes normally when the measurement fails" case.
  • The ok recipe correctly gets no fetch-time directive (directive: key !== PREFLIGHT_TOPIC[0]) and no survey invite ("ok" is absent from SURVEYED_TOPICS), and measureEnvironment never creates .taskless/ (confirmed against install/freshness.ts and the "writes nothing under an existing .taskless/" test).
  • SKILL.md's preflight section and openspec/specs/* updates read consistently with the implementation (hand steps to the user, re-run agent ok to confirm, no guess at an unknown launcher in the agent-facing recipe).

Comment thread packages/cli/test/agent-ok.test.ts Resolved
Comment thread packages/cli/src/prompts/recipes.ts Resolved
@theCodeDrift

Copy link
Copy Markdown
Member Author

@claude /review

@github-actions

github-actions Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @theCodeDrift's task in 2m 23s —— View job


Review complete

Review mode: incremental — read 2 prior review thread(s) before reviewing. Both prior threads are unresolved and carry no *— AI Coding Agent*/*- AI Coding Agent* marker, so both are treated as STILL OPEN and re-raised below. No other reviews or top-level comments contained additional findings to track.

  • Read .prior-review.json
  • Fetch PR diff and file list
  • Read changed files in full where needed
  • Review correctness, security, performance, tests, clarity
  • Post inline comments for concrete issues
  • Post final summary

Findings

[Unchanged since last review] Real network call in a unit test (packages/cli/test/agent-ok.test.ts:182 and :187, inline comment posted) — beforeEach (line 78) still clears the suite-wide NO_UPDATE_NOTIFIER=1 (set in vite.config.ts to keep every test off the registry) back to "". "captures cli_agent with the ok topic" and "is listed in the agent index" still run agent ok / agent without stubRegistry() or a stubbed fetch, so they still reach the real measureEnvironment path with the real, unmocked global fetch and make a live call to https://registry.npmjs.org. Nothing in this PR's current state has added a stub to either test; this still contradicts the PR description's claim that no test reaches the registry, and remains a flakiness/external-dependency risk.

[Unchanged since last review] Leftover stale doc comment (packages/cli/src/prompts/recipes.ts:835, inline comment posted) — the one-line JSDoc comment for resolveInvocation is still sitting directly above the new multi-paragraph JSDoc block this PR adds for the same function. Cosmetic only; should be deleted.

Nothing new was found beyond these two still-open items. The rest of the diff — freshness.ts's measureCli/measureSkill/measureRules/environmentSteps/environmentNotice, latest-version.ts's strict-semver gate, check.ts's outOfDateNotice wiring (verified it reuses the existing warnNotice/runNotices dual-path pattern already used elsewhere in the file, not a new duplication bug), the ok recipe's lack of a fetch-time directive and survey invite, recipes.ts's environmentStatusBlock, the new dogfood-scaffold-current.test.ts guard, and the freshness.test.ts / latest-version.test.ts / check-environment-notice.test.ts coverage — all read correctly and consistently with the PR description on this pass.
(branch)

Comment thread packages/cli/test/agent-ok.test.ts Resolved
Comment thread packages/cli/src/prompts/recipes.ts Resolved
@theCodeDrift

Copy link
Copy Markdown
Member Author

Re: @github-actions — "Claude finished @theCodeDrift's task in 4m 3s / 2m 23s" (both reviews)
#606 (comment)

Both findings are fixed in 4ec21c2. Every agent ok test now stubs fetch from beforeEach, so no test reaches the registry. The stale doc comment above resolveInvocation is removed. All four threads are answered and resolved.

— AI Coding Agent

@theCodeDrift
theCodeDrift merged commit 06c2e6b into main Nov 3, 2026
8 checks passed
@theCodeDrift
theCodeDrift deleted the agent-ok-environment-report branch November 3, 2026 19:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant