diff --git a/.changeset/agent-ok-environment-report.md b/.changeset/agent-ok-environment-report.md new file mode 100644 index 00000000..8d0e0375 --- /dev/null +++ b/.changeset/agent-ok-environment-report.md @@ -0,0 +1,7 @@ +--- +"@taskless/cli": patch +--- + +`taskless agent ok` is now a recipe that says whether your Taskless environment is up to date. It still confirms the CLI can run. It checks three things: whether a newer CLI is published, whether the installed skill matches this CLI, and whether the rules owe a reconcile. When all three are current it says "Your Taskless environment is up to date and you can proceed." Otherwise it lists numbered steps with the exact commands that fix each one. The Taskless skill tells the agent to hand those steps to you to run yourself (with `!` in Claude Code, or in another terminal) and to run `agent ok` again once you have. A local `taskless check` adds the same report as a single notice, without changing its results or exit code. + +The newer-CLI check asks only the npm registry, once, with a one-second timeout. It accepts the answer only if it is a strict semantic version, and says nothing if the lookup fails. It is skipped in CI and when `NO_UPDATE_NOTIFIER` is set, and under `NO_UPDATE_NOTIFIER` a local `check` shows no notice at all. `agent ok` reads `.taskless/` but never writes to it. diff --git a/.taskless/taskless.json b/.taskless/taskless.json index a1235c41..d81bc43a 100644 --- a/.taskless/taskless.json +++ b/.taskless/taskless.json @@ -25,10 +25,10 @@ "onboarded": true }, "rules": { - "reconciledTo": "0.11.0-self", + "reconciledTo": "0.13.4", "engines": { - "sg": "0.45.2", - "vale": "3.18.0" + "sg": "0.51.0", + "vale": "3.24.0" } } } diff --git a/openspec/changes/archive/2026-11-03-agent-ok-update-notice/.openspec.yaml b/openspec/changes/archive/2026-11-03-agent-ok-update-notice/.openspec.yaml new file mode 100644 index 00000000..a7587c5f --- /dev/null +++ b/openspec/changes/archive/2026-11-03-agent-ok-update-notice/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-11-02 diff --git a/openspec/changes/archive/2026-11-03-agent-ok-update-notice/design.md b/openspec/changes/archive/2026-11-03-agent-ok-update-notice/design.md new file mode 100644 index 00000000..bd8dbf2a --- /dev/null +++ b/openspec/changes/archive/2026-11-03-agent-ok-update-notice/design.md @@ -0,0 +1,61 @@ +## Context + +`agent ok` is a built-in branch in `packages/cli/src/commands/agent.ts` that prints one sentence. Everything needed to judge staleness already exists, except a registry lookup: + +- `checkStaleness(cwd)` (`install/install.ts`) compares each installed skill's version with the embedded one. `info` reports it. +- `reconcileStatus(cwd, manifest.rules)` and `getReconcileNotice(status, invocation)` (`rules/reconcile-status.ts`) decide whether `update` is due, and word it. +- `buildLauncherPackage()` (`install/launcher.ts`) names the package this build is (`@taskless/cli`, `@taskless/cli-nightly`), or returns `undefined` on a `self` build. +- `compareSemver` (`util/version-compare.ts`) orders nightlies correctly. +- Nightlies publish to `@taskless/cli-nightly` under `latest`, so both channels ask the same question of their own package. +- `isCiEnvironment(process.env.CI)` (`util/interactive.ts`) is the CI test `check` already uses. + +Recipes already have a pattern for state measured at fetch time: `HOST_TOOLS` and `LEGACY_LINTERS` are whole passages chosen from options the CLI fills in (`hostTools`, `linters`). `agent.ts` asks the template's `variables` whether a measurement is needed, so no topic name is hard-coded. The prompts module must not read `process` or do I/O, because Workers import it. + +## Goals / Non-Goals + +**Goals:** + +- `ok` is an ordinary recipe whose body is one caller-measured passage. +- One freshness module answers "what is out of date" for both `agent ok` and `check`, so the two cannot disagree. +- Neither command gets slower than the timeout, fails because of the lookup, or writes anything for it. + +**Non-Goals:** + +- Caching the registry answer across runs. +- Running any fix from the CLI. +- A freshness notice on commands other than `ok` and `check`. + +## Decisions + +**A new `ENVIRONMENT_STATUS` variable, following `HOST_TOOLS`.** `RecipeOptions` gains `environment?: EnvironmentReport`, and an exported `ENVIRONMENT_VARIABLES` set contains the variable's name. The renderer picks one of three passages: up to date, out of date with numbered steps, or (with `environment` undefined) not checked. The prompts module only formats the report, and `agent.ts` does the measuring, exactly as with `detectHostTools`. Alternative considered: keeping `ok` built in and printing the passages from `agent.ts`. That loses the header, the one-command paragraph, and the prompts export for free, which is what "ok is a recipe" asks for. + +**`ok` skips the fetch-time directive.** That directive tells the agent to refetch the topic for every task, which contradicts a once-per-session preflight. The `agent` command passes `directive: false` for this one topic. The skip is keyed on the topic name, as the survey gate's topic set is. + +**The latest version comes from the npm registry alone, and is validated as strict semver.** `GET https://registry.npmjs.org/-/package//dist-tags` returns a few dozen bytes and measured ~0.15–0.2s from a developer machine, uncached at the edge and therefore always fresh. The request uses the global `fetch` with `AbortSignal.timeout(1000)`. The answer is accepted only when `latest` matches the strict semver grammar (`MAJOR.MINOR.PATCH`, optional `-prerelease` and `+build`, nothing else), because the version is spliced into a command the user is told to run. Any throw, non-2xx response, missing `latest`, or string that fails validation means "unknown". The host is fixed. A mirror that blocks the public registry times out, silently. + +Alternatives considered: racing jsDelivr's `@latest/package.json` against the registry. It is ~35ms when warm, but 0.8–1.0s cold and up to 12 hours stale, and it adds a second party, outside npm, to the path of a string the user executes. npm is already the party we trust for the install itself, so it stays the only one. `npm view` was also rejected, because it is slower, needs npm on `PATH`, and is a subprocess that sandboxes block. + +**The freshness module is pure apart from its injected reads.** `measureEnvironment({ cwd, fetch, ci })` returns `{ parts: Array<"cli" | "skill" | "rules">, cli?: { running, latest, package }, skill?: {...}, rules?: ReconcileStatus }`, and runs the three checks in parallel. Step wording lives in the prompts module (for the recipe) and in one short notice formatter (for `check`). Both take the same report and the same invocation, and the rules sentence reuses `getReconcileNotice`. + +**The fix for the cli part pins the exact version.** It is the detected launcher plus `@` with no arguments. When the launcher is unknown, the command is rendered with the same agent-fill marker every recipe uses, rather than a guessed launcher. + +**`check` starts the measurement in parallel with the scan, and awaits it at the end.** The scan usually takes longer than the timeout, so a local `check` (and a pre-commit hook) gains no latency in the common case, and at most 1s in the worst. The notice joins the existing notice list, so text and `--json` handling come free. When `isCiEnvironment` is true, the measurement is never started. + +**The user runs the steps, and the agent re-checks with `agent ok`.** A harness whose permissions block Taskless commands usually blocks a newer CLI or `update` too, so an agent that offers to run the steps runs straight into the block. The skill tells the agent to hand the steps to the user to run themselves: in Claude Code with the `!` prefix, which runs the line in the session's shell, or in another terminal. When the user says they are done, the agent runs `agent ok` again to confirm. That rerun is the one exception to once per session. It is distinct from the blocked-preflight guidance, which does not offer `!`, because there `!` would clear only the one preflight call and leave every later Taskless command blocked. Here the steps are one-off commands, and running them once is the whole job. + +**`NO_UPDATE_NOTIFIER` turns the lookup off.** It is the convention update-notifier and many CLIs honor, so a user who has opted out elsewhere is opted out here too. It is also how the test suite stays off the network: the suite runs as a release build with `CI` empty (so it can test local behavior), which would otherwise send every spawned `check` and `agent ok` to the registry. `vite.config.ts` sets it for every test, and the lookup's own tests clear it and stub `fetch`. This was found during implementation. + +**In `check`, `NO_UPDATE_NOTIFIER` silences the whole notice, and the notice is one line.** Found when the full suite ran. Fixture projects carry stale local state, so the skill and rules parts fired in tests that assert an exact notice list. More to the point, a user who opted out of update notices does not want a local one on every commit either. `agent ok` still reports local staleness under the opt-out, because the user ran the preflight on purpose. The notice also numbers its steps inline on one line, because `vale-orchestration.test.ts` holds `check`'s `--json` notices to one line each, and a consumer should not have to split an element. + +**`--anonymous` does not gate the lookup.** It means "skip the Taskless API"; npm is not it. Gating on it would also contradict `check`'s "--anonymous is a no-op" requirement. + +## Risks / Trade-offs + +- [A notice on every local `check` until the user upgrades, including from git hooks] → That is the point. It is one marked line and never fails the run. +- [`ok` and local `check` now touch the network] → Bounded at 1s, silent, and skipped in CI. +- [The `ok` output changes from one line to a recipe] → The skill that reads it ships in the same package and is updated in this change. An older installed skill still sees the run succeed, which is all it checks for. +- [Tests reaching the registry] → `fetch` is injected into the freshness module, and the command tests stub the global `fetch` or set `CI`. + +## Migration Plan + +None. The skill and the commands ship together. Rolling back is a revert. diff --git a/openspec/changes/archive/2026-11-03-agent-ok-update-notice/proposal.md b/openspec/changes/archive/2026-11-03-agent-ok-update-notice/proposal.md new file mode 100644 index 00000000..d6d3fb1a --- /dev/null +++ b/openspec/changes/archive/2026-11-03-agent-ok-update-notice/proposal.md @@ -0,0 +1,43 @@ +## Why + +`taskless agent ok` is the first Taskless command of every session, so it is the one place an agent is guaranteed to look. Today it only proves the CLI can run. A project can still be running a CLI several releases behind, carry skill files from another version, or owe a rules reconcile, and nothing tells the agent until a later command refuses or someone runs `info` by hand. Developers who run `check` locally, by hand or from a git hook, get no hint either. + +## What Changes + +- `ok` becomes a recipe, `packages/cli/src/agent/ok.md`, served like any other topic: it has the `# Topic:` header and the one-command-per-call paragraph. Its body is a passage the CLI fills from state it measures at fetch time: + - When nothing is out of date: "Your Taskless environment is up to date and you can proceed." + - Otherwise: "Your Taskless environment is out of date, the following steps are needed in order for commands to work reliably", followed by one numbered step per stale part, each naming what is stale, why it matters, and the exact command that fixes it. +- Three parts are measured, in this order: + - **cli**: a newer version of this build's package is published as `latest`. It is asked of the npm registry alone with a 1s timeout, and the answer is accepted only as strict semver, since it ends up in a command the user runs. The lookup stays silent on failure, and is skipped in CI. `--anonymous` does not skip it: that flag means "skip the Taskless API", and the npm registry is not it. + - **skill**: the installed skill is not the version this CLI carries, as `info` already reports. + - **rules**: the rules owe a reconcile, as `info` and `init` already report. +- `ok` reads `.taskless/` but never writes it, and it always exits 0. +- `taskless check`, when not in CI, runs the same measurement alongside its scan and, when something is out of date, adds one notice saying so and naming the fix. That notice is a `Notice:` line in text output and an entry in `notices` under `--json`. It never changes the exit code or the results. +- The skill's preflight section tells the agent what the `ok` recipe's two outcomes mean. On out of date, the agent hands the steps to the user to run themselves (with `!` in Claude Code, or in another terminal), because a harness that blocks Taskless commands blocks these too, and then re-runs `agent ok` to confirm. Otherwise the preflight stays once per session. +- **BREAKING (output format)**: `agent ok` no longer prints the bare confirmation sentence. Its only consumer is the skill shipped in the same package. + +## Capabilities + +### New Capabilities + +None. + +### Modified Capabilities + +- `cli-agent`: the "ok topic confirms the CLI can run" requirement changes from a built-in one-line confirmation to a recipe that reports whether the environment is up to date. +- `cli-check`: a new requirement for the out-of-date notice outside CI. +- `skill-taskless`: a new requirement that the skill acts on the `ok` recipe's out-of-date steps. + +## Impact + +- `packages/cli/src/agent/ok.md` (new) and `packages/cli/src/prompts/recipes.ts`: a new caller-measured passage variable, following the existing `HOST_TOOLS` pattern. +- A new freshness module (registry lookup plus the existing local checks), shared by `agent` and `check`. +- `packages/cli/src/commands/agent.ts`: the built-in `ok` branch is removed, and `ok` is served through the recipe path. +- `packages/cli/src/commands/check.ts`: the notice. +- `skills/taskless/SKILL.md`: the preflight section, with a skill version bump. +- The `agent ok` command line is unchanged, so the permission rules `init` writes still match. +- `agent ok` and local `check` gain one small GET to `registry.npmjs.org`, bounded by a 1s timeout. No other host is contacted. + +## Delivery + +Single PR: one recipe, one shared module, two command call sites, one skill paragraph, and their tests, within the ~1200-line guideline. diff --git a/openspec/changes/archive/2026-11-03-agent-ok-update-notice/specs/cli-agent/spec.md b/openspec/changes/archive/2026-11-03-agent-ok-update-notice/specs/cli-agent/spec.md new file mode 100644 index 00000000..f29dc508 --- /dev/null +++ b/openspec/changes/archive/2026-11-03-agent-ok-update-notice/specs/cli-agent/spec.md @@ -0,0 +1,118 @@ +## MODIFIED Requirements + +### Requirement: ok topic confirms the CLI can run + +`taskless agent ok` SHALL serve the recipe at `packages/cli/src/agent/ok.md` through the same render path as every other topic, and exit 0. Serving it at all confirms the CLI can run. The rendered recipe SHALL carry the `# Topic:` header and the one-command-per-call paragraph. It SHALL NOT carry the fetch-time directive, because the preflight runs once per session rather than once per task, and SHALL NOT carry a survey invite. + +The recipe's body SHALL be one of two passages, chosen from the environment state the CLI measured when the recipe was fetched: + +- When nothing is out of date: `Your Taskless environment is up to date and you can proceed.` +- When one or more parts are out of date: `Your Taskless environment is out of date, the following steps are needed in order for commands to work reliably:`, followed by one numbered step per out-of-date part, in the order cli, skill, rules. Each step SHALL name what is out of date, say why it matters, and give the exact command that fixes it, spelled with the detected invocation. After the steps, the passage SHALL tell the agent to give the steps to the user to run themselves, rather than running them, and to run ` agent ok` again once the user says they are done. + +The three parts SHALL be decided as follows: + +- **cli**: a newer version of the package this build is (`@taskless/cli` for a release build, `@taskless/cli-nightly` for a nightly) is published as `latest` than the running version, by semver precedence. The latest version SHALL be asked of the npm registry only, at most once per run, with a timeout of at most one second. The answer SHALL be used only when it is a strict semantic version (`MAJOR.MINOR.PATCH` with optional prerelease and build identifiers, and nothing else), because it is written into a command the user runs. A lookup that fails, times out, or returns a missing or invalid version SHALL leave this part out, and SHALL NOT change the output's form or the exit code. The lookup SHALL be skipped, and the part left out, in a CI environment, when `NO_UPDATE_NOTIFIER` is set to a non-empty value (the opt-out convention many CLIs honor), and on a build that is not a published package. The fixing command SHALL run that package at exactly the newer version with no arguments. +- **skill**: the Taskless skill installed in the project is not the version this CLI carries, by the same comparison `taskless info` reports. The fixing command SHALL be the invocation run with no arguments. When the cli part is also present, the skill step SHALL say that the cli step's command also refreshes the skill and SHALL give no separate command. +- **rules**: the project's rules are not reconciled to this CLI, by the same status `taskless info` and `taskless init` report. The fixing command SHALL be ` update`. + +In a directory with no `.taskless/`, the skill and rules parts SHALL be left out. The command MAY read `.taskless/`, and SHALL NOT create or write it. + +When the recipe is rendered by a caller that supplies no measured state, as the `@taskless/cli/prompts` export does, the body SHALL say that the environment was not checked, and SHALL claim neither outcome. + +It SHALL emit the command's `cli_agent` intent event with `topic: "ok"`. The topic SHALL appear in the `taskless agent` index. + +#### Scenario: ok prints one confirmation line + +- **WHEN** `taskless agent ok` is run in a project where nothing is out of date +- **THEN** stdout SHALL begin with the `# Topic: ok` header line +- **AND** SHALL contain `Your Taskless environment is up to date and you can proceed.` +- **AND** the CLI SHALL exit with code 0 + +#### Scenario: ok carries no fetch-time directive + +- **WHEN** `taskless agent ok` is run +- **THEN** its output SHALL NOT contain the fetch-time directive + +#### Scenario: ok names the detected invocation + +- **WHEN** `taskless agent ok` is run by a launcher the CLI detects, and reports an out-of-date part +- **THEN** each fixing command SHALL be spelled with that launcher + +#### Scenario: ok does not guess an unknown launcher + +- **WHEN** `taskless agent ok` is run, the launcher cannot be determined, and an out-of-date part is reported +- **THEN** each fixing command SHALL be spelled with the same agent-fill marker every recipe uses for an unknown invocation, and SHALL NOT name a guessed launcher + +#### Scenario: ok runs outside a Taskless project + +- **WHEN** `taskless agent ok` is run in a directory with no `.taskless/` +- **THEN** it SHALL serve the recipe and exit 0 +- **AND** SHALL report neither the skill nor the rules part +- **AND** SHALL NOT create `.taskless/` + +#### Scenario: ok reports a newer published CLI + +- **WHEN** `taskless agent ok` is run and the published `latest` for this build's package is newer than the running version +- **THEN** the output SHALL contain the out-of-date passage +- **AND** a step SHALL name the running version and the newer one +- **AND** SHALL give a command that runs that package at exactly the newer version with no arguments + +#### Scenario: The out-of-date passage hands the steps to the user + +- **WHEN** `taskless agent ok` reports any out-of-date part +- **THEN** the output SHALL tell the agent to have the user run the steps +- **AND** SHALL tell the agent to run ` agent ok` again once the user is done + +#### Scenario: A failed registry lookup is silent + +- **WHEN** `taskless agent ok` is run, nothing local is out of date, and the registry lookup fails or exceeds its timeout +- **THEN** the output SHALL contain the up-to-date passage +- **AND** the CLI SHALL exit with code 0 + +#### Scenario: An invalid version is ignored + +- **WHEN** `taskless agent ok` is run and the registry returns a `latest` that is not a strict semantic version +- **THEN** the cli part SHALL be left out +- **AND** the output SHALL NOT contain that string + +#### Scenario: The registry is not asked in CI + +- **WHEN** `taskless agent ok` is run in a CI environment, or with `NO_UPDATE_NOTIFIER` set +- **THEN** the CLI SHALL NOT contact the registry + +#### Scenario: ok reports an out-of-date skill + +- **WHEN** `taskless agent ok` is run in a project whose installed Taskless skill is not the version this CLI carries, and no newer CLI is reported +- **THEN** the output SHALL contain the out-of-date passage +- **AND** a step SHALL give the invocation run with no arguments + +#### Scenario: A newer CLI subsumes the skill refresh + +- **WHEN** `taskless agent ok` reports both the cli and skill parts +- **THEN** the skill step SHALL point to the cli step's command and SHALL give no command of its own + +#### Scenario: ok reports rules owing a reconcile + +- **WHEN** `taskless agent ok` is run in a project whose rules are not reconciled to this CLI +- **THEN** a step SHALL give ` update` + +#### Scenario: ok does not write the project + +- **WHEN** `taskless agent ok` is run in a project with a `.taskless/` directory +- **THEN** no file under `.taskless/` SHALL be created or modified + +#### Scenario: The prompts export claims no outcome + +- **WHEN** the `ok` recipe is rendered with no measured state +- **THEN** it SHALL say the environment was not checked +- **AND** SHALL contain neither the up-to-date nor the out-of-date passage + +#### Scenario: ok is in the index + +- **WHEN** `taskless agent` is run with no arguments +- **THEN** the index SHALL include a row for `ok` + +#### Scenario: ok captures its topic + +- **WHEN** `taskless agent ok` is run +- **THEN** PostHog SHALL receive a `cli_agent` event whose `topic` property is `ok` diff --git a/openspec/changes/archive/2026-11-03-agent-ok-update-notice/specs/cli-check/spec.md b/openspec/changes/archive/2026-11-03-agent-ok-update-notice/specs/cli-check/spec.md new file mode 100644 index 00000000..f22e8b0a --- /dev/null +++ b/openspec/changes/archive/2026-11-03-agent-ok-update-notice/specs/cli-check/spec.md @@ -0,0 +1,40 @@ +## ADDED Requirements + +### Requirement: Check warns when Taskless is out of date outside CI + +When `taskless check` runs outside a CI environment, it SHALL measure whether the Taskless environment is out of date, with the same cli, skill, and rules parts and the same rules that `taskless agent ok` uses, and SHALL do so alongside the scan rather than before it. When one or more parts are out of date, `check` SHALL publish one notice that names each out-of-date part and the command that fixes it. In text output, the notice SHALL print through the same marked-notice path as every other notice. Under `--json`, it SHALL be an entry in the `notices` array. + +The measurement SHALL NOT change the exit code, the results, or which rules run. A latest-version lookup that fails or times out SHALL leave the cli part out, silently. In a CI environment, `check` SHALL NOT measure, SHALL NOT contact the registry, and SHALL publish no such notice. When `NO_UPDATE_NOTIFIER` is set to a non-empty value, `check` SHALL NOT measure, SHALL NOT contact the registry, and SHALL publish no such notice. The notice SHALL be a single line, numbering its steps inline, so that each `--json` `notices` element is one line. + +#### Scenario: Out-of-date environment adds a notice locally + +- **WHEN** `taskless check` runs outside CI in a project whose environment is out of date +- **THEN** its output SHALL carry one notice naming the out-of-date parts and their fixing commands +- **AND** the exit code SHALL be the one the scan alone would produce + +#### Scenario: The notice is in the JSON envelope + +- **WHEN** `taskless check --json` runs outside CI in a project whose environment is out of date +- **THEN** the `notices` array SHALL contain the out-of-date notice +- **AND** stdout SHALL remain a single JSON envelope + +#### Scenario: The notice is one line + +- **WHEN** `taskless check --json` reports several out-of-date parts +- **THEN** the out-of-date notice SHALL be one element containing no line break + +#### Scenario: Up-to-date environment adds nothing + +- **WHEN** `taskless check` runs outside CI in a project whose environment is up to date +- **THEN** it SHALL publish no out-of-date notice + +#### Scenario: CI is not measured + +- **WHEN** `taskless check` runs in a CI environment, or with `NO_UPDATE_NOTIFIER` set +- **THEN** it SHALL NOT contact the registry +- **AND** SHALL publish no out-of-date notice + +#### Scenario: A failed lookup does not affect the run + +- **WHEN** `taskless check` runs outside CI and the latest-version lookup fails or times out +- **THEN** the run SHALL complete with the results and exit code the scan alone would produce diff --git a/openspec/changes/archive/2026-11-03-agent-ok-update-notice/specs/skill-taskless/spec.md b/openspec/changes/archive/2026-11-03-agent-ok-update-notice/specs/skill-taskless/spec.md new file mode 100644 index 00000000..7cef0761 --- /dev/null +++ b/openspec/changes/archive/2026-11-03-agent-ok-update-notice/specs/skill-taskless/spec.md @@ -0,0 +1,34 @@ +## ADDED Requirements + +### Requirement: Skill acts on the preflight's environment report + +The skill body SHALL tell the agent that `%(TASKLESS_CLI)s agent ok` returns a recipe whose body reports whether the Taskless environment is up to date. When it reports the environment is up to date, the body SHALL instruct the agent to proceed with the user's request. + +When it reports the environment is out of date, the body SHALL instruct the agent to give the user the listed steps and their commands once, for the user to run themselves, and SHALL NOT instruct the agent to run them. The body SHALL name the ways to run them: in Claude Code, typing the command with the `!` prefix, which runs it in the session's shell; or in another terminal. The reason is that a harness whose permissions block Taskless commands usually blocks these steps too. The body SHALL instruct the agent to carry on with the user's request, and, when the user says they ran the steps, to run `%(TASKLESS_CLI)s agent ok` again to confirm. + +That confirming run is the only time the body SHALL instruct the agent to run the preflight again in a session once it has passed. + +The `!` prefix offered here is distinct from the blocked-preflight guidance, which does not offer it. There, `!` would clear only the one preflight call and leave every later Taskless command blocked. Here, the steps are one-off commands, and running them once is the whole job. + +#### Scenario: Skill body explains both outcomes + +- **WHEN** the skill body is read from source +- **THEN** it SHALL describe the up-to-date and the out-of-date outcomes of `agent ok` + +#### Scenario: Skill body hands the steps to the user + +- **WHEN** the skill body is read from source +- **THEN** it SHALL instruct the agent, on an out-of-date report, to give the user the listed steps once, to run themselves +- **AND** SHALL name the `!` prefix in Claude Code and another terminal as the ways to run them +- **AND** SHALL NOT instruct the agent to run the steps itself +- **AND** SHALL instruct the agent to carry on with the user's request + +#### Scenario: Skill body re-checks after the user updates + +- **WHEN** the skill body is read from source +- **THEN** it SHALL instruct the agent, once the user says they ran the steps, to run `agent ok` again + +#### Scenario: Skill body runs the preflight once otherwise + +- **WHEN** the skill body is read from source +- **THEN** it SHALL NOT instruct the agent to run `agent ok` again in a session after it has passed, except to confirm the user's update diff --git a/openspec/changes/archive/2026-11-03-agent-ok-update-notice/tasks.md b/openspec/changes/archive/2026-11-03-agent-ok-update-notice/tasks.md new file mode 100644 index 00000000..23a307a6 --- /dev/null +++ b/openspec/changes/archive/2026-11-03-agent-ok-update-notice/tasks.md @@ -0,0 +1,23 @@ +## 1. Freshness measurement + +- [x] 1.1 Add a latest-version lookup that takes a package name and an injectable `fetch`, asks the registry's dist-tags endpoint with a 1s timeout, and returns `latest` only when it is strict semver, otherwise `undefined`. Verify with unit tests covering success, a prerelease version, a non-2xx response, a throw, the timeout, a payload with no `latest`, and rejected strings (`latest`, `1.0`, `1.0.0 && echo x`, `1.0.0\n`, a leading `v`). +- [x] 1.2 Add `measureEnvironment({ cwd, fetch, ci })`: it runs the lookup (skipped in CI, when `NO_UPDATE_NOTIFIER` is set, and when `buildLauncherPackage()` is `undefined`), `checkStaleness`, and `reconcileStatus` in parallel, and returns the out-of-date parts in cli/skill/rules order with their details, leaving out skill and rules when there is no `.taskless/`. Verify with unit tests for each part alone, all three together, nothing stale, no `.taskless/`, and CI (fetch never called). + +## 2. The `ok` recipe + +- [x] 2.1 Add `ENVIRONMENT_STATUS` to `prompts/recipes.ts`, with `RecipeOptions.environment`, an exported `ENVIRONMENT_VARIABLES`, and the three passages (up to date, out of date with numbered steps, not checked). Steps are spelled with the resolved invocation; the cli step pins the exact version; when cli is present, the skill step points to it and gives no command of its own; the rules step reuses `getReconcileNotice`. Verify with renderer unit tests for each passage, and that rendering with no options gives the not-checked passage. +- [x] 2.2 Add `packages/cli/src/agent/ok.md` (header, `%(ONE_COMMAND_PER_CALL)s`, `%(ENVIRONMENT_STATUS)s`). Remove the built-in `ok` branch and `PREFLIGHT_TOPIC` special-casing from `commands/agent.ts`, serving `ok` through the recipe path: measure when the template uses `ENVIRONMENT_VARIABLES`, pass `directive: false` for `ok`, and keep it listed under "Preflight" in the index. Update `test/agent-ok.test.ts` for the header, both outcomes, no directive, no `.taskless/` created, and no file under an existing `.taskless/` modified. Verify that the CLI tests and the recipe and prompts-export parity tests pass. + +## 3. `check` notice + +- [x] 3.1 In `commands/check.ts`, when `isCiEnvironment(process.env.CI)` is false, start `measureEnvironment` alongside the scan, await it before printing notices, and publish one out-of-date notice through the existing notice path. Verify with tests: the notice appears in text and in `--json` `notices`; the exit code and results are unchanged; nothing is measured in CI; a failed lookup changes nothing. + +## 4. Skill and release + +- [x] 4.1 Update the preflight section of `skills/taskless/SKILL.md`: `agent ok` returns a recipe that says whether the environment is up to date; if it is, proceed; if it is out of date, give the user the steps to run themselves (with `!` in Claude Code, or in another terminal) rather than running them, carry on with the request, and when the user says they ran them, run `agent ok` again to confirm; otherwise do not rerun the preflight once it has passed. Leave the skill version alone: it tracks the CLI package version, which the changeset moves at release (`scripts/sync-skill-versions.ts` is a no-op here). Verify that the skill tests pass and that `pnpm build` passes its skill-version assertion. +- [x] 4.2 Add a `patch` changeset for `@taskless/cli` describing the `ok` environment report and the `check` notice. Verify that `.changeset/` holds the new file. + +## 5. Checks + +- [x] 5.1 Run `pnpm typecheck` and `pnpm lint`, and fix anything they report. +- [x] 5.2 Run `pnpm build`, then `pnpm cli agent ok` in this repo and in an empty temporary directory, and `pnpm cli check` here, and confirm the recipe renders, both exit 0, the empty directory gains no `.taskless/`, and `check`'s exit code matches a run with `CI=1`. diff --git a/openspec/specs/cli-agent/spec.md b/openspec/specs/cli-agent/spec.md index 7e34ee6b..d4765e85 100644 --- a/openspec/specs/cli-agent/spec.md +++ b/openspec/specs/cli-agent/spec.md @@ -427,32 +427,111 @@ An agent topic `report` SHALL be registered. The CLI SHALL embed `packages/cli/s ### Requirement: ok topic confirms the CLI can run -`taskless agent ok` SHALL print exactly one line, `This message confirms you can run Taskless via .`, and exit 0. `` SHALL be the invocation detected for this process. When the launcher cannot be determined, the line SHALL read `This message confirms you can run Taskless via the command you just ran.` rather than guessing a launcher. +`taskless agent ok` SHALL serve the recipe at `packages/cli/src/agent/ok.md` through the same render path as every other topic, and exit 0. Serving it at all confirms the CLI can run. The rendered recipe SHALL carry the `# Topic:` header and the one-command-per-call paragraph. It SHALL NOT carry the fetch-time directive, because the preflight runs once per session rather than once per task, and SHALL NOT carry a survey invite. -The topic is built into the `agent` command, not a recipe file. It SHALL NOT carry the `# Topic:` header, the fetch-time directive, or a survey invite, and it SHALL NOT read or create `.taskless/`. It SHALL emit the command's `cli_agent` intent event with `topic: "ok"`. The topic SHALL appear in the `taskless agent` index. +The recipe's body SHALL be one of two passages, chosen from the environment state the CLI measured when the recipe was fetched: + +- When nothing is out of date: `Your Taskless environment is up to date and you can proceed.` +- When one or more parts are out of date: `Your Taskless environment is out of date, the following steps are needed in order for commands to work reliably:`, followed by one numbered step per out-of-date part, in the order cli, skill, rules. Each step SHALL name what is out of date, say why it matters, and give the exact command that fixes it, spelled with the detected invocation. After the steps, the passage SHALL tell the agent to give the steps to the user to run themselves, rather than running them, and to run ` agent ok` again once the user says they are done. + +The three parts SHALL be decided as follows: + +- **cli**: a newer version of the package this build is (`@taskless/cli` for a release build, `@taskless/cli-nightly` for a nightly) is published as `latest` than the running version, by semver precedence. The latest version SHALL be asked of the npm registry only, at most once per run, with a timeout of at most one second. The answer SHALL be used only when it is a strict semantic version (`MAJOR.MINOR.PATCH` with optional prerelease and build identifiers, and nothing else), because it is written into a command the user runs. A lookup that fails, times out, or returns a missing or invalid version SHALL leave this part out, and SHALL NOT change the output's form or the exit code. The lookup SHALL be skipped, and the part left out, in a CI environment, when `NO_UPDATE_NOTIFIER` is set to a non-empty value (the opt-out convention many CLIs honor), and on a build that is not a published package. The fixing command SHALL run that package at exactly the newer version with no arguments. +- **skill**: the Taskless skill installed in the project is not the version this CLI carries, by the same comparison `taskless info` reports. The fixing command SHALL be the invocation run with no arguments. When the cli part is also present, the skill step SHALL say that the cli step's command also refreshes the skill and SHALL give no separate command. +- **rules**: the project's rules are not reconciled to this CLI, by the same status `taskless info` and `taskless init` report. The fixing command SHALL be ` update`. + +In a directory with no `.taskless/`, the skill and rules parts SHALL be left out. The command MAY read `.taskless/`, and SHALL NOT create or write it. + +When the recipe is rendered by a caller that supplies no measured state, as the `@taskless/cli/prompts` export does, the body SHALL say that the environment was not checked, and SHALL claim neither outcome. + +It SHALL emit the command's `cli_agent` intent event with `topic: "ok"`. The topic SHALL appear in the `taskless agent` index. #### Scenario: ok prints one confirmation line -- **WHEN** `taskless agent ok` is run -- **THEN** stdout SHALL be a single line beginning `This message confirms you can run Taskless via` +- **WHEN** `taskless agent ok` is run in a project where nothing is out of date +- **THEN** stdout SHALL begin with the `# Topic: ok` header line +- **AND** SHALL contain `Your Taskless environment is up to date and you can proceed.` - **AND** the CLI SHALL exit with code 0 +#### Scenario: ok carries no fetch-time directive + +- **WHEN** `taskless agent ok` is run +- **THEN** its output SHALL NOT contain the fetch-time directive + #### Scenario: ok names the detected invocation -- **WHEN** `taskless agent ok` is run by a launcher the CLI detects -- **THEN** the line SHALL name that launcher and the pinned package specifier +- **WHEN** `taskless agent ok` is run by a launcher the CLI detects, and reports an out-of-date part +- **THEN** each fixing command SHALL be spelled with that launcher #### Scenario: ok does not guess an unknown launcher -- **WHEN** `taskless agent ok` is run and the launcher cannot be determined -- **THEN** the line SHALL say `via the command you just ran` and SHALL NOT name a launcher +- **WHEN** `taskless agent ok` is run, the launcher cannot be determined, and an out-of-date part is reported +- **THEN** each fixing command SHALL be spelled with the same agent-fill marker every recipe uses for an unknown invocation, and SHALL NOT name a guessed launcher #### Scenario: ok runs outside a Taskless project - **WHEN** `taskless agent ok` is run in a directory with no `.taskless/` -- **THEN** it SHALL print the confirmation line and exit 0 +- **THEN** it SHALL serve the recipe and exit 0 +- **AND** SHALL report neither the skill nor the rules part - **AND** SHALL NOT create `.taskless/` +#### Scenario: ok reports a newer published CLI + +- **WHEN** `taskless agent ok` is run and the published `latest` for this build's package is newer than the running version +- **THEN** the output SHALL contain the out-of-date passage +- **AND** a step SHALL name the running version and the newer one +- **AND** SHALL give a command that runs that package at exactly the newer version with no arguments + +#### Scenario: The out-of-date passage hands the steps to the user + +- **WHEN** `taskless agent ok` reports any out-of-date part +- **THEN** the output SHALL tell the agent to have the user run the steps +- **AND** SHALL tell the agent to run ` agent ok` again once the user is done + +#### Scenario: A failed registry lookup is silent + +- **WHEN** `taskless agent ok` is run, nothing local is out of date, and the registry lookup fails or exceeds its timeout +- **THEN** the output SHALL contain the up-to-date passage +- **AND** the CLI SHALL exit with code 0 + +#### Scenario: An invalid version is ignored + +- **WHEN** `taskless agent ok` is run and the registry returns a `latest` that is not a strict semantic version +- **THEN** the cli part SHALL be left out +- **AND** the output SHALL NOT contain that string + +#### Scenario: The registry is not asked in CI + +- **WHEN** `taskless agent ok` is run in a CI environment, or with `NO_UPDATE_NOTIFIER` set +- **THEN** the CLI SHALL NOT contact the registry + +#### Scenario: ok reports an out-of-date skill + +- **WHEN** `taskless agent ok` is run in a project whose installed Taskless skill is not the version this CLI carries, and no newer CLI is reported +- **THEN** the output SHALL contain the out-of-date passage +- **AND** a step SHALL give the invocation run with no arguments + +#### Scenario: A newer CLI subsumes the skill refresh + +- **WHEN** `taskless agent ok` reports both the cli and skill parts +- **THEN** the skill step SHALL point to the cli step's command and SHALL give no command of its own + +#### Scenario: ok reports rules owing a reconcile + +- **WHEN** `taskless agent ok` is run in a project whose rules are not reconciled to this CLI +- **THEN** a step SHALL give ` update` + +#### Scenario: ok does not write the project + +- **WHEN** `taskless agent ok` is run in a project with a `.taskless/` directory +- **THEN** no file under `.taskless/` SHALL be created or modified + +#### Scenario: The prompts export claims no outcome + +- **WHEN** the `ok` recipe is rendered with no measured state +- **THEN** it SHALL say the environment was not checked +- **AND** SHALL contain neither the up-to-date nor the out-of-date passage + #### Scenario: ok is in the index - **WHEN** `taskless agent` is run with no arguments diff --git a/openspec/specs/cli-check/spec.md b/openspec/specs/cli-check/spec.md index 32911d95..16b582df 100644 --- a/openspec/specs/cli-check/spec.md +++ b/openspec/specs/cli-check/spec.md @@ -813,3 +813,42 @@ The remedy SHALL be specific to the cause: - **WHEN** `taskless check --json` runs unverified - **THEN** the `notices` array SHALL contain the notice - **AND** `success`, `results`, and `skipped` SHALL keep their existing meaning + +### Requirement: Check warns when Taskless is out of date outside CI + +When `taskless check` runs outside a CI environment, it SHALL measure whether the Taskless environment is out of date, with the same cli, skill, and rules parts and the same rules that `taskless agent ok` uses, and SHALL do so alongside the scan rather than before it. When one or more parts are out of date, `check` SHALL publish one notice that names each out-of-date part and the command that fixes it. In text output, the notice SHALL print through the same marked-notice path as every other notice. Under `--json`, it SHALL be an entry in the `notices` array. + +The measurement SHALL NOT change the exit code, the results, or which rules run. A latest-version lookup that fails or times out SHALL leave the cli part out, silently. In a CI environment, `check` SHALL NOT measure, SHALL NOT contact the registry, and SHALL publish no such notice. When `NO_UPDATE_NOTIFIER` is set to a non-empty value, `check` SHALL NOT measure, SHALL NOT contact the registry, and SHALL publish no such notice. The notice SHALL be a single line, numbering its steps inline, so that each `--json` `notices` element is one line. + +#### Scenario: Out-of-date environment adds a notice locally + +- **WHEN** `taskless check` runs outside CI in a project whose environment is out of date +- **THEN** its output SHALL carry one notice naming the out-of-date parts and their fixing commands +- **AND** the exit code SHALL be the one the scan alone would produce + +#### Scenario: The notice is in the JSON envelope + +- **WHEN** `taskless check --json` runs outside CI in a project whose environment is out of date +- **THEN** the `notices` array SHALL contain the out-of-date notice +- **AND** stdout SHALL remain a single JSON envelope + +#### Scenario: The notice is one line + +- **WHEN** `taskless check --json` reports several out-of-date parts +- **THEN** the out-of-date notice SHALL be one element containing no line break + +#### Scenario: Up-to-date environment adds nothing + +- **WHEN** `taskless check` runs outside CI in a project whose environment is up to date +- **THEN** it SHALL publish no out-of-date notice + +#### Scenario: CI is not measured + +- **WHEN** `taskless check` runs in a CI environment, or with `NO_UPDATE_NOTIFIER` set +- **THEN** it SHALL NOT contact the registry +- **AND** SHALL publish no out-of-date notice + +#### Scenario: A failed lookup does not affect the run + +- **WHEN** `taskless check` runs outside CI and the latest-version lookup fails or times out +- **THEN** the run SHALL complete with the results and exit code the scan alone would produce diff --git a/openspec/specs/skill-taskless/spec.md b/openspec/specs/skill-taskless/spec.md index 79109208..e41dee66 100644 --- a/openspec/specs/skill-taskless/spec.md +++ b/openspec/specs/skill-taskless/spec.md @@ -255,3 +255,36 @@ The skill SHALL say that a "No such file or directory" error from the launcher m - **WHEN** the installed skill is read - **THEN** it SHALL say a "No such file or directory" error from the launcher means the shell is not at the project root - **AND** it SHALL NOT suggest running the CLI through `npx` or `node` instead + +### Requirement: Skill acts on the preflight's environment report + +The skill body SHALL tell the agent that `%(TASKLESS_CLI)s agent ok` returns a recipe whose body reports whether the Taskless environment is up to date. When it reports the environment is up to date, the body SHALL instruct the agent to proceed with the user's request. + +When it reports the environment is out of date, the body SHALL instruct the agent to give the user the listed steps and their commands once, for the user to run themselves, and SHALL NOT instruct the agent to run them. The body SHALL name the ways to run them: in Claude Code, typing the command with the `!` prefix, which runs it in the session's shell; or in another terminal. The reason is that a harness whose permissions block Taskless commands usually blocks these steps too. The body SHALL instruct the agent to carry on with the user's request, and, when the user says they ran the steps, to run `%(TASKLESS_CLI)s agent ok` again to confirm. + +That confirming run is the only time the body SHALL instruct the agent to run the preflight again in a session once it has passed. + +The `!` prefix offered here is distinct from the blocked-preflight guidance, which does not offer it. There, `!` would clear only the one preflight call and leave every later Taskless command blocked. Here, the steps are one-off commands, and running them once is the whole job. + +#### Scenario: Skill body explains both outcomes + +- **WHEN** the skill body is read from source +- **THEN** it SHALL describe the up-to-date and the out-of-date outcomes of `agent ok` + +#### Scenario: Skill body hands the steps to the user + +- **WHEN** the skill body is read from source +- **THEN** it SHALL instruct the agent, on an out-of-date report, to give the user the listed steps once, to run themselves +- **AND** SHALL name the `!` prefix in Claude Code and another terminal as the ways to run them +- **AND** SHALL NOT instruct the agent to run the steps itself +- **AND** SHALL instruct the agent to carry on with the user's request + +#### Scenario: Skill body re-checks after the user updates + +- **WHEN** the skill body is read from source +- **THEN** it SHALL instruct the agent, once the user says they ran the steps, to run `agent ok` again + +#### Scenario: Skill body runs the preflight once otherwise + +- **WHEN** the skill body is read from source +- **THEN** it SHALL NOT instruct the agent to run `agent ok` again in a session after it has passed, except to confirm the user's update diff --git a/packages/cli/src/agent/ok.md b/packages/cli/src/agent/ok.md new file mode 100644 index 00000000..e7df3fe9 --- /dev/null +++ b/packages/cli/src/agent/ok.md @@ -0,0 +1,13 @@ +# Topic: ok (CLI v%(CLI_VERSION)s / topic v1) + +%(ONE_COMMAND_PER_CALL)s + +## Goal + +Confirm you can run Taskless here, and learn whether the Taskless +environment needs updating before you rely on it. Receiving this text +is the confirmation: the command you ran works. + +## Environment + +%(ENVIRONMENT_STATUS)s diff --git a/packages/cli/src/commands/agent.ts b/packages/cli/src/commands/agent.ts index fd4664d8..e59a5ada 100644 --- a/packages/cli/src/commands/agent.ts +++ b/packages/cli/src/commands/agent.ts @@ -11,17 +11,22 @@ import { import { getTelemetry } from "../telemetry"; import { detectHostTools } from "../detect/host-tools"; import { detectRepository } from "../detect/scan"; +import { environmentSteps, measureEnvironment } from "../install/freshness"; import { + ENVIRONMENT_VARIABLES, HOST_TOOL_VARIABLES, LINTER_VARIABLES, getRawRecipe, getRecipe, + PACKAGE_MANAGER_DLX_MARKER, + resolveInvocation, toolVariants, } from "../prompts/recipes"; import { withSurveyInvite } from "../survey/invite"; import { applyCliInvocation } from "../util/invocation"; import { detectCliInvocation, + detectLauncherCommand, getCliPrefix, processLauncherContext, } from "../util/package-manager"; @@ -53,18 +58,15 @@ const FEEDBACK_TOPICS: ReadonlyArray<[string, string]> = [ ]; /** - * The preflight topic. Built into this command rather than kept as a recipe - * file, because it is a confirmation and not a recipe: the recipe template's - * sections and the fetch-time directive ("fetch again next task") would wrap - * one line in instructions that do not apply to it. + * The preflight topic, listed under its own heading in the index. It is an + * ordinary recipe (`agent/ok.md`) served through the same path as every other + * topic, with two differences handled below: its body is the environment + * report measured at fetch time, and it carries no fetch-time directive, + * because the skill runs it once per session rather than once per task. * - * Reaching it at all is the signal. A harness that refuses to run the CLI - * (a permission check that blocks download-and-run commands, say) refuses - * this first, before an agent has started any work, rather than halfway into - * a task whose consent question sits inside a recipe it never received. So it - * touches no project state (no `.taskless/`, manifest, or auth) and is safe to - * run anywhere, first. Its one side effect is the intent event every topic - * captures, with telemetry's usual anonymous-id write. + * Reaching it at all is the confirmation. A harness that refuses to run the + * CLI refuses this first, before an agent has started any work. It reads + * `.taskless/` to judge the environment and writes nothing there. */ const PREFLIGHT_TOPIC: [string, string] = [ "ok", @@ -253,20 +255,6 @@ export function createAgentCommand(subCommands: SubCommandsDef) { const key = positionals[0]!; - if (key === PREFLIGHT_TOPIC[0]) { - telemetry.capture("cli_agent", { topic: key }); - // The detected invocation, the same value recipes are rendered with, - // so the preflight and every later recipe spell the CLI alike. When - // the launcher is unknown, say so rather than fall back to `npx` as - // `getCliPrefix()` would: an agent copies this text, and a guessed - // launcher is worse than none. - const invocation = detectCliInvocation(processLauncherContext()); - console.log( - `This message confirms you can run Taskless via ${invocation ?? "the command you just ran"}.` - ); - return; - } - const tool = args.tool as string | undefined; // Refusal is a statement about a topic's tool variants, so it applies // only to a topic that has them. Elsewhere `--tool` selects nothing and @@ -314,17 +302,41 @@ export function createAgentCommand(subCommands: SubCommandsDef) { ? await detectRepository(cwd) : undefined; const linters = scan?.linters.map((linter) => linter.name); + // The environment report: what is out of date, measured now. Only the + // topics whose template asks for it pay for the registry lookup. + const launcherContext = processLauncherContext(); + const environment = template?.variables.some((name) => + ENVIRONMENT_VARIABLES.has(name) + ) + ? { + steps: environmentSteps( + await measureEnvironment({ + cwd, + ci: process.env.CI, + noUpdateNotifier: process.env.NO_UPDATE_NOTIFIER, + }), + { + invocation: resolveInvocation({ invocation }), + launcher: + detectLauncherCommand(launcherContext) ?? + PACKAGE_MANAGER_DLX_MARKER, + } + ), + } + : undefined; const recipe = getRecipe(key, { anonymous: args.anonymous, tool, invocation, hostTools, linters, + environment, // What this command serves IS a fetch, so the served text says so: // resolved now, fetch again next task, and a session that installed // or upgraded mid-way holds a stale skill. The prompts export leaves // it off; see `RecipeOptions.directive`. - directive: true, + // Except the preflight, which runs once per session. + directive: key !== PREFLIGHT_TOPIC[0], }); if (recipe) { diff --git a/packages/cli/src/commands/check.ts b/packages/cli/src/commands/check.ts index 7c1e5b71..ef0c85ef 100644 --- a/packages/cli/src/commands/check.ts +++ b/packages/cli/src/commands/check.ts @@ -25,7 +25,16 @@ import { resolveRuleSelection, type RuleSelection } from "../rules/rule-filter"; import { scanProject } from "../rules/scan-project"; import { isCiEnvironment } from "../util/interactive"; import { markNotice } from "../util/notices"; -import { getCliPrefix } from "../util/package-manager"; +import { + detectLauncherCommand, + getCliPrefix, + processLauncherContext, +} from "../util/package-manager"; +import { + environmentNotice, + environmentSteps, + measureEnvironment, +} from "../install/freshness"; async function pathExists(absolutePath: string): Promise { try { @@ -195,6 +204,28 @@ function parseTimeoutMs(raw: string | undefined): number | undefined { return Math.round(seconds * 1000); } +/** + * The out-of-date notice for a local run, or `undefined` when there is none to + * give. Never measured in CI, where nobody acts on it, nor under + * `NO_UPDATE_NOTIFIER`, the usual opt-out from update notices. Advisory: a + * failure to measure is no notice, never an error. + */ +async function outOfDateNotice(cwd: string): Promise { + if (isCiEnvironment(process.env.CI)) return undefined; + if ((process.env.NO_UPDATE_NOTIFIER ?? "") !== "") return undefined; + try { + const measured = await measureEnvironment({ cwd, ci: process.env.CI }); + return environmentNotice( + environmentSteps(measured, { + invocation: getCliPrefix(), + launcher: detectLauncherCommand(processLauncherContext()) ?? "npx", + }) + ); + } catch { + return undefined; + } +} + export const checkCommand = defineCommand({ meta: { name: "check", @@ -460,6 +491,12 @@ export const checkCommand = defineCommand({ return; } + // Whether Taskless itself is out of date, measured alongside the scan so + // a local run (or a git hook) waits on the registry only if the scan + // finishes first, and never past the lookup's timeout. Started here, after + // every early return, so a run that scans nothing leaves no request open. + const environment = outOfDateNotice(cwd); + const preserve = Boolean(args["preserve-logs"]); try { const { plan, dispatched, runDirectory } = await scanProject(cwd, { @@ -482,10 +519,13 @@ export const checkCommand = defineCommand({ const results = dispatched.results; for (const notice of dispatched.notices) warnNotice(notice); + const outOfDate = await environment; + if (outOfDate !== undefined) warnNotice(outOfDate); const runNotices = [ ...pathNotices, ...plan.notices, ...dispatched.notices, + ...(outOfDate === undefined ? [] : [outOfDate]), ]; for (const failure of dispatched.failures) warn(`Error: ${failure}`); diff --git a/packages/cli/src/install/freshness.ts b/packages/cli/src/install/freshness.ts new file mode 100644 index 00000000..8c4bd3b4 --- /dev/null +++ b/packages/cli/src/install/freshness.ts @@ -0,0 +1,207 @@ +import { access } from "node:fs/promises"; +import { join } from "node:path"; + +import { readManifest } from "../filesystem/manifest"; +import type { EnvironmentStep } from "../prompts/recipes"; +import { + getReconcileNotice, + reconcileStatus, + type ReconcileStatus, +} from "../rules/reconcile-status"; +import { isCiEnvironment } from "../util/interactive"; +import { compareSemver } from "../util/version-compare"; +import { CLI_VERSION } from "../version"; +import { checkStaleness } from "./install"; +import { fetchLatestVersion } from "./latest-version"; +import { buildLauncherPackage, type LauncherPackage } from "./launcher"; + +/** + * Whether the Taskless environment is out of date, measured once and shared + * by `agent ok` and `check` so the two cannot disagree. + * + * Three parts, each answered by the check that already owns it: a newer + * published CLI (the npm registry), a skill that is not this CLI's (the same + * comparison `info` reports), and rules owing a reconcile (the same status + * `info` and `init` report). Everything here reads; nothing writes. + */ +export interface EnvironmentMeasurement { + cli?: { packageName: LauncherPackage; running: string; latest: string }; + skill?: Array<{ name: string; installed?: string; current: string }>; + rules?: ReconcileStatus; +} + +export interface MeasureOptions { + cwd: string; + /** `process.env.CI`. In CI nothing is measured over the network. */ + ci?: string; + /** + * `process.env.NO_UPDATE_NOTIFIER`, the convention update-notifier and many + * CLIs honour. Any non-empty value turns the registry lookup off. + */ + noUpdateNotifier?: string; + /** Injected for tests; the global `fetch` otherwise. */ + fetch?: typeof fetch; + /** The running version; this build's own otherwise. */ + running?: string; + /** The package this build is; `undefined` for a `self` build. */ + packageName?: LauncherPackage | null; +} + +async function exists(path: string): Promise { + try { + await access(path); + return true; + } catch { + return false; + } +} + +async function measureCli( + options: MeasureOptions +): Promise { + if (isCiEnvironment(options.ci)) return undefined; + if (options.noUpdateNotifier !== undefined && options.noUpdateNotifier !== "") + return undefined; + const packageName = + options.packageName === null + ? undefined + : (options.packageName ?? buildLauncherPackage()); + const running = options.running ?? CLI_VERSION; + if (packageName === undefined || running === "unknown") return undefined; + const latest = await fetchLatestVersion(packageName, { + fetch: options.fetch, + }); + if (latest === undefined || compareSemver(latest, running) <= 0) { + return undefined; + } + return { packageName, running, latest }; +} + +async function measureSkill( + cwd: string +): Promise { + const harnesses = await checkStaleness(cwd); + const stale = new Map< + string, + { name: string; installed?: string; current: string } + >(); + for (const harness of harnesses) { + for (const skill of harness.skills) { + if (skill.current) continue; + stale.set(skill.name, { + name: skill.name, + installed: skill.installedVersion, + current: skill.currentVersion, + }); + } + } + return stale.size === 0 ? undefined : [...stale.values()]; +} + +async function measureRules( + directory: string, + cwd: string +): Promise { + try { + const { manifest } = await readManifest(directory); + const status = await reconcileStatus(cwd, manifest.rules); + return status.required ? status : undefined; + } catch { + // An unreadable manifest is reported by the command that needs it, with + // its own remedy. An advisory check has nothing useful to add. + return undefined; + } +} + +export async function measureEnvironment( + options: MeasureOptions +): Promise { + const directory = join(options.cwd, ".taskless"); + // Outside a `.taskless/` project there is no skill or rules state to judge. + if (!(await exists(directory))) return { cli: await measureCli(options) }; + const [cli, skill, rules] = await Promise.all([ + measureCli(options), + measureSkill(options.cwd), + measureRules(directory, options.cwd), + ]); + return { cli, skill, rules }; +} + +/** + * The steps that bring the environment up to date, in the order they are to + * be run: cli, skill, rules. `update` has to run with the CLI the project + * keeps, so it comes after a CLI move. + * + * `invocation` spells the current CLI. `launcher` is the command word that + * runs a package (`npx`, `pnpm dlx`), used for the newer CLI, which the + * pinned invocation cannot reach. + */ +export function environmentSteps( + measurement: EnvironmentMeasurement, + spelling: { invocation: string; launcher: string } +): EnvironmentStep[] { + const steps: EnvironmentStep[] = []; + const { cli, skill, rules } = measurement; + if (cli !== undefined) { + steps.push({ + part: "cli", + text: + `A newer Taskless CLI is published: ${cli.packageName} ${cli.latest}, and this project runs ${cli.running}. ` + + "Running the newer version with no arguments moves the project to it and refreshes the installed skills.", + command: `${spelling.launcher} ${cli.packageName}@${cli.latest}`, + }); + } + if (skill !== undefined) { + const listed = skill + .map( + ({ name, installed, current }) => + `${name} ${installed ?? "(missing)"}, expected ${current}` + ) + .join("; "); + steps.push( + cli === undefined + ? { + part: "skill", + text: + `The installed Taskless skill is not this CLI's (${listed}), so its instructions may not match the commands. ` + + "Running the CLI with no arguments reinstalls it.", + command: spelling.invocation, + } + : { + part: "skill", + text: `The installed Taskless skill is not this CLI's (${listed}). The newer CLI above refreshes it, so it needs no command of its own.`, + } + ); + } + if (rules !== undefined) { + const notice = + getReconcileNotice(rules, spelling.invocation) ?? + "This project's rules are not reconciled to this CLI."; + steps.push({ + part: "rules", + text: + cli === undefined + ? notice + : `${notice} After moving to the newer CLI, check again: it may owe a different reconcile.`, + command: `${spelling.invocation} update`, + }); + } + return steps; +} + +/** + * The one notice `check` publishes when the environment is out of date, or + * `undefined` when it is not. + */ +export function environmentNotice( + steps: EnvironmentStep[] +): string | undefined { + if (steps.length === 0) return undefined; + // One line: a `--json` consumer reads `notices` one element per notice, and + // text output marks it once. Steps are numbered inline instead. + const parts = steps.map((step, index) => { + const run = step.command === undefined ? "" : ` Run: ${step.command}`; + return `(${String(index + 1)}) ${step.text}${run}`; + }); + return `Your Taskless environment is out of date: ${parts.join(" ")}`; +} diff --git a/packages/cli/src/install/latest-version.ts b/packages/cli/src/install/latest-version.ts new file mode 100644 index 00000000..54c8ad09 --- /dev/null +++ b/packages/cli/src/install/latest-version.ts @@ -0,0 +1,59 @@ +/** + * The newest published version of a package, from the npm registry alone. + * + * One host, on purpose: npm is already the party trusted for the install + * itself, so asking it adds nobody new to the path. A CDN mirror was measured + * faster when warm and rejected, since it would put a second party in front + * of a string that ends up in a command the user runs. + */ + +/** How long a lookup may take before it is abandoned as "unknown". */ +export const LATEST_VERSION_TIMEOUT_MS = 1000; + +/** + * Strict semantic version, as semver.org publishes it: `MAJOR.MINOR.PATCH`, + * an optional `-prerelease`, an optional `+build`, and nothing else. + * + * The answer is spliced into a command a user is told to run, so anything + * looser (a leading `v`, a range, trailing whitespace, a shell operator) is + * refused rather than repaired. + */ +const STRICT_SEMVER = + /^(?:0|[1-9]\d*)\.(?:0|[1-9]\d*)\.(?:0|[1-9]\d*)(?:-(?:(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+[0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*)?$/; + +export function isStrictSemver(value: string): boolean { + return STRICT_SEMVER.test(value); +} + +/** + * The `latest` dist-tag of `packageName`, or `undefined` when the registry + * cannot say within the timeout, answers with anything but a 2xx, or names + * something that is not a strict semantic version. Never throws: an unknown + * answer is an ordinary outcome for an advisory check. + */ +export async function fetchLatestVersion( + packageName: string, + options: { fetch?: typeof fetch; timeoutMs?: number } = {} +): Promise { + const fetcher = options.fetch ?? fetch; + try { + const response = await fetcher( + `https://registry.npmjs.org/-/package/${packageName}/dist-tags`, + { + signal: AbortSignal.timeout( + options.timeoutMs ?? LATEST_VERSION_TIMEOUT_MS + ), + headers: { accept: "application/json" }, + } + ); + if (!response.ok) return undefined; + const body: unknown = await response.json(); + if (typeof body !== "object" || body === null) return undefined; + const latest = (body as Record).latest; + return typeof latest === "string" && isStrictSemver(latest) + ? latest + : undefined; + } catch { + return undefined; + } +} diff --git a/packages/cli/src/prompts/index.ts b/packages/cli/src/prompts/index.ts index 2d27f10f..e8afde0e 100644 --- a/packages/cli/src/prompts/index.ts +++ b/packages/cli/src/prompts/index.ts @@ -102,6 +102,7 @@ export const INTERNAL_TOPICS = [ "improve-rule", "info", "init", + "ok", "onboard", "recover-rule", "report", diff --git a/packages/cli/src/prompts/recipes.ts b/packages/cli/src/prompts/recipes.ts index 65e1693c..1bbbe371 100644 --- a/packages/cli/src/prompts/recipes.ts +++ b/packages/cli/src/prompts/recipes.ts @@ -109,7 +109,7 @@ const TOPIC_INPUT_SCHEMAS: Record = { }; /** Agent-fill marker used when the caller does not supply a real value. */ -const PACKAGE_MANAGER_DLX_MARKER = ""; +export const PACKAGE_MANAGER_DLX_MARKER = ""; /** * Agent-fill marker for the CLI invocation itself. @@ -175,6 +175,21 @@ export interface HostTool { reason?: string; } +/** + * One step that brings the Taskless environment up to date, as the CLI + * measured and worded it (`src/install/freshness.ts`). Pre-rendered on the CLI + * side because measuring it reads the filesystem and the network, which this + * module may not do; the renderer only lays the steps out. + */ +export interface EnvironmentStep { + /** Which part is out of date. */ + part: "cli" | "skill" | "rules"; + /** What is out of date and why it matters, in a sentence or two. */ + text: string; + /** The exact command that fixes it, or absent when another step's does. */ + command?: string; +} + /** Options accepted by the shared render path. */ export interface RecipeOptions { /** @@ -268,6 +283,15 @@ export interface RecipeOptions { * @default false */ directive?: boolean; + /** + * What the CLI measured about the Taskless environment when the recipe was + * fetched, for the `ok` topic's passage. An empty `steps` means up to date. + * + * AN ARGUMENT, NEVER AN AMBIENT READ, like {@link hostTools}. Omitting it + * renders a passage that says the environment was not checked, never one + * that claims either outcome. + */ + environment?: { steps: EnvironmentStep[] }; /** * What the host has on its `PATH`, for the passages that condition on it. * @@ -509,6 +533,51 @@ export const LINTER_VARIABLES: ReadonlySet = new Set([ "LEGACY_LINTERS", ]); +/** + * The variable names whose value depends on {@link RecipeOptions.environment}. + * Exported for the same reason as {@link HOST_TOOL_VARIABLES}: the `agent` + * command asks the template whether the measurement is worth running. + */ +export const ENVIRONMENT_VARIABLES: ReadonlySet = new Set([ + "ENVIRONMENT_STATUS", +]); + +/** + * The `ok` topic's passage: up to date, out of date with its steps, or (with + * nothing measured) not checked. The two measured sentences are fixed text a + * caller can match on. + */ +function environmentStatusBlock( + environment: RecipeOptions["environment"], + invocation: string +): string { + if (environment === undefined) { + return `The Taskless environment was not checked when this text was +rendered, so it says nothing about whether Taskless is up to date.`; + } + if (environment.steps.length === 0) { + return "Your Taskless environment is up to date and you can proceed."; + } + const steps = environment.steps + .map((step, index) => { + const body = `${String(index + 1)}. ${step.text}`; + return step.command === undefined + ? body + : `${body}\n \`\`\`\n ${step.command}\n \`\`\``; + }) + .join("\n\n"); + return `Your Taskless environment is out of date, the following steps are needed in order for commands to work reliably: + +${steps} + +Do not run these steps yourself. Give them to the user, once, to run +themselves: in Claude Code by typing each command with the \`!\` prefix, +which runs it in this session's shell, or in another terminal. A +harness that blocks Taskless commands usually blocks these too. Then +carry on with the user's request. When the user says they ran the +steps, run \`${invocation} agent ok\` again to confirm.`; +} + /** * One paragraph per linter that has a legacy recipe, in the order `route` * prints them. Each says what the tool is for and what step 1 always checks @@ -763,8 +832,15 @@ export function oneCommandPerCall(invocation?: string): string { ); } -/** The invocation a render should use, resolved the same way `TASKLESS_CLI` is. */ -function resolveInvocation(options: RecipeOptions): string { +/** + * The invocation a rendering spells commands with: the caller's, else the + * marker a released build uses when it cannot know its launcher, else this + * build's own. Exported so a CLI-side caller pre-rendering text for a recipe + * (the `ok` environment steps) spells it exactly as the recipe around it. + */ +export function resolveInvocation( + options: Pick +): string { return ( options.invocation ?? (isProductionInvocation() ? TASKLESS_CLI_MARKER : buildInvocation()) @@ -842,6 +918,10 @@ export function buildVariables( " " ), HOST_TOOLS: indentBlock(hostToolsStep(options.hostTools), " "), + ENVIRONMENT_STATUS: environmentStatusBlock( + options.environment, + resolveInvocation(options) + ), LEGACY_LINTERS: legacyLintersBlock( options.linters, options.mechanics === false diff --git a/packages/cli/src/util/package-manager.ts b/packages/cli/src/util/package-manager.ts index 8e0b3543..48745890 100644 --- a/packages/cli/src/util/package-manager.ts +++ b/packages/cli/src/util/package-manager.ts @@ -140,6 +140,19 @@ const LAUNCHER_COMMANDS: Record = { "pnpm-dlx": "pnpm dlx", }; +/** + * The command word of the launcher that started this process (`npx`, + * `pnpm dlx`), or `undefined` when nothing says. For spelling a command that + * runs a DIFFERENT version of this package, where the pinned invocation is + * the wrong thing to reuse. + */ +export function detectLauncherCommand( + context: LauncherContext +): string | undefined { + const detected = detectLauncher(context); + return detected === undefined ? undefined : LAUNCHER_COMMANDS[detected]; +} + /** The prefix a launcher-form build invocation carries. */ const NPX_PREFIX = "npx "; diff --git a/packages/cli/test/agent-ok.test.ts b/packages/cli/test/agent-ok.test.ts index 5957dc13..91be9f08 100644 --- a/packages/cli/test/agent-ok.test.ts +++ b/packages/cli/test/agent-ok.test.ts @@ -1,4 +1,12 @@ -import { existsSync, mkdtempSync, rmSync } from "node:fs"; +import { + existsSync, + mkdirSync, + mkdtempSync, + readdirSync, + rmSync, + statSync, + writeFileSync, +} from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; @@ -33,6 +41,33 @@ async function runAgent( }); } +/** + * Stub the registry: `latest` names the published version, or `undefined` + * for a registry that answers with nothing usable. The suite runs as a + * release build, so without this `agent ok` would ask the real registry. + */ +function stubRegistry(latest?: string) { + const fetcher = vi.fn(() => + Promise.resolve( + latest === undefined + ? new Response("{}", { status: 404 }) + : Response.json({ latest }) + ) + ); + vi.stubGlobal("fetch", fetcher); + return fetcher; +} + +/** Every path under `root` with its size and mtime. */ +function snapshot(root: string): Record { + const entries: Record = {}; + for (const name of readdirSync(root, { recursive: true }) as string[]) { + const stats = statSync(join(root, name)); + entries[name] = `${String(stats.size)}:${String(stats.mtimeMs)}`; + } + return entries; +} + describe("agent ok", () => { let directory: string; let logSpy: ReturnType; @@ -40,6 +75,12 @@ describe("agent ok", () => { beforeEach(() => { capture.mockClear(); + // Cleared so these tests can drive the lookup, which means every test in + // this file must stub `fetch`: the default registry answers nothing, and a + // test that needs a version stubs its own over it. Without this a test that + // never mentions the registry would reach the real one. + vi.stubEnv("NO_UPDATE_NOTIFIER", ""); + stubRegistry(); process.exitCode = undefined; directory = mkdtempSync(join(tmpdir(), "taskless-agent-ok-")); logSpy = vi.spyOn(console, "log").mockImplementation(() => {}); @@ -50,6 +91,7 @@ describe("agent ok", () => { logSpy.mockRestore(); errorSpy.mockRestore(); vi.unstubAllEnvs(); + vi.unstubAllGlobals(); rmSync(directory, { recursive: true, force: true }); }); @@ -57,40 +99,91 @@ describe("agent ok", () => { return logSpy.mock.calls.map((call) => String(call[0])); } - it("prints one confirmation line and succeeds", async () => { + function output(): string { + return printed().join("\n"); + } + + it("serves the recipe with the up-to-date passage and succeeds", async () => { + stubRegistry(); await runAgent(directory, ["agent", "ok"]); - expect(printed()).toHaveLength(1); - expect(printed()[0]).toMatch( - /^This message confirms you can run Taskless via .+\.$/ + expect(output()).toMatch(/^# Topic: ok /); + expect(output()).toContain( + "Your Taskless environment is up to date and you can proceed." ); expect(errorSpy).not.toHaveBeenCalled(); expect(process.exitCode).toBeUndefined(); }); - it("names the detected launcher and package", async () => { + it("carries no fetch-time directive", async () => { + stubRegistry(); + await runAgent(directory, ["agent", "ok"]); + expect(output()).not.toContain("Resolved by the CLI when you fetched it"); + }); + + it("reports a newer published CLI and hands the steps to the user", async () => { vi.stubEnv("npm_command", "exec"); vi.stubEnv("npm_lifecycle_event", "npx"); + stubRegistry("999.0.0"); await runAgent(directory, ["agent", "ok"]); - expect(printed()[0]).toMatch( - /^This message confirms you can run Taskless via npx @taskless\/cli\S*@\S+\.$/ + expect(output()).toContain( + "Your Taskless environment is out of date, the following steps are needed in order for commands to work reliably:" ); + expect(output()).toContain("npx @taskless/cli@999.0.0"); + expect(output()).toContain("Do not run these steps yourself."); + expect(output()).toMatch(/run `npx @taskless\/cli@\S+ agent ok` again/); }); it("does not guess a launcher it cannot detect", async () => { vi.stubEnv("npm_command", ""); vi.stubEnv("npm_lifecycle_event", ""); vi.stubEnv("npm_config_user_agent", ""); + stubRegistry("999.0.0"); + await runAgent(directory, ["agent", "ok"]); + expect(output()).toContain(" @taskless/cli@999.0.0"); + }); + + it("is silent when the registry lookup fails", async () => { + vi.stubGlobal( + "fetch", + vi.fn(() => Promise.reject(new Error("offline"))) + ); await runAgent(directory, ["agent", "ok"]); - expect(printed()[0]).toBe( - "This message confirms you can run Taskless via the command you just ran." + expect(output()).toContain( + "Your Taskless environment is up to date and you can proceed." ); + expect(process.exitCode).toBeUndefined(); + }); + + it("does not ask the registry in CI", async () => { + vi.stubEnv("CI", "true"); + const fetcher = stubRegistry("999.0.0"); + await runAgent(directory, ["agent", "ok"]); + expect(fetcher).not.toHaveBeenCalled(); + }); + + it("does not ask the registry when NO_UPDATE_NOTIFIER is set", async () => { + vi.stubEnv("NO_UPDATE_NOTIFIER", "1"); + const fetcher = stubRegistry("999.0.0"); + await runAgent(directory, ["agent", "ok"]); + expect(fetcher).not.toHaveBeenCalled(); }); it("does not create .taskless/", async () => { + stubRegistry(); await runAgent(directory, ["agent", "ok"]); expect(existsSync(join(directory, ".taskless"))).toBe(false); }); + it("writes nothing under an existing .taskless/", async () => { + stubRegistry(); + const taskless = join(directory, ".taskless"); + mkdirSync(taskless); + writeFileSync(join(taskless, "taskless.json"), '{"version": 1}\n'); + const before = snapshot(taskless); + await runAgent(directory, ["agent", "ok"]); + expect(snapshot(taskless)).toEqual(before); + }); + it("captures cli_agent with the ok topic", async () => { await runAgent(directory, ["agent", "ok"]); expect(capture).toHaveBeenCalledWith("cli_agent", { topic: "ok" }); diff --git a/packages/cli/test/check-environment-notice.test.ts b/packages/cli/test/check-environment-notice.test.ts new file mode 100644 index 00000000..0ae5f2ab --- /dev/null +++ b/packages/cli/test/check-environment-notice.test.ts @@ -0,0 +1,170 @@ +import { cp, mkdtemp, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; + +import { runCommand } from "citty"; +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; + +import type { EnvironmentMeasurement } from "../src/install/freshness"; +import { migrateFixture } from "./support/current-project"; + +/** + * `check`'s out-of-date notice, run in-process so the measurement can be + * controlled. What the measurement finds is `freshness.test.ts`'s subject; + * this file is about how `check` wires it in: one notice, in both output + * forms, never in CI, and never touching the exit code or the results. + */ + +const measureEnvironment = vi.fn<() => Promise>(); +vi.mock("../src/install/freshness", async (importOriginal) => { + const actual = + await importOriginal(); + return { ...actual, measureEnvironment: () => measureEnvironment() }; +}); +vi.mock("../src/telemetry", () => ({ + getTelemetry: vi.fn(() => + Promise.resolve({ capture: vi.fn(), shutdown: () => Promise.resolve() }) + ), + shutdownTelemetry: () => Promise.resolve(), + isTelemetryEnabled: () => false, +})); + +const { checkCommand } = await import("../src/commands/check"); + +const fixturesDirectory = resolve( + import.meta.dirname, + "fixtures/mixed-engines-project" +); + +const NEWER_CLI: EnvironmentMeasurement = { + cli: { packageName: "@taskless/cli", running: "1.0.0", latest: "9.0.0" }, +}; + +interface CheckJson { + success: boolean; + results: unknown[]; + notices?: string[]; +} + +/** Results as sorted JSON strings, since engines finish in any order. */ +function sorted(results: unknown[]): string[] { + return results.map((result) => JSON.stringify(result)).toSorted(); +} + +describe("check out-of-date notice", () => { + let project: string; + let logSpy: ReturnType; + let errorSpy: ReturnType; + + beforeEach(async () => { + project = await mkdtemp(join(tmpdir(), "taskless-check-environment-")); + await cp(fixturesDirectory, project, { recursive: true }); + await migrateFixture(["-d", project]); + measureEnvironment.mockReset(); + vi.stubEnv("NO_UPDATE_NOTIFIER", ""); + process.exitCode = undefined; + logSpy = vi.spyOn(console, "log").mockImplementation(() => {}); + errorSpy = vi.spyOn(console, "error").mockImplementation(() => {}); + }); + + afterEach(async () => { + logSpy.mockRestore(); + errorSpy.mockRestore(); + vi.unstubAllEnvs(); + process.exitCode = undefined; + await rm(project, { recursive: true, force: true }); + }); + + async function check(...extra: string[]): Promise { + await runCommand(checkCommand, { rawArgs: ["-d", project, ...extra] }); + const code = process.exitCode; + process.exitCode = undefined; + return typeof code === "number" ? code : undefined; + } + + function json(): CheckJson { + const line = logSpy.mock.calls.map((call) => String(call[0])).at(-1); + return JSON.parse(line ?? "{}") as CheckJson; + } + + function stderr(): string { + return errorSpy.mock.calls.map((call) => String(call[0])).join("\n"); + } + + it("adds one notice to --json notices without changing the run", async () => { + measureEnvironment.mockResolvedValue({}); + const baselineCode = await check("--json"); + const baseline = json(); + + logSpy.mockClear(); + measureEnvironment.mockResolvedValue(NEWER_CLI); + const code = await check("--json"); + const output = json(); + + expect(code).toBe(baselineCode); + expect(sorted(output.results)).toEqual(sorted(baseline.results)); + const added = (output.notices ?? []).filter((notice) => + notice.includes("out of date") + ); + expect(added).toHaveLength(1); + expect(added[0]).toContain("@taskless/cli@9.0.0"); + }); + + it("prints the notice as a marked line in text output", async () => { + measureEnvironment.mockResolvedValue(NEWER_CLI); + await check(); + expect(stderr()).toContain( + "Notice: Your Taskless environment is out of date:" + ); + }); + + it("adds nothing when the environment is up to date", async () => { + measureEnvironment.mockResolvedValue({}); + await check("--json"); + expect(json().notices ?? []).not.toContainEqual( + expect.stringContaining("out of date") + ); + }); + + it("does not measure in CI", async () => { + vi.stubEnv("CI", "true"); + measureEnvironment.mockResolvedValue(NEWER_CLI); + await check("--json"); + expect(measureEnvironment).not.toHaveBeenCalled(); + expect(json().notices ?? []).not.toContainEqual( + expect.stringContaining("out of date") + ); + }); + + it("does not measure under NO_UPDATE_NOTIFIER", async () => { + vi.stubEnv("NO_UPDATE_NOTIFIER", "1"); + measureEnvironment.mockResolvedValue(NEWER_CLI); + await check("--json"); + expect(measureEnvironment).not.toHaveBeenCalled(); + }); + + it("publishes the notice as one line", async () => { + measureEnvironment.mockResolvedValue({ + ...NEWER_CLI, + skill: [{ name: "taskless", installed: "0.9.0", current: "1.0.0" }], + }); + await check("--json"); + const added = (json().notices ?? []).filter((notice) => + notice.includes("out of date") + ); + expect(added).toHaveLength(1); + expect(added[0]).not.toContain("\n"); + }); + + it("completes normally when the measurement fails", async () => { + measureEnvironment.mockResolvedValue({}); + const baselineCode = await check("--json"); + const baseline = json(); + + logSpy.mockClear(); + measureEnvironment.mockRejectedValue(new Error("boom")); + const code = await check("--json"); + expect(code).toBe(baselineCode); + expect(sorted(json().results)).toEqual(sorted(baseline.results)); + }); +}); diff --git a/packages/cli/test/dogfood-scaffold-current.test.ts b/packages/cli/test/dogfood-scaffold-current.test.ts index d20952fb..7bb46712 100644 --- a/packages/cli/test/dogfood-scaffold-current.test.ts +++ b/packages/cli/test/dogfood-scaffold-current.test.ts @@ -2,6 +2,10 @@ import { readFileSync } from "node:fs"; import { join } from "node:path"; import { describe, expect, it } from "vitest"; import { LATEST_SCHEMA_VERSION } from "../src/filesystem/migrate"; +import { readManifest } from "../src/filesystem/manifest"; +import { reconcileStatus } from "../src/rules/reconcile-status"; + +const repositoryRoot = join(import.meta.dirname, "..", "..", ".."); /** * This repository's own `.taskless/` is on the scaffold version this CLI @@ -27,14 +31,7 @@ import { LATEST_SCHEMA_VERSION } from "../src/filesystem/migrate"; */ describe("the repository's own scaffold", () => { it("is on the latest schema version", () => { - const manifestPath = join( - import.meta.dirname, - "..", - "..", - "..", - ".taskless", - "taskless.json" - ); + const manifestPath = join(repositoryRoot, ".taskless", "taskless.json"); const manifest = JSON.parse(readFileSync(manifestPath, "utf8")) as { version?: unknown; }; @@ -47,3 +44,41 @@ describe("the repository's own scaffold", () => { ).toBe(LATEST_SCHEMA_VERSION); }); }); + +/** + * This repository's own rules are reconciled to the engines this CLI bundles. + * + * `pnpm cli` sets `TASKLESS_ALLOW_VERSION_MISMATCHES=1`, which the dev loop + * needs because a local build is never the version the project records. That + * override also lifts the engine gate, so `pnpm lint` ran the house-style + * rules on engines they were never reconciled against, with no refusal. + * Measured: recorded at ast-grep 0.45.2 and Vale 3.18.0 from 2026-08-29 while + * the CLI bundled 0.51.0 and 3.24.0, with two mechanical rewrites pending. + * + * So the PR that bumps an engine is the one that reconciles our rules, the way + * the scaffold check above makes the PR that adds a migration apply it. + * + * Deliberately NOT `reconcile.required`. That is also true whenever the CLI + * version moves past the marker, which every release does, so asserting it + * would turn the automatic Version Packages PR red on each release. An engine + * bump is where rule matching actually changes, and recording it with + * `update --rules` also stamps the walk. + */ +describe("the repository's own rules", () => { + it("are reconciled to the bundled engines", async () => { + const { manifest } = await readManifest(join(repositoryRoot, ".taskless")); + const status = await reconcileStatus(repositoryRoot, manifest.rules); + const moved = status.engines + .map( + ({ engine, recorded, bundled }) => `${engine} ${recorded} -> ${bundled}` + ) + .join(", "); + + expect( + { engines: status.engines, ruleMigrations: status.ruleMigrations }, + `.taskless/taskless.json records the rules against engines this CLI does not bundle (${moved}). ` + + "Run `pnpm build`, walk `pnpm cli agent update`, then `pnpm cli update --rules`, " + + "and commit the rewritten .taskless/." + ).toEqual({ engines: [], ruleMigrations: [] }); + }); +}); diff --git a/packages/cli/test/freshness.test.ts b/packages/cli/test/freshness.test.ts new file mode 100644 index 00000000..3c5180ab --- /dev/null +++ b/packages/cli/test/freshness.test.ts @@ -0,0 +1,216 @@ +import { mkdirSync, mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; + +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; + +import type { ReconcileStatus } from "../src/rules/reconcile-status"; + +const checkStaleness = vi.fn(); +vi.mock("../src/install/install", () => ({ + checkStaleness: (cwd: string): unknown => checkStaleness(cwd), +})); + +const reconcileStatus = vi.fn(); +vi.mock("../src/rules/reconcile-status", async (importOriginal) => { + const actual = + await importOriginal(); + return { + ...actual, + reconcileStatus: (...args: unknown[]): unknown => reconcileStatus(...args), + }; +}); + +vi.mock("../src/filesystem/manifest", () => ({ + readManifest: () => Promise.resolve({ manifest: { rules: {} }, raw: {} }), +})); + +const { environmentNotice, environmentSteps, measureEnvironment } = + await import("../src/install/freshness"); + +const CURRENT_SKILLS = [ + { + name: "Claude Code", + skills: [ + { + name: "taskless", + installedVersion: "1.0.0", + currentVersion: "1.0.0", + current: true, + }, + ], + }, +]; +const STALE_SKILLS = [ + { + name: "Claude Code", + skills: [ + { + name: "taskless", + installedVersion: "0.9.0", + currentVersion: "1.0.0", + current: false, + }, + ], + }, +]; +const RECONCILED: ReconcileStatus = { + required: false, + walk: null, + engines: [], + ruleMigrations: [], +}; +const OWES_RECONCILE: ReconcileStatus = { + required: true, + walk: { from: "0.9.0", to: "1.0.0" }, + engines: [], + ruleMigrations: [], +}; + +function registry(latest: string): typeof fetch { + return vi.fn(() => + Promise.resolve(Response.json({ latest })) + ) as unknown as typeof fetch; +} + +const SPELLING = { invocation: "npx @taskless/cli@1.0.0", launcher: "npx" }; + +describe("measureEnvironment", () => { + let directory: string; + + beforeEach(() => { + directory = mkdtempSync(join(tmpdir(), "taskless-freshness-")); + checkStaleness.mockReset().mockResolvedValue(CURRENT_SKILLS); + reconcileStatus.mockReset().mockResolvedValue(RECONCILED); + }); + + afterEach(() => { + rmSync(directory, { recursive: true, force: true }); + }); + + function project(): void { + mkdirSync(join(directory, ".taskless")); + } + + function measure(fetcher: typeof fetch, ci?: string) { + return measureEnvironment({ + cwd: directory, + ci, + fetch: fetcher, + running: "1.0.0", + packageName: "@taskless/cli", + }); + } + + it("reports nothing when everything is current", async () => { + project(); + const measured = await measure(registry("1.0.0")); + expect(measured).toEqual({ + cli: undefined, + skill: undefined, + rules: undefined, + }); + expect(environmentSteps(measured, SPELLING)).toEqual([]); + }); + + it("reports a newer published CLI with an exact pin", async () => { + const measured = await measure(registry("1.1.0")); + expect(measured.cli).toEqual({ + packageName: "@taskless/cli", + running: "1.0.0", + latest: "1.1.0", + }); + const [step] = environmentSteps(measured, SPELLING); + expect(step?.part).toBe("cli"); + expect(step?.command).toBe("npx @taskless/cli@1.1.0"); + expect(step?.text).toContain("1.0.0"); + }); + + it("does not report an older published version", async () => { + const measured = await measure(registry("0.9.0")); + expect(measured.cli).toBeUndefined(); + }); + + it("reports a stale skill with the bare invocation", async () => { + project(); + checkStaleness.mockResolvedValue(STALE_SKILLS); + const measured = await measure(registry("1.0.0")); + const steps = environmentSteps(measured, SPELLING); + expect(steps).toEqual([ + expect.objectContaining({ + part: "skill", + command: "npx @taskless/cli@1.0.0", + }), + ]); + }); + + it("reports rules owing a reconcile with update", async () => { + project(); + reconcileStatus.mockResolvedValue(OWES_RECONCILE); + const steps = environmentSteps(await measure(registry("1.0.0")), SPELLING); + expect(steps).toEqual([ + expect.objectContaining({ + part: "rules", + command: "npx @taskless/cli@1.0.0 update", + }), + ]); + }); + + it("orders all three, and the newer CLI absorbs the skill command", async () => { + project(); + checkStaleness.mockResolvedValue(STALE_SKILLS); + reconcileStatus.mockResolvedValue(OWES_RECONCILE); + const steps = environmentSteps(await measure(registry("2.0.0")), SPELLING); + expect(steps.map((step) => step.part)).toEqual(["cli", "skill", "rules"]); + expect(steps[1]?.command).toBeUndefined(); + const notice = environmentNotice(steps); + expect(notice).toContain("(1) "); + expect(notice).toContain("Run: npx @taskless/cli@2.0.0"); + expect(notice).not.toContain("\n"); + }); + + it("leaves out skill and rules outside a .taskless/ project", async () => { + checkStaleness.mockResolvedValue(STALE_SKILLS); + reconcileStatus.mockResolvedValue(OWES_RECONCILE); + const measured = await measure(registry("1.0.0")); + expect(measured.skill).toBeUndefined(); + expect(measured.rules).toBeUndefined(); + expect(checkStaleness).not.toHaveBeenCalled(); + }); + + it("never asks the registry in CI", async () => { + const fetcher = registry("9.9.9"); + const measured = await measure(fetcher, "true"); + expect(fetcher).not.toHaveBeenCalled(); + expect(measured.cli).toBeUndefined(); + }); + + it("never asks the registry when NO_UPDATE_NOTIFIER is set", async () => { + const fetcher = registry("9.9.9"); + const measured = await measureEnvironment({ + cwd: directory, + fetch: fetcher, + noUpdateNotifier: "1", + running: "1.0.0", + packageName: "@taskless/cli", + }); + expect(fetcher).not.toHaveBeenCalled(); + expect(measured.cli).toBeUndefined(); + }); + + it("never asks the registry for a build that is not a package", async () => { + const fetcher = registry("9.9.9"); + const measured = await measureEnvironment({ + cwd: directory, + fetch: fetcher, + running: "1.0.0", + packageName: null, + }); + expect(fetcher).not.toHaveBeenCalled(); + expect(measured.cli).toBeUndefined(); + }); + + it("has no notice when nothing is out of date", () => { + expect(environmentNotice([])).toBeUndefined(); + }); +}); diff --git a/packages/cli/test/installed-documentation.test.ts b/packages/cli/test/installed-documentation.test.ts index aac1f0aa..6de8d684 100644 --- a/packages/cli/test/installed-documentation.test.ts +++ b/packages/cli/test/installed-documentation.test.ts @@ -190,6 +190,17 @@ describe("this repository's own installed Taskless docs", () => { const prose = skill.replaceAll(/\s+/g, " "); expect(prose).toContain("%(TASKLESS_CLI)s agent ok"); expect(prose).toContain("before any other Taskless command"); + // The preflight's environment report: both outcomes, the steps handed to + // the user rather than run, and the one confirming rerun. + expect(prose).toContain("the environment is up to date"); + expect(prose).toContain("the environment is out of date"); + expect(prose).toContain("Do not run them yourself"); + expect(prose).toContain("with the `!` prefix"); + expect(prose).toContain("another terminal"); + expect(prose).toContain("Then carry on with the user's request"); + expect(prose).toContain( + "When the user says they ran the steps, run `%(TASKLESS_CLI)s agent ok` again to confirm" + ); expect(prose).toContain( "allow rule for exactly the invocation `%(TASKLESS_CLI)s`" ); diff --git a/packages/cli/test/latest-version.test.ts b/packages/cli/test/latest-version.test.ts new file mode 100644 index 00000000..da3a9f7e --- /dev/null +++ b/packages/cli/test/latest-version.test.ts @@ -0,0 +1,98 @@ +import { describe, expect, it, vi } from "vitest"; + +import { + fetchLatestVersion, + isStrictSemver, +} from "../src/install/latest-version"; + +function respond(body: unknown, status = 200): typeof fetch { + return vi.fn(() => + Promise.resolve( + Response.json(body, { + status, + headers: { "content-type": "application/json" }, + }) + ) + ) as unknown as typeof fetch; +} + +describe("fetchLatestVersion", () => { + it("returns latest from the registry's dist-tags", async () => { + const fetcher = respond({ latest: "0.14.0", next: "0.15.0-1" }); + await expect( + fetchLatestVersion("@taskless/cli", { fetch: fetcher }) + ).resolves.toBe("0.14.0"); + expect(fetcher).toHaveBeenCalledWith( + "https://registry.npmjs.org/-/package/@taskless/cli/dist-tags", + expect.any(Object) + ); + }); + + it("accepts a nightly prerelease version", async () => { + await expect( + fetchLatestVersion("@taskless/cli-nightly", { + fetch: respond({ latest: "0.13.5-20261102164414x312732c" }), + }) + ).resolves.toBe("0.13.5-20261102164414x312732c"); + }); + + it("is unknown on a non-2xx response", async () => { + await expect( + fetchLatestVersion("@taskless/cli", { + fetch: respond({ latest: "0.14.0" }, 503), + }) + ).resolves.toBeUndefined(); + }); + + it("is unknown when fetch throws", async () => { + const fetcher = vi.fn(() => + Promise.reject(new Error("offline")) + ) as unknown as typeof fetch; + await expect( + fetchLatestVersion("@taskless/cli", { fetch: fetcher }) + ).resolves.toBeUndefined(); + }); + + it("is unknown when the timeout passes", async () => { + const fetcher = vi.fn( + (_url: string, init?: RequestInit) => + new Promise((_resolve, reject) => { + init?.signal?.addEventListener("abort", () => { + reject(new Error("aborted")); + }); + }) + ) as unknown as typeof fetch; + await expect( + fetchLatestVersion("@taskless/cli", { fetch: fetcher, timeoutMs: 10 }) + ).resolves.toBeUndefined(); + }); + + it("is unknown when the payload has no latest", async () => { + await expect( + fetchLatestVersion("@taskless/cli", { fetch: respond({ next: "1.0.0" }) }) + ).resolves.toBeUndefined(); + }); + + it.each([ + "latest", + "1.0", + "1.0.0 && echo x", + "1.0.0\n", + "v1.0.0", + "1.0.0; rm -rf /", + "01.0.0", + ])("refuses %j as a version", async (latest) => { + await expect( + fetchLatestVersion("@taskless/cli", { fetch: respond({ latest }) }) + ).resolves.toBeUndefined(); + }); +}); + +describe("isStrictSemver", () => { + it.each(["0.0.0", "1.2.3", "1.2.3-alpha.1", "1.2.3+build.5", "1.2.3-x+y"])( + "accepts %s", + (value) => { + expect(isStrictSemver(value)).toBe(true); + } + ); +}); diff --git a/packages/cli/test/prompts.test.ts b/packages/cli/test/prompts.test.ts index 39712991..e27b9652 100644 --- a/packages/cli/test/prompts.test.ts +++ b/packages/cli/test/prompts.test.ts @@ -214,6 +214,7 @@ describe("the CLI invocation variable", () => { "AST_GREP_VERSION", "CLI_VERSION", "DETECT_EVIDENCE", + "ENVIRONMENT_STATUS", "FIXTURE_LAYOUT", "FIXTURE_SOURCES", "HOST_TOOLS", @@ -917,3 +918,51 @@ describe("fixture sources", () => { expect(rendered).not.toContain("otherwise run"); }); }); + +describe("environment status", () => { + const invocation = "npx @taskless/cli@1.0.0"; + + it("says nothing was checked when no environment is supplied", () => { + const text = getRecipe("ok", { invocation }) ?? ""; + expect(text).toContain("was not checked"); + expect(text).not.toContain("Your Taskless environment is up to date"); + expect(text).not.toContain("Your Taskless environment is out of date"); + }); + + it("says up to date when there are no steps", () => { + const text = getRecipe("ok", { invocation, environment: { steps: [] } }); + expect(text).toContain( + "Your Taskless environment is up to date and you can proceed." + ); + }); + + it("numbers each step and fences its command", () => { + const text = + getRecipe("ok", { + invocation, + environment: { + steps: [ + { + part: "cli", + text: "Newer CLI.", + command: "npx @taskless/cli@2.0.0", + }, + { part: "skill", text: "Refreshed by the step above." }, + { + part: "rules", + text: "Rules owe a reconcile.", + command: `${invocation} update`, + }, + ], + }, + }) ?? ""; + expect(text).toContain( + "Your Taskless environment is out of date, the following steps are needed in order for commands to work reliably:" + ); + expect(text).toContain( + "1. Newer CLI.\n ```\n npx @taskless/cli@2.0.0\n ```" + ); + expect(text).toContain("2. Refreshed by the step above.\n\n3."); + expect(text).toContain(`run \`${invocation} agent ok\` again`); + }); +}); diff --git a/packages/cli/vite.config.ts b/packages/cli/vite.config.ts index ee4e3d8e..6ea0c8f4 100644 --- a/packages/cli/vite.config.ts +++ b/packages/cli/vite.config.ts @@ -563,6 +563,10 @@ export default defineConfig({ TASKLESS_TELEMETRY_DISABLED: "1", DO_NOT_TRACK: "1", CI: "", + // No test reaches the npm registry. `agent ok` and local `check` ask + // it for the latest version; tests of that lookup clear this and stub + // `fetch` instead. + NO_UPDATE_NOTIFIER: "1", }, // Two projects, because `__TASKLESS_CLI__` is a compile-time define rather // than a value a test can stub: whatever this config resolves is what every diff --git a/skills/taskless/SKILL.md b/skills/taskless/SKILL.md index bc594935..9223a617 100644 --- a/skills/taskless/SKILL.md +++ b/skills/taskless/SKILL.md @@ -36,11 +36,23 @@ each CLI version. ## Preflight: confirm you can run Taskless Once per session, before any other Taskless command, run -`%(TASKLESS_CLI)s agent ok`. It touches no project files, and prints one line -confirming the invocation works. Once you see it, go straight to the user's -request: no survey of the install, and no menu of what Taskless can do. -`init` writes an allow rule for this exact invocation into the permission -settings of each harness it installs for, so the preflight usually passes. +`%(TASKLESS_CLI)s agent ok`. It writes no project files. Receiving its recipe +confirms the invocation works, and the recipe says whether the Taskless +environment is up to date. `init` writes an allow rule for this exact +invocation into the permission settings of each harness it installs for, so +the preflight usually passes. + +If it says the environment is up to date, go straight to the user's request: +no survey of the install, and no menu of what Taskless can do. + +If it says the environment is out of date, it lists numbered steps with their +commands. Do not run them yourself, since a harness that blocks Taskless +commands usually blocks these too. Give them to the user once, to run +themselves: in Claude Code by typing each command with the `!` prefix, which +runs it in this session's shell, or in another terminal. Then carry on with +the user's request. When the user says they ran the steps, run +`%(TASKLESS_CLI)s agent ok` again to confirm. Apart from that confirmation, +do not run the preflight again in the session once it has passed. Run every Taskless command on a line of its own that begins with `%(TASKLESS_CLI)s`, with nothing around it: no variable assignment, `cd`,