Skip to content

About

No description, website, or topics provided.

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

aviator-cli

aviator is Aviator's CLI for submitting verifications and creating runbooks over the Aviator REST API.

Install

With Homebrew:

brew trust aviator-co/tap  # Homebrew 6+ only loads trusted third-party taps
brew install aviator-co/tap/aviator

With Go:

go install github.com/aviator-co/aviator-cli/cmd/aviator@latest

Binaries for Linux, macOS, and Windows are also attached to every release.

Authentication

Sign in through your browser:

aviator login

This runs an OAuth authorization-code flow with PKCE, briefly listening on a loopback port for the browser to be redirected back to, and stores the session in your OS keychain, refreshing it automatically. aviator logout removes the stored session; Aviator has no token revocation endpoint, so a token that was already issued stays valid until it expires.

For CI and other headless environments, set a static API token instead:

export AVIATOR_API_TOKEN=<your-api-token>

Credentials are used in this order: AVIATOR_API_TOKEN, then aviator.apiToken from the config file, then the keychain session from aviator login.

Configuration

The CLI reads configuration from (first match wins):

  • $XDG_CONFIG_HOME/aviator/config.yaml
  • $HOME/.config/aviator/config.yaml
  • $HOME/.aviator/config.yaml
  • a repo-local <git-common-dir>/aviator/config.yaml (merged on top)
aviator:
  apiHost: https://api.aviator.co   # override for on-prem
  apiToken: <your-api-token>        # optional; prefer `aviator login`

Environment variables override the config file:

  • AVIATOR_API_TOKEN
  • AVIATOR_API_HOST
  • AVIATOR_NO_UPDATE_NOTIFIER: set to anything to stop the new-version notice

Usage

Two commands, two different jobs:

  • aviator verify: you wrote the code, Aviator verifies the PR against your acceptance criteria.
  • aviator runbook: Aviator's agent writes the code from your spec and opens its own PR.

Set up your agents

Add hooks that have your coding agents capture a change's intent and acceptance criteria for Verify as you open a PR, and read a PR's decisions when you review it:

aviator init                 # in this repo, committed for your whole team
aviator init --scope self    # just you, in every repo on this machine
aviator init --scope local   # just you, in this repo only

--scope local writes .claude/settings.local.json and covers Claude Code only. Re-run init any time to update; aviator hooks uninstall removes the hooks.

Submit for verification

aviator verify \
  --repo acme/web \
  --intent "Ensure the feature flag gates the new banner" \
  --criteria "Banner hidden when flag off" \
  --criteria "Banner shown when flag on" \
  --working-branch feature/banner \
  --target-branch main \
  --spec ./spec.md

--criteria is repeatable; alternatively pass --criteria-file (one criterion per line, # comments ignored). --working-branch, --target-branch, and --spec are optional, though without --working-branch the session can only bind to a PR through a Review: <url> line in the PR body.

--preview <name> picks one of the repo's preview configs for the session's runs instead of the repo default. It works on aviator verify r/123 and aviator edit r/123 too, and sticks: later runs on the session, automatic ones included, keep using it until it's changed again.

--scenario-guidance "<note>" tells Verify how to check the change (say, which seeded user to log in as). It's read when the session's first scenarios are written, normally on its first run, and can't be changed or read back after.

One review tracks exactly one PR. Stacked or multi-PR work needs one submission per PR, each with its own --working-branch, intent, and criteria. To update the intent or criteria on a session that already exists, use aviator edit rather than submitting the branch again. aviator sessions says whether a branch already has one.

Pass --json to print the submission as a single JSON object (id, url, working_branch, target_branch, criteria_count) instead of the human summary.

Create a runbook

Aviator's agent implements the spec and opens its own PR. For code you wrote yourself, use aviator verify instead.

aviator runbook \
  --repo acme/web \
  --intent "Migrate the settings page to the new design system" \
  --spec ./spec.md \
  --criteria "Settings page renders with the new components" \
  --target-branch main

--intent is required; --title, --spec, --criteria/--criteria-file, --target-branch, and --author-email are optional. --oneshot is on by default. --json prints id, url, status, and criteria_count as a single JSON object.

Find a session

aviator sessions --repo acme/web                            # active sessions, newest first
aviator sessions --repo acme/web --branch feature/banner    # sessions on one branch
aviator sessions --repo acme/web --pr 1201                  # sessions linked to one PR, whoever created them
aviator sessions --repo acme/web --all                      # everyone's sessions, not just yours
ID    BRANCH          PRS
r/42  feature/banner  #1201

Run it before aviator verify to see whether a branch already has a session. Submitting the same branch again creates a second session instead of updating the first, and a PR opened from that branch then links to neither. For what a session contains, use aviator show r/42.

--status (default active, or any status with --pr) chooses what to list, --limit (default 20, max 100) sets the page size, and --page steps through the pages. --json prints {"sessions": [...], "has_more": bool}, with id, url, working_branch, pull_requests, and the version that aviator edit --expected-version takes.

Inspect a session

aviator show r/123            # session summary, with its verification runs
aviator results r/123         # latest verification results
aviator results r/123 --run 887     # the results of an earlier run
aviator scenarios r/123       # what the latest run exercised, with evidence ids
aviator scenarios r/123 --run 887   # the same for an earlier run
aviator evidence 4567 -o trace.json   # download one evidence file, e.g. a run trace
aviator edit r/123 --expected-version 4 --criteria "..."
aviator edit r/123 --intent "..."
aviator edit r/123 --preview docs                  # switch preview config, no run
aviator dismiss r/123 --key <key>                  # delete a task criterion
aviator dismiss r/123 --criteria-json \
  '[{"stable_key": "<key>", "action": "waive", "justification": "..."}]'
aviator dismiss r/123 --criteria-json \
  '[{"baseline_invariant_id": 42, "category": "accepted_risk", "justification": "..."}]'

Record decisions

A decision is a question, possible bug, risk, or choice raised about a session's PR, so reviewers can see why the code looks the way it does. Each one has a comment thread and is either open or settled with an outcome.

aviator decisions list r/123                  # a session's decisions
aviator decisions list r/123 --status open    # only the open ones
aviator decisions show r/123 7                # one decision with its comments
aviator decisions apply r/123 --file batch.json   # record a batch, - reads stdin

apply creates, changes, settles, reopens, and comments on decisions in one request, so either all of it is recorded or none of it is. aviator decisions apply --help has the batch format. It needs a user access token (aviator login or a personal token). Anyone can settle, reopen, and comment, but only the user who raised a decision can change its type, title, summary, or anchor. There's no way to delete one. --json prints the result as a single JSON object.

Submit evidence from your own run

When Aviator can't reach a preview of the change, run it yourself and submit what you saw. Aviator judges that evidence instead of collecting its own.

aviator verify --repo owner/repo --intent "..." --criteria "..." --external-evidence
aviator evidence submit r/123 evidence.json
aviator results r/123         # says when evidence is submitted and no run has started

--external-evidence creates the session as a draft that runs nothing until evidence arrives. The manifest is JSON:

{
  "commit_sha": "4f2c9e1a7b3d5c6e8f9a0b1c2d3e4f5a6b7c8d9e",
  "scenarios": [
    {
      "summary": "Toggle the flag and reload the home page",
      "steps": [
        {"text": "Open the home page with the flag off"},
        {
          "text": "Capture the page without the banner",
          "evidence": [
            {"type": "screenshot", "file": "shots/home.png", "label": "Home page, no banner"}
          ]
        }
      ]
    }
  ]
}

File paths are relative to the manifest, and commit_sha is the full SHA the evidence was captured against (--commit overrides it). A run starts once every file is uploaded and the evidence is for the PR's head commit. Submitting again replaces the earlier submission and skips files that are already uploaded. --json prints one files entry per declared file with its path, evidence_id and status, and the command exits non-zero when any file didn't make it.

Manifest reference: https://docs.aviator.co/verify/how-to-guides/submitting-evidence-from-your-own-run

Describe how a change is built

Artifacts are the diagrams and notes a session's review page shows beside the code: each one has a Mermaid diagram, a markdown body, or both, plus the files it's about. A PR author's coding agent usually writes them.

aviator artifacts list r/123                           # the current set, one row per artifact
aviator artifacts show r/123 31                        # one artifact by the id list prints, with its Mermaid source
aviator artifacts submit r/123 --file artifacts.json   # replace the set, - reads stdin

A submit replaces the session's whole set, so send every artifact every time. Earlier sets stay attached to the verification runs that started with them. To change one artifact, edit the artifact_set from list --json and submit it again without its id and created_at or the artifacts' ids, which the server rejects. Put in your own author and the current head_sha, or drop head_sha so file links follow the run's commit. aviator artifacts submit --help has the file format. It needs a user access token (aviator login or a personal token). --json prints the result as a single JSON object.

Manage baseline invariants

Baseline invariants are standing rules every verification checks on top of a session's own criteria. Each one is owned by a GitHub team: its own owner teams, or the account's default owner team when it has none. Listing works with any API token, account-scoped ones included. Everything else acts as the caller, so it needs a user access token (aviator login or a personal token). Any user can create; editing, deleting, approving, and changing owners also need the user to be a maintainer, an admin, or directly on the invariant's owner team.

aviator invariants list                          # newest first, 20 per page (--limit, --page)
aviator invariants list --repo acme/web          # what applies to one repo
aviator invariants list --status pending         # waiting on approval
aviator invariants list --source manual          # only hand-written rules
aviator invariants list --ids 42,43              # specific rules by id
aviator invariants list --owner-team acme/payments   # what one team owns
aviator invariants list --unowned                # no owner team and no default
aviator invariants categories                    # slugs that --category accepts
aviator invariants owner-teams                   # teams by invariants owned, 20 per page (--limit, --page)
aviator invariants insights --since 7d           # top applied, failing and waived rules (default last 30 days)
aviator invariants list --usage --sort waived    # usage columns (--sort applied|failure_rate|waived|waive_rate|shadow_failed)
aviator invariants list --mode shadow            # rules by current mode (off, shadow, enforced)
aviator invariants list --usage --never-applied  # enabled rules no review has applied
aviator invariants show 42 --by-repo             # usage per version and repo, plus its waivers
aviator invariants reviews 42 --outcome fail     # reviews the rule was applied on, with each outcome
aviator invariants reviews 42 --shadow           # only reviews with a shadow verdict (--shadow=false for the rest)
aviator invariants waivers --user me             # waivers across rules (--invariant, --category, --repo)

aviator invariants create \
  --title "Webhook handlers verify signatures" \
  --body "Every new webhook handler must verify the request signature before acting." \
  --category security \
  --repo acme/web --repo acme/api \
  --condition 'file_path_glob=src/webhooks/**' \
  --condition 'language!=markdown'

aviator invariants edit 42 --title "..." --body-file rule.md
aviator invariants edit 42 --account-scoped      # applies to every repo
aviator invariants edit 42 --no-conditions --disable
aviator invariants approve 42 43                 # pending -> active
aviator invariants reject 42                     # keeps the record, stops applying
aviator invariants set-status pending 42         # any status, e.g. restore a rejected one
aviator invariants edit 42 --owner-team acme/payments  # replace the owner teams
aviator invariants edit 42 --default-owner       # fall back to the default owner team
aviator invariants set-owner acme/infra 42 43    # hand several to one team
aviator invariants set-owner default 42 43       # back to the default owner team
aviator invariants delete 42
ID   STATUS   CATEGORY       SOURCE             SCOPE                  OWNER           TITLE
#42  active   security       manual             2 repos                @acme/payments  Webhook handlers verify signatures
#41  pending  test_coverage  ai_generated_docs  all repos +conditions  @acme/infra     Every package has a test file
                                                                                       reason: CONTRIBUTING.md asks for tests on every package.

The scope column is a summary; the full repo list and conditions are in the create/edit output and in --json. AI-drafted rules show the reason they were proposed, which is what approve and reject are deciding on. Omit --repo on create for an account-wide invariant. On edit, --repo and --condition replace the current set, so repeat them for the full new set. --category takes one of the account's category slugs (see aviator invariants categories); an unknown slug is rejected with the valid ones listed. --enable/--disable only apply to active invariants; approve a pending one first. The owner column shows the effective owner, so an invariant without its own owner teams shows the default owner team; --owner-team filters match it the same way. --json on any subcommand prints the result as a single JSON object.

Contributing

See CONTRIBUTING.md for development and release setup.

License

MIT

About

No description, website, or topics provided.

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages