Skip to content

Release PRs with an agent-drafted, editable changelog - #200

Closed
calvinleng-science wants to merge 1 commit into
mainfrom
calvin/release-notes-publish
Closed

calvinleng-science wants to merge 1 commit into
mainfrom
calvin/release-notes-publish

Conversation

@calvinleng-science

@calvinleng-science calvinleng-science commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

What

New release flow

  1. release-open (Actions → Run workflow; choose patch / minor / major, or an exact version). It bumps version= in setup.py on a release/v<version> branch and opens a PR into main labelled release. The changelog sits between <!-- changelog:start --> / <!-- changelog:end --> in the PR description.
    • Stops if a release PR is already open, if nothing changed since the last v* tag, or if setup.py no longer matches the last tag. Versions are bumped only by this workflow from now on.
  2. Edit the changelog in the PR description, then merge.
  3. release-tag (on merge of a release PR) checks the changelog isn't empty and tags the merge commit v<version>. It then starts release.yml on that tag with workflow_dispatch, because a tag pushed with GITHUB_TOKEN doesn't trigger workflows.
  4. release.yml runs as before: test PyPI → validate → PyPI. It also accepts workflow_dispatch; sdist only runs when the ref is a v* tag. A new github-release job then creates the GitHub Release from the release PR's changelog. A hand-pushed tag has no release PR, so it falls back to GitHub's generated notes since the previous tag.

Pushing a v* tag by hand still works.

Changelog drafted by Claude Code

.github/actions/draft-changelog runs Claude Code headless on a self-hosted runner labelled changelog-agent. It reads the commits since the last tag, looking at the diffs where a message isn't clear, and writes user-facing notes under New / Fixed / Changed.

  • Read-only tools only: git log/show/diff, Read, Grep and Glob. It loads no settings, hooks or MCP servers from the box, and stops at $5 or 9 minutes.
  • The job that opens the PR waits at most 10 minutes for the draft. Otherwise it uses a plain list of commit subjects and cancels the queued draft job, so that run shows as cancelled.
  • This repo is public, so the runner label is used only by release-open, which someone with write access starts by hand. The self-hosted job has read-only repo access; pushing and opening the PR happen on GitHub-hosted runners.
  • Setup and security notes: .github/actions/draft-changelog/RUNNER.md.

Example, run locally for v2.8.1..v2.8.4:

Changed

  • Breaking: synapsectl now exits with a non-zero status when a command fails: 1 on error and 130 when cancelled with Ctrl+C. Before, most failures exited with 0, so scripts or CI jobs that ran failing commands without stopping may now stop at those commands. Error messages now go to stderr instead of stdout. (Improve CLI error reporting #194)

Publishing these notes to the docs site (downloads.science.xyz) is left for a follow-up.

Before merging

  • Settings → Actions → General → "Allow GitHub Actions to create and approve pull requests" is on, or release-open can't open the PR.
  • Settings → Actions → General → fork PR workflows: "Require approval for all outside collaborators".

Runner (not blocking the merge)

A runner directory is staged on carbon22 (~/synapse-python-changelog-actions-runner) with Claude Code installed. It still needs:

  1. Registering by a repo admin (config.sh --labels changelog-agent).
  2. sudo ./svc.sh install <runner user>.
  3. A Claude login token in .env.

Until then, release PRs get the plain commit list after 10 minutes. That machine also runs other runners, so moving this one to its own user is recommended (see RUNNER.md).

Notes

  • GitHub doesn't run workflows for PRs that a workflow opens with GITHUB_TOKEN, so main / cla-check won't run on release PRs. They only change setup.py, and main has no required checks.
  • Every release now creates a public GitHub Release.

Testing

  • actionlint: nothing new, apart from the custom changelog-agent label.
  • Checked against this repo: the version bump and setup.py edit (v2.8.4 → 2.8.5 / 2.9.0 / 3.0.0), changelog extraction, the release-PR lookup and the jobs-API query on real runs.
  • The changelog agent was run locally (14 s), and on carbon22 without a login to confirm the fallback.
  • No workflow here has run end to end yet.

@calvinleng-science
calvinleng-science force-pushed the calvin/release-notes-publish branch from 193dd48 to e33280b Compare October 27, 2026 20:15
release-open (manual) bumps setup.py on a release/v<version> branch and opens
a PR into main labelled `release`. Its description holds a changelog drafted
by Claude Code on the self-hosted `changelog-agent` runner with read-only
repo tools (.github/actions/draft-changelog, setup in RUNNER.md). If the agent
hasn't finished within 10 minutes, the PR gets the commit subjects instead.

Merging it runs release-tag, which tags the merge commit and dispatches
release.yml. After PyPI, release.yml creates the GitHub Release from the PR's
(edited) changelog; hand-pushed tags get GitHub's generated notes.
@calvinleng-science calvinleng-science changed the title Release PRs with an agent-drafted changelog, and release notes on the docs site Release PRs with an agent-drafted, editable changelog Oct 27, 2026
@calvinleng-science
calvinleng-science force-pushed the calvin/release-notes-publish branch from bbcc648 to 6ac7546 Compare October 27, 2026 22:41
@calvinleng-science

Copy link
Copy Markdown
Contributor Author

Closing for now: the self-hosted changelog-agent runner can't be set up for this repo yet. The work is kept on local branches for later.

@calvinleng-science
calvinleng-science deleted the calvin/release-notes-publish branch October 27, 2026 22:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant