Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 91 additions & 0 deletions .claude/skills/playwright-triage/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
---
name: playwright-triage
description: Triage and fix one microsoft/playwright-python issue, or do a maintainer's task, unattended inside the Issue triage GitHub Actions workflow (.github/workflows/triage.yml), which a needs-triage, needs-triage-mac or needs-triage-win label starts, or a maintainer starts by hand with an issue, a task or both. Reproduces the issue on the runner, then leaves a verified fix proposal for the workflow to open as a draft PR, an investigation note for it to post on the issue, or an upstream bug report for a maintainer to file in microsoft/playwright. Only for that workflow.
---

# Playwright Python Issue Triage

You run unattended in the [Issue triage](../../../.github/workflows/triage.yml) workflow of
`microsoft/playwright-python`. First read the shared guide, `$AGENT_GUIDES_DIR/triage.md`, a copy of
[microsoft/playwright/.github/actions/run-agent/guides/triage.md](https://github.com/microsoft/playwright/blob/main/.github/actions/run-agent/guides/triage.md),
and then [AGENTS.md](../../../AGENTS.md), both in full. The guide says how to triage an issue in any
Playwright repo. AGENTS.md says how to build, test and lint this one, and where a regression test
goes. This file covers what neither has: the checkouts, how to run a published version, and the
outcome that this repo adds, an upstream bug report.

## The checkouts

`$REPO_DIR` is the checkout of `microsoft/playwright-python` that the guide describes. Its requirements
are installed (`pip install -r local-requirements.txt`, `pip install -r requirements.txt`,
`pip install -e .`), the wheel is built (`python -m build --wheel`, which builds the driver into
`playwright/driver/`), and Chromium, Firefox and WebKit are installed with
`python -m playwright install --with-deps`. It is the working directory, and the checkout that the
guides refer to. The clone is shallow: run `git fetch --unshallow --tags origin` in it before you
bisect or use `git tag --contains`.

`$PLAYWRIGHT_DIR` is `$REPO_DIR/.upstream/`, a full clone of `microsoft/playwright` at the commit
in `DRIVER_VERSION`, created by `python scripts/upstream.py checkout`. The driver was built from it.
Read upstream code there. Do not edit, commit or switch branches in it: the wheel build and
`./scripts/lint.sh` stop when it is not at that commit or has local changes. Before you run upstream
tests in it, it needs `npm ci`, `npm run build` and `npx playwright install`. The wheel build
(`scripts/build_driver.py`) ran the first two there; run them again when `node_modules/` is missing.

## Published versions

To run a published version, for example to bisect, create `$SCRATCH_DIR/playwright-python-<n>/<version>/`
and run there:

```bash
python -m venv env && source env/bin/activate # env\Scripts\activate on Windows
pip install playwright==<version>
python -m playwright install
```

The fixtures of the repo are not available there. Write the condensed repro as a plain script with
`sync_playwright()` or `async_playwright()` in that directory, with `page.set_content()` in place of
the test server, and run it from there. In `$REPO_DIR`, the source tree shadows the installed package.

The driver of a release is in `DRIVER_VERSION` at its tag: `git show v<version>:DRIVER_VERSION`
prints the upstream commit it was built from, or, for releases up to 1.64, the `playwright-core`
version of the driver; `npm view playwright-core@<core version> gitHead` prints its commit. Match a
regression in the driver to commits with `git -C $PLAYWRIGHT_DIR log <good>..<bad> -- <path>`.

## Upstream bugs

The client talks to the Node driver that is built from `$PLAYWRIGHT_DIR`, and `microsoft/playwright`
and this repo are one team. Reproduce the bug in Python first. Then write the same calls as an
upstream test and run it in `$PLAYWRIGHT_DIR`. The triage skill of `microsoft/playwright`, in
`$PLAYWRIGHT_DIR/.claude/skills/playwright-triage/`, says where such a test goes, which fixtures it
uses and how to run it. The rest of that skill is not for this run. Put the test in a new spec
file, and delete the file when you are done, so that the checkout is clean.

- The Node test passes: the bug is in the Python client. Fix it here.
- The Node test fails too: the root cause is upstream, and the fix lands in `microsoft/playwright`.
Do not work around it in `playwright/_impl/`. Run `git -C $PLAYWRIGHT_DIR fetch origin main`.
When `git -C $PLAYWRIGHT_DIR log HEAD..origin/main -- <path>` shows a fix that already landed,
the note says that it reaches this repo with the next roll. Otherwise write the report.
- The Python API lacks a member or an option: the generated API has everything that the upstream
docs in `$PLAYWRIGHT_DIR/docs/src/api` give Python (`* langs:` decides; no `langs` means every
language). The gap is in the docs, so it is an upstream change too. Do not edit a generated file,
and do not add to `scripts/codegen/python_rules.js` to go around the docs.

Create `$OUTPUT_DIR/upstream/` with:

- `report.md`: the report for an engineer of `microsoft/playwright` who did not read the Python
issue: what fails, expected versus actual, the versions (the Python package, the commit in
`DRIVER_VERSION`, the browser, the platform), the root cause when you found it, and the condensed
Node test as the repro.
- `report.json`: `{"title": "..."}`, at most 256 characters, for example
`route.fulfill() drops the charset of Content-Type`.

The comment job appends the title and the body to your comment, for a maintainer to file in
`microsoft/playwright`. The bot never files it. Write `report.json` last, when `report.md` and the
comment are done. Like `proposal.json`, it marks the report as ready.

## PRs from third parties

Do not run a PR that changes `setup.py`, `pyproject.toml`, `setup.cfg`, `local-requirements.txt`,
`requirements.txt`, `DRIVER_VERSION`, `NODE_VERSION`, `.pre-commit-config.yaml`, `test.sh`,
`scripts/`, `utils/`, `.github/`, `.azure-pipelines/`, `.claude/`, or a generated file. A PR that
changes a ported test under `tests/{async,sync}/{page,library}`, or adds a mark to one, has faults:
the ports have zero deviations.
36 changes: 36 additions & 0 deletions .github/actions/setup-triage-runner/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: 'Set up the triage runner'
description: >
Prepares the environment that the triage and revision agents work in: the checkout built as CI
builds it, with the browsers and the Copilot CLI installed, and the upstream checkout in .upstream/
that the driver is built from. Exports PLAYWRIGHT_DIR, the path of that checkout, and CI_ENV_EXEC,
the wrapper from microsoft/playwright that runs a command under xvfb on Linux and at normal I/O
priority on Windows.
runs:
using: composite
steps:
- uses: microsoft/playwright/.github/actions/ci-env@main
- uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: '3.12'
- uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0
with:
node-version: '24'
- name: Install the requirements
shell: bash
run: |
python -m pip install --upgrade pip
pip install -r local-requirements.txt
pip install -r requirements.txt
pip install -e .
- name: Check out upstream and build the wheel
shell: bash
run: |
python scripts/upstream.py checkout
python -m build --wheel
echo "PLAYWRIGHT_DIR=${GITHUB_WORKSPACE//\\//}/.upstream" >> "$GITHUB_ENV"
- name: Install the browsers
shell: bash
run: python -m playwright install --with-deps chromium firefox webkit
- name: Install the Copilot CLI
shell: bash
run: npm install -g @github/copilot
132 changes: 132 additions & 0 deletions .github/workflows/revision.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
# Copilot addresses the maintainers' review comments on a draft PR that the triage workflow opened,
# without write credentials. A fresh job pushes the new commit to the PR's branch using the app, so
# CI runs on it. The steps that are the same for every Playwright repo are actions in
# microsoft/playwright, and the shared guide they carry says how to revise; the prompt below has
# what is specific to this repo.
name: PR revision

on:
# Apply needs-revision, needs-revision-mac or needs-revision-win; re-apply it to trigger another pass.
# pull_request_target always runs this file from main; the jobs only accept the triage app's own PRs.
pull_request_target:
types: [labeled]

permissions: {}

jobs:
prepare:
name: Collect the maintainers' comments
if: >-
github.repository == 'microsoft/playwright-python' &&
contains(fromJSON('["needs-revision","needs-revision-mac","needs-revision-win"]'), github.event.label.name) &&
github.event.pull_request.state == 'open' &&
github.event.pull_request.user.login == 'microsoft-playwright-automation[bot]' &&
github.event.pull_request.head.repo.full_name == github.repository
runs-on: ubuntu-latest
permissions:
pull-requests: write
env:
PLATFORM: ${{ fromJSON('{"needs-revision":"linux","needs-revision-mac":"mac","needs-revision-win":"windows"}')[github.event.label.name] }}
outputs:
head: ${{ steps.collect.outputs.head }}
base: ${{ steps.collect.outputs.base }}
branch: ${{ steps.collect.outputs.branch }}
platform: ${{ env.PLATFORM }}
steps:
- name: Collect the PR and the maintainers' comments
id: collect
uses: microsoft/playwright/.github/actions/collect-review@main
with:
pull-request: ${{ github.event.pull_request.number }}
- name: Post the run link
env:
GH_TOKEN: ${{ github.token }}
PR: ${{ github.event.pull_request.number }}
PLATFORM_NAME: ${{ fromJSON('{"linux":"Linux","mac":"macOS","windows":"Windows"}')[env.PLATFORM] }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
run: |
body=$(printf "Hi, I'm the Playwright bot. I started to address the review comments on %s: [agent run](%s)\n<!-- playwright-revision-started -->" "$PLATFORM_NAME" "$RUN_URL")
gh pr comment "$PR" --body "$body" || echo "::warning::Could not post the run link"

revise:
name: Revise PR ${{ github.event.pull_request.number }} (${{ needs.prepare.outputs.platform }})
needs: prepare
runs-on: ${{ fromJSON('{"linux":"playwright-x64-ubuntu24-64-core","mac":"macos-15-xlarge","windows":"playwright-x64-server2025-64-core"}')[needs.prepare.outputs.platform] }}
timeout-minutes: 240
concurrency:
group: revision-${{ github.event.pull_request.number }}
cancel-in-progress: false
defaults:
run:
shell: bash
env:
PR: ${{ github.event.pull_request.number }}
HEAD_SHA: ${{ needs.prepare.outputs.head }}
BASE_SHA: ${{ needs.prepare.outputs.base }}
TRIAGE_PLATFORM: ${{ needs.prepare.outputs.platform }}
permissions:
copilot-requests: write
outputs:
has_revision: ${{ steps.agent.outputs.has-result }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ needs.prepare.outputs.head }}
persist-credentials: false
- name: Fetch the base and record the directories
run: |
git fetch --no-tags --depth=1 origin "$BASE_SHA"
echo "OUTPUT_DIR=${RUNNER_TEMP//\\//}/revision" >> "$GITHUB_ENV"
echo "REVIEW_DIR=${RUNNER_TEMP//\\//}/review" >> "$GITHUB_ENV"
echo "SCRATCH_DIR=${RUNNER_TEMP//\\//}/scratch" >> "$GITHUB_ENV"
echo "REPO_DIR=${GITHUB_WORKSPACE//\\//}" >> "$GITHUB_ENV"
mkdir -p "${RUNNER_TEMP//\\//}/revision" "${RUNNER_TEMP//\\//}/scratch"
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
name: review
path: ${{ runner.temp }}/review
- name: Set up the runner
uses: ./.github/actions/setup-triage-runner
- name: Revise with Copilot CLI
id: agent
# No GitHub tools and no URL access: the agent reads the PR only from the review the prepare job filtered.
uses: microsoft/playwright/.github/actions/run-agent@main
with:
prompt: |
Follow the shared guide revision.md and this repo's AGENTS.md. Read both in full before starting. $PLAYWRIGHT_DIR is the .upstream/ checkout of microsoft/playwright at the commit in DRIVER_VERSION; do not edit, commit or switch branches in it.

Pull request: https://github.com/microsoft/playwright-python/pull/${{ github.event.pull_request.number }}
output-dir: ${{ env.OUTPUT_DIR }}
result-file: commit/commit.json
artifact-name: revision
exec: ${{ env.CI_ENV_EXEC }}

push:
name: Push the revision
needs: [prepare, revise]
# commit.json is written last, so a finished revision is pushed even when the job failed later.
if: ${{ !cancelled() && needs.revise.outputs.has_revision == 'true' }}
runs-on: ubuntu-latest
steps:
- uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0
id: app-token
with:
app-id: ${{ vars.PLAYWRIGHT_APP_ID }}
private-key: ${{ secrets.PLAYWRIGHT_PRIVATE_KEY }}
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ needs.prepare.outputs.head }}
token: ${{ steps.app-token.outputs.token }}
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
name: revision
path: ${{ runner.temp }}/revision
# Never execute code or scripts from the proposed changes in the app job.
- name: Validate and push
uses: microsoft/playwright/.github/actions/publish-patch@main
with:
token: ${{ steps.app-token.outputs.token }}
dir: ${{ runner.temp }}/revision/commit
patch: revision.patch
metadata: commit.json
branch: ${{ needs.prepare.outputs.branch }}
Loading
Loading