From 10a147baf97ac992e4883543c34900fb1dd5c224 Mon Sep 17 00:00:00 2001 From: Pavel Feldman Date: Wed, 28 Oct 2026 20:38:09 -0700 Subject: [PATCH] devops(triage): add the issue triage and revision bots The needs-triage and needs-revision labels start the workflows. They call the shared actions from microsoft/playwright; setup-triage-runner builds this repo and installs the browsers. The triage skill links to the shared guide and keeps only what is specific to this repo. The agent reads AGENTS.md for build and test. --- .claude/skills/playwright-triage/SKILL.md | 91 +++++++ .../actions/setup-triage-runner/action.yml | 36 +++ .github/workflows/revision.yml | 132 ++++++++++ .github/workflows/triage.yml | 227 ++++++++++++++++++ AGENTS.md | 30 +++ 5 files changed, 516 insertions(+) create mode 100644 .claude/skills/playwright-triage/SKILL.md create mode 100644 .github/actions/setup-triage-runner/action.yml create mode 100644 .github/workflows/revision.yml create mode 100644 .github/workflows/triage.yml diff --git a/.claude/skills/playwright-triage/SKILL.md b/.claude/skills/playwright-triage/SKILL.md new file mode 100644 index 000000000..a81efd618 --- /dev/null +++ b/.claude/skills/playwright-triage/SKILL.md @@ -0,0 +1,91 @@ +--- +name: playwright-triage +description: Triage and fix one microsoft/playwright-python issue, or do a maintainer's task, unattended inside the Issue triage GitHub Actions workflow (.github/workflows/triage.yml), which a needs-triage, needs-triage-mac or needs-triage-win label starts, or a maintainer starts by hand with an issue, a task or both. Reproduces the issue on the runner, then leaves a verified fix proposal for the workflow to open as a draft PR, an investigation note for it to post on the issue, or an upstream bug report for a maintainer to file in microsoft/playwright. Only for that workflow. +--- + +# Playwright Python Issue Triage + +You run unattended in the [Issue triage](../../../.github/workflows/triage.yml) workflow of +`microsoft/playwright-python`. First read the shared guide, `$AGENT_GUIDES_DIR/triage.md`, a copy of +[microsoft/playwright/.github/actions/run-agent/guides/triage.md](https://github.com/microsoft/playwright/blob/main/.github/actions/run-agent/guides/triage.md), +and then [AGENTS.md](../../../AGENTS.md), both in full. The guide says how to triage an issue in any +Playwright repo. AGENTS.md says how to build, test and lint this one, and where a regression test +goes. This file covers what neither has: the checkouts, how to run a published version, and the +outcome that this repo adds, an upstream bug report. + +## The checkouts + +`$REPO_DIR` is the checkout of `microsoft/playwright-python` that the guide describes. Its requirements +are installed (`pip install -r local-requirements.txt`, `pip install -r requirements.txt`, +`pip install -e .`), the wheel is built (`python -m build --wheel`, which builds the driver into +`playwright/driver/`), and Chromium, Firefox and WebKit are installed with +`python -m playwright install --with-deps`. It is the working directory, and the checkout that the +guides refer to. The clone is shallow: run `git fetch --unshallow --tags origin` in it before you +bisect or use `git tag --contains`. + +`$PLAYWRIGHT_DIR` is `$REPO_DIR/.upstream/`, a full clone of `microsoft/playwright` at the commit +in `DRIVER_VERSION`, created by `python scripts/upstream.py checkout`. The driver was built from it. +Read upstream code there. Do not edit, commit or switch branches in it: the wheel build and +`./scripts/lint.sh` stop when it is not at that commit or has local changes. Before you run upstream +tests in it, it needs `npm ci`, `npm run build` and `npx playwright install`. The wheel build +(`scripts/build_driver.py`) ran the first two there; run them again when `node_modules/` is missing. + +## Published versions + +To run a published version, for example to bisect, create `$SCRATCH_DIR/playwright-python-//` +and run there: + +```bash +python -m venv env && source env/bin/activate # env\Scripts\activate on Windows +pip install playwright== +python -m playwright install +``` + +The fixtures of the repo are not available there. Write the condensed repro as a plain script with +`sync_playwright()` or `async_playwright()` in that directory, with `page.set_content()` in place of +the test server, and run it from there. In `$REPO_DIR`, the source tree shadows the installed package. + +The driver of a release is in `DRIVER_VERSION` at its tag: `git show v:DRIVER_VERSION` +prints the upstream commit it was built from, or, for releases up to 1.64, the `playwright-core` +version of the driver; `npm view playwright-core@ gitHead` prints its commit. Match a +regression in the driver to commits with `git -C $PLAYWRIGHT_DIR log .. -- `. + +## Upstream bugs + +The client talks to the Node driver that is built from `$PLAYWRIGHT_DIR`, and `microsoft/playwright` +and this repo are one team. Reproduce the bug in Python first. Then write the same calls as an +upstream test and run it in `$PLAYWRIGHT_DIR`. The triage skill of `microsoft/playwright`, in +`$PLAYWRIGHT_DIR/.claude/skills/playwright-triage/`, says where such a test goes, which fixtures it +uses and how to run it. The rest of that skill is not for this run. Put the test in a new spec +file, and delete the file when you are done, so that the checkout is clean. + +- The Node test passes: the bug is in the Python client. Fix it here. +- The Node test fails too: the root cause is upstream, and the fix lands in `microsoft/playwright`. + Do not work around it in `playwright/_impl/`. Run `git -C $PLAYWRIGHT_DIR fetch origin main`. + When `git -C $PLAYWRIGHT_DIR log HEAD..origin/main -- ` shows a fix that already landed, + the note says that it reaches this repo with the next roll. Otherwise write the report. +- The Python API lacks a member or an option: the generated API has everything that the upstream + docs in `$PLAYWRIGHT_DIR/docs/src/api` give Python (`* langs:` decides; no `langs` means every + language). The gap is in the docs, so it is an upstream change too. Do not edit a generated file, + and do not add to `scripts/codegen/python_rules.js` to go around the docs. + +Create `$OUTPUT_DIR/upstream/` with: + +- `report.md`: the report for an engineer of `microsoft/playwright` who did not read the Python + issue: what fails, expected versus actual, the versions (the Python package, the commit in + `DRIVER_VERSION`, the browser, the platform), the root cause when you found it, and the condensed + Node test as the repro. +- `report.json`: `{"title": "..."}`, at most 256 characters, for example + `route.fulfill() drops the charset of Content-Type`. + +The comment job appends the title and the body to your comment, for a maintainer to file in +`microsoft/playwright`. The bot never files it. Write `report.json` last, when `report.md` and the +comment are done. Like `proposal.json`, it marks the report as ready. + +## PRs from third parties + +Do not run a PR that changes `setup.py`, `pyproject.toml`, `setup.cfg`, `local-requirements.txt`, +`requirements.txt`, `DRIVER_VERSION`, `NODE_VERSION`, `.pre-commit-config.yaml`, `test.sh`, +`scripts/`, `utils/`, `.github/`, `.azure-pipelines/`, `.claude/`, or a generated file. A PR that +changes a ported test under `tests/{async,sync}/{page,library}`, or adds a mark to one, has faults: +the ports have zero deviations. diff --git a/.github/actions/setup-triage-runner/action.yml b/.github/actions/setup-triage-runner/action.yml new file mode 100644 index 000000000..0e1588634 --- /dev/null +++ b/.github/actions/setup-triage-runner/action.yml @@ -0,0 +1,36 @@ +name: 'Set up the triage runner' +description: > + Prepares the environment that the triage and revision agents work in: the checkout built as CI + builds it, with the browsers and the Copilot CLI installed, and the upstream checkout in .upstream/ + that the driver is built from. Exports PLAYWRIGHT_DIR, the path of that checkout, and CI_ENV_EXEC, + the wrapper from microsoft/playwright that runs a command under xvfb on Linux and at normal I/O + priority on Windows. +runs: + using: composite + steps: + - uses: microsoft/playwright/.github/actions/ci-env@main + - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 + with: + python-version: '3.12' + - uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0 + with: + node-version: '24' + - name: Install the requirements + shell: bash + run: | + python -m pip install --upgrade pip + pip install -r local-requirements.txt + pip install -r requirements.txt + pip install -e . + - name: Check out upstream and build the wheel + shell: bash + run: | + python scripts/upstream.py checkout + python -m build --wheel + echo "PLAYWRIGHT_DIR=${GITHUB_WORKSPACE//\\//}/.upstream" >> "$GITHUB_ENV" + - name: Install the browsers + shell: bash + run: python -m playwright install --with-deps chromium firefox webkit + - name: Install the Copilot CLI + shell: bash + run: npm install -g @github/copilot diff --git a/.github/workflows/revision.yml b/.github/workflows/revision.yml new file mode 100644 index 000000000..7b4ee534b --- /dev/null +++ b/.github/workflows/revision.yml @@ -0,0 +1,132 @@ +# Copilot addresses the maintainers' review comments on a draft PR that the triage workflow opened, +# without write credentials. A fresh job pushes the new commit to the PR's branch using the app, so +# CI runs on it. The steps that are the same for every Playwright repo are actions in +# microsoft/playwright, and the shared guide they carry says how to revise; the prompt below has +# what is specific to this repo. +name: PR revision + +on: + # Apply needs-revision, needs-revision-mac or needs-revision-win; re-apply it to trigger another pass. + # pull_request_target always runs this file from main; the jobs only accept the triage app's own PRs. + pull_request_target: + types: [labeled] + +permissions: {} + +jobs: + prepare: + name: Collect the maintainers' comments + if: >- + github.repository == 'microsoft/playwright-python' && + contains(fromJSON('["needs-revision","needs-revision-mac","needs-revision-win"]'), github.event.label.name) && + github.event.pull_request.state == 'open' && + github.event.pull_request.user.login == 'microsoft-playwright-automation[bot]' && + github.event.pull_request.head.repo.full_name == github.repository + runs-on: ubuntu-latest + permissions: + pull-requests: write + env: + PLATFORM: ${{ fromJSON('{"needs-revision":"linux","needs-revision-mac":"mac","needs-revision-win":"windows"}')[github.event.label.name] }} + outputs: + head: ${{ steps.collect.outputs.head }} + base: ${{ steps.collect.outputs.base }} + branch: ${{ steps.collect.outputs.branch }} + platform: ${{ env.PLATFORM }} + steps: + - name: Collect the PR and the maintainers' comments + id: collect + uses: microsoft/playwright/.github/actions/collect-review@main + with: + pull-request: ${{ github.event.pull_request.number }} + - name: Post the run link + env: + GH_TOKEN: ${{ github.token }} + PR: ${{ github.event.pull_request.number }} + PLATFORM_NAME: ${{ fromJSON('{"linux":"Linux","mac":"macOS","windows":"Windows"}')[env.PLATFORM] }} + RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + run: | + body=$(printf "Hi, I'm the Playwright bot. I started to address the review comments on %s: [agent run](%s)\n" "$PLATFORM_NAME" "$RUN_URL") + gh pr comment "$PR" --body "$body" || echo "::warning::Could not post the run link" + + revise: + name: Revise PR ${{ github.event.pull_request.number }} (${{ needs.prepare.outputs.platform }}) + needs: prepare + runs-on: ${{ fromJSON('{"linux":"playwright-x64-ubuntu24-64-core","mac":"macos-15-xlarge","windows":"playwright-x64-server2025-64-core"}')[needs.prepare.outputs.platform] }} + timeout-minutes: 240 + concurrency: + group: revision-${{ github.event.pull_request.number }} + cancel-in-progress: false + defaults: + run: + shell: bash + env: + PR: ${{ github.event.pull_request.number }} + HEAD_SHA: ${{ needs.prepare.outputs.head }} + BASE_SHA: ${{ needs.prepare.outputs.base }} + TRIAGE_PLATFORM: ${{ needs.prepare.outputs.platform }} + permissions: + copilot-requests: write + outputs: + has_revision: ${{ steps.agent.outputs.has-result }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ needs.prepare.outputs.head }} + persist-credentials: false + - name: Fetch the base and record the directories + run: | + git fetch --no-tags --depth=1 origin "$BASE_SHA" + echo "OUTPUT_DIR=${RUNNER_TEMP//\\//}/revision" >> "$GITHUB_ENV" + echo "REVIEW_DIR=${RUNNER_TEMP//\\//}/review" >> "$GITHUB_ENV" + echo "SCRATCH_DIR=${RUNNER_TEMP//\\//}/scratch" >> "$GITHUB_ENV" + echo "REPO_DIR=${GITHUB_WORKSPACE//\\//}" >> "$GITHUB_ENV" + mkdir -p "${RUNNER_TEMP//\\//}/revision" "${RUNNER_TEMP//\\//}/scratch" + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + name: review + path: ${{ runner.temp }}/review + - name: Set up the runner + uses: ./.github/actions/setup-triage-runner + - name: Revise with Copilot CLI + id: agent + # No GitHub tools and no URL access: the agent reads the PR only from the review the prepare job filtered. + uses: microsoft/playwright/.github/actions/run-agent@main + with: + prompt: | + Follow the shared guide revision.md and this repo's AGENTS.md. Read both in full before starting. $PLAYWRIGHT_DIR is the .upstream/ checkout of microsoft/playwright at the commit in DRIVER_VERSION; do not edit, commit or switch branches in it. + + Pull request: https://github.com/microsoft/playwright-python/pull/${{ github.event.pull_request.number }} + output-dir: ${{ env.OUTPUT_DIR }} + result-file: commit/commit.json + artifact-name: revision + exec: ${{ env.CI_ENV_EXEC }} + + push: + name: Push the revision + needs: [prepare, revise] + # commit.json is written last, so a finished revision is pushed even when the job failed later. + if: ${{ !cancelled() && needs.revise.outputs.has_revision == 'true' }} + runs-on: ubuntu-latest + steps: + - uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0 + id: app-token + with: + app-id: ${{ vars.PLAYWRIGHT_APP_ID }} + private-key: ${{ secrets.PLAYWRIGHT_PRIVATE_KEY }} + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ needs.prepare.outputs.head }} + token: ${{ steps.app-token.outputs.token }} + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + name: revision + path: ${{ runner.temp }}/revision + # Never execute code or scripts from the proposed changes in the app job. + - name: Validate and push + uses: microsoft/playwright/.github/actions/publish-patch@main + with: + token: ${{ steps.app-token.outputs.token }} + dir: ${{ runner.temp }}/revision/commit + patch: revision.patch + metadata: commit.json + branch: ${{ needs.prepare.outputs.branch }} diff --git a/.github/workflows/triage.yml b/.github/workflows/triage.yml new file mode 100644 index 000000000..064e213be --- /dev/null +++ b/.github/workflows/triage.yml @@ -0,0 +1,227 @@ +# Copilot triages and fixes an issue, or works on a task given by hand, without write credentials. +# Fresh jobs post the run link on the issue, open the completed fix as a draft PR using the app, so +# CI runs on it, and comment on the issue. The steps that are the same for every Playwright repo are +# actions in microsoft/playwright. The skill in .claude/skills/playwright-triage/ has what is +# specific to this repo. +name: Issue triage + +on: + # Apply needs-triage, needs-triage-mac or needs-triage-win; re-apply it to trigger another pass. + issues: + types: [labeled] + workflow_dispatch: + inputs: + issue: + description: "Issue number to triage; leave empty to work on the task alone" + type: string + required: false + default: "" + platform: + description: "Platform to triage on" + type: choice + required: true + default: linux + options: + - linux + - mac + - windows + task: + description: "Task to direct the triage, such as what to focus on; the whole job when there is no issue" + type: string + required: false + default: "" + +permissions: {} + +jobs: + start: + name: Post the run link on the issue + # The triage job's condition, for runs that have an issue. + if: >- + github.repository == 'microsoft/playwright-python' && + ((github.event_name == 'workflow_dispatch' && inputs.issue != '') || + contains(fromJSON('["needs-triage","needs-triage-mac","needs-triage-win"]'), github.event.label.name)) + runs-on: ubuntu-latest + permissions: + issues: write + outputs: + comment: ${{ steps.post.outputs.id }} + env: + ISSUE: ${{ github.event.issue.number || inputs.issue }} + PLATFORM_NAME: ${{ fromJSON('{"linux":"Linux","mac":"macOS","windows":"Windows","needs-triage":"Linux","needs-triage-mac":"macOS","needs-triage-win":"Windows"}')[inputs.platform || github.event.label.name] }} + GH_TOKEN: ${{ github.token }} + GH_REPO: ${{ github.repository }} + WORKFLOW_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + steps: + - name: Post the run link + id: post + run: | + [[ "$ISSUE" =~ ^[1-9][0-9]*$ ]] || { echo "issue must be a positive integer" >&2; exit 1; } + body=$(printf "Hi, I'm the Playwright bot. I started to triage this issue on %s: [agent run](%s)\n" "$PLATFORM_NAME" "$WORKFLOW_URL") + id=$(gh api "repos/$GH_REPO/issues/$ISSUE/comments" -f body="$body" --jq .node_id) + echo "id=$id" >> "$GITHUB_OUTPUT" + + triage: + name: Triage ${{ (github.event.issue.number || inputs.issue) && format('issue {0}', github.event.issue.number || inputs.issue) || 'task' }} (${{ inputs.platform || github.event.label.name }}) + if: >- + github.repository == 'microsoft/playwright-python' && + (github.event_name == 'workflow_dispatch' || + contains(fromJSON('["needs-triage","needs-triage-mac","needs-triage-win"]'), github.event.label.name)) + # Falls back to a runner name for unrelated labels, where the job is skipped anyway. + runs-on: ${{ fromJSON('{"linux":"playwright-x64-ubuntu24-64-core","mac":"macos-15-xlarge","windows":"playwright-x64-server2025-64-core"}')[inputs.platform || fromJSON('{"needs-triage":"linux","needs-triage-mac":"mac","needs-triage-win":"windows"}')[github.event.label.name]] || 'ubuntu-latest' }} + timeout-minutes: 240 + concurrency: + group: triage-${{ github.event.issue.number || inputs.issue || github.run_id }} + cancel-in-progress: false + defaults: + run: + shell: bash + env: + ISSUE: ${{ github.event.issue.number || inputs.issue }} + TRIAGE_PLATFORM: ${{ inputs.platform || fromJSON('{"needs-triage":"linux","needs-triage-mac":"mac","needs-triage-win":"windows"}')[github.event.label.name] }} + TASK: ${{ inputs.task }} + permissions: + copilot-requests: write + outputs: + base: ${{ steps.base.outputs.sha }} + setup: ${{ steps.setup.outcome }} + agent: ${{ steps.agent.outputs.outcome }} + has_fix: ${{ steps.agent.outputs.has-result }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: main + persist-credentials: false + - name: Record the base and the directories + id: base + run: | + if [[ -n "$ISSUE" ]]; then + [[ "$ISSUE" =~ ^[1-9][0-9]*$ ]] || { echo "issue must be a positive integer" >&2; exit 1; } + elif [[ -z "${TASK//[[:space:]]/}" ]]; then + echo "give an issue, a task or both" >&2; exit 1 + fi + echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT" + echo "OUTPUT_DIR=${RUNNER_TEMP//\\//}/triage" >> "$GITHUB_ENV" + echo "SCRATCH_DIR=${RUNNER_TEMP//\\//}/scratch" >> "$GITHUB_ENV" + echo "REPO_DIR=${GITHUB_WORKSPACE//\\//}" >> "$GITHUB_ENV" + mkdir -p "${RUNNER_TEMP//\\//}/triage" "${RUNNER_TEMP//\\//}/scratch" + - name: Set up the runner + id: setup + uses: ./.github/actions/setup-triage-runner + - name: Write the prompt + id: prompt + run: | + delimiter="PROMPT_$RANDOM$RANDOM" + { + echo "prompt<<$delimiter" + echo "Follow .claude/skills/playwright-triage/SKILL.md. Read it, the guide and the AGENTS.md" + echo "it points to in full before starting." + if [[ -n "$ISSUE" ]]; then + printf '\nIssue: https://github.com/%s/issues/%s\n' "$GITHUB_REPOSITORY" "$ISSUE" + fi + if [[ -n "$TASK" ]]; then + printf '\nTask: %s\n' "$TASK" + fi + echo "$delimiter" + } >> "$GITHUB_OUTPUT" + - name: Triage with Copilot CLI + id: agent + uses: microsoft/playwright/.github/actions/run-agent@main + with: + prompt: ${{ steps.prompt.outputs.prompt }} + output-dir: ${{ env.OUTPUT_DIR }} + result-file: fix/proposal.json + artifact-name: triage + github-tools: true + exec: ${{ env.CI_ENV_EXEC }} + - name: Check the upstream bug report + run: | + if [[ -f "$OUTPUT_DIR/upstream/report.json" ]]; then + title=$(node -p 'String(JSON.parse(require("fs").readFileSync(process.argv[1], "utf8")).title ?? "")' "$OUTPUT_DIR/upstream/report.json" || true) + if [[ -z "$title" ]]; then + echo "::warning::upstream/report.json has no title, so the report is not appended to the comment" + else + echo "Upstream bug report: $title" >> "$GITHUB_STEP_SUMMARY" + fi + fi + + publish: + name: Open draft pull request + needs: triage + # proposal.json is written last, so a finished proposal is published even when the job failed later. + if: ${{ !cancelled() && needs.triage.outputs.has_fix == 'true' }} + runs-on: ubuntu-latest + outputs: + pr: ${{ steps.publish.outputs.pull-request }} + steps: + - uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0 + id: app-token + with: + app-id: ${{ vars.PLAYWRIGHT_APP_ID }} + private-key: ${{ secrets.PLAYWRIGHT_PRIVATE_KEY }} + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ needs.triage.outputs.base }} + token: ${{ steps.app-token.outputs.token }} + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + name: triage + path: ${{ runner.temp }}/triage + # Never execute code or scripts from the proposed changes in the app job. + - name: Validate, push and open draft + id: publish + uses: microsoft/playwright/.github/actions/publish-patch@main + with: + token: ${{ steps.app-token.outputs.token }} + dir: ${{ runner.temp }}/triage/fix + patch: fix.patch + metadata: proposal.json + body-file: pr-body.md + branch: triage/${{ github.event.issue.number || inputs.issue || 'task' }}/${{ github.run_id }}-${{ github.run_attempt }} + commit-body: "${{ (github.event.issue.number || inputs.issue) && format('Fixes: https://github.com/{0}/issues/{1}', github.repository, github.event.issue.number || inputs.issue) || '' }}" + pull-request: true + + comment: + name: Comment on the issue + needs: [start, triage, publish] + # Runs after every triage run that has an issue, including failed ones, and whatever publish did. + if: ${{ !cancelled() && needs.triage.result != 'skipped' && (github.event_name == 'issues' || inputs.issue != '') }} + runs-on: ubuntu-latest + permissions: + issues: write + steps: + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + # A failed triage run may not have uploaded anything. + continue-on-error: true + with: + name: triage + path: ${{ runner.temp }}/triage + # The bot never files upstream; the comment hands the report to a maintainer. + - name: Describe the upstream bug report + id: upstream + env: + OUTPUT_DIR: ${{ runner.temp }}/triage + run: | + extra="" + if [[ -s "$OUTPUT_DIR/upstream/report.md" && -f "$OUTPUT_DIR/upstream/report.json" ]]; then + title=$(jq -r '.title // empty' "$OUTPUT_DIR/upstream/report.json") + if [[ -n "$title" ]]; then + extra=$(printf 'Upstream bug report, for a maintainer to file in microsoft/playwright:\n\n**Title:** %s\n\n
\nBody\n\n````markdown\n%s\n````\n\n
' "$title" "$(cat "$OUTPUT_DIR/upstream/report.md")") + fi + fi + delimiter="EXTRA_$RANDOM$RANDOM" + { echo "extra<<$delimiter"; echo "$extra"; echo "$delimiter"; } >> "$GITHUB_OUTPUT" + - name: Post the triage comment + uses: microsoft/playwright/.github/actions/triage-comment@main + with: + issue: ${{ github.event.issue.number || inputs.issue }} + dir: ${{ runner.temp }}/triage + footer: Triaged on ${{ fromJSON('{"linux":"Linux","mac":"macOS","windows":"Windows","needs-triage":"Linux","needs-triage-mac":"macOS","needs-triage-win":"Windows"}')[inputs.platform || github.event.label.name] }} by the Playwright bot + job-result: ${{ needs.triage.result }} + setup-outcome: ${{ needs.triage.outputs.setup }} + agent-outcome: ${{ needs.triage.outputs.agent }} + setup-failure-reason: setting up the runner, building the project or downloading the browsers failed + pull-request: ${{ needs.publish.outputs.pr }} + publish-result: ${{ needs.publish.result }} + extra: ${{ steps.upstream.outputs.extra }} + start-comment: ${{ needs.start.outputs.comment }} diff --git a/AGENTS.md b/AGENTS.md index bd3a84377..4d32dc134 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -36,6 +36,10 @@ When `DRIVER_VERSION` changes, for example after a pull, run `tests/async` in separate pytest runs: a sync test fails in a process where an async test ran. +The install is editable: a change under `playwright/` needs no build, the next +`pytest` runs it. Run `mypy playwright` after it. Rebuild the wheel only when +`DRIVER_VERSION` or `NODE_VERSION` changes. + ## The port `playwright/_impl/` is a line-by-line port of the JS client, @@ -105,6 +109,32 @@ left). - Python-only tests (the sync API, asyncio, regressions) go in `tests/{async,sync}/python/`. Keep them few: extend a test before you add one. +### Regression tests + +- A regression test for a bug of the Python client goes in `tests/async/python/` or + `tests/sync/python/`, in the file named after the upstream file of the feature + (`test_page.py`), as one added case in an existing test. Write it once, in the + flavor that the issue uses. When a port carries `xfail(reason="python: ...", + strict=True)` for the bug, remove the mark instead: that port is the regression test. +- Tag the test with the issue in a comment on the first line of its body: + `# Regression test for https://github.com/microsoft/playwright-python/issues/.` +- The fixtures are `page`, `context`, `browser`, `browser_type`, `server`, + `https_server`, `browser_name`, `assetdir` and, in sync, `sync_gather`; + `tests/AGENTS.md` has the whole map. Use `server.set_route(path, handler)`, + `server.PREFIX` and `server.EMPTY_PAGE`. A handler takes a Twisted request: + `setHeader`, `setResponseCode`, `write`, `finish`. +- Annotate every parameter and the return value (`-> None`): mypy and pyright check + the tests. A `Protocol violations` failure is a bug in the client, not in the test. +- Good models: `test_response_headers_should_work` in + `tests/async/python/test_network.py`, and + `test_sync_calls_should_raise_after_driver_exit` in `tests/sync/python/test_sync.py`. +- Run one file with one browser: `pytest tests/async/python/test_page.py --browser + chromium`, then `--browser firefox` and `--browser webkit`. Without `--browser`, + pytest runs all three. `-k ` selects one test, `--headed` runs the browser + headed. The timeout of a test is 30 s; `@pytest.mark.slow` triples it. A change in + `playwright/_impl/` serves both APIs: run the ported files of the feature in both + flavors. + ## Rolls Use the `playwright-roll` skill.