diff --git a/.claude/skills/playwright-triage/SKILL.md b/.claude/skills/playwright-triage/SKILL.md new file mode 100644 index 000000000..a81efd618 --- /dev/null +++ b/.claude/skills/playwright-triage/SKILL.md @@ -0,0 +1,91 @@ +--- +name: playwright-triage +description: Triage and fix one microsoft/playwright-python issue, or do a maintainer's task, unattended inside the Issue triage GitHub Actions workflow (.github/workflows/triage.yml), which a needs-triage, needs-triage-mac or needs-triage-win label starts, or a maintainer starts by hand with an issue, a task or both. Reproduces the issue on the runner, then leaves a verified fix proposal for the workflow to open as a draft PR, an investigation note for it to post on the issue, or an upstream bug report for a maintainer to file in microsoft/playwright. Only for that workflow. +--- + +# Playwright Python Issue Triage + +You run unattended in the [Issue triage](../../../.github/workflows/triage.yml) workflow of +`microsoft/playwright-python`. First read the shared guide, `$AGENT_GUIDES_DIR/triage.md`, a copy of +[microsoft/playwright/.github/actions/run-agent/guides/triage.md](https://github.com/microsoft/playwright/blob/main/.github/actions/run-agent/guides/triage.md), +and then [AGENTS.md](../../../AGENTS.md), both in full. The guide says how to triage an issue in any +Playwright repo. AGENTS.md says how to build, test and lint this one, and where a regression test +goes. This file covers what neither has: the checkouts, how to run a published version, and the +outcome that this repo adds, an upstream bug report. + +## The checkouts + +`$REPO_DIR` is the checkout of `microsoft/playwright-python` that the guide describes. Its requirements +are installed (`pip install -r local-requirements.txt`, `pip install -r requirements.txt`, +`pip install -e .`), the wheel is built (`python -m build --wheel`, which builds the driver into +`playwright/driver/`), and Chromium, Firefox and WebKit are installed with +`python -m playwright install --with-deps`. It is the working directory, and the checkout that the +guides refer to. The clone is shallow: run `git fetch --unshallow --tags origin` in it before you +bisect or use `git tag --contains`. + +`$PLAYWRIGHT_DIR` is `$REPO_DIR/.upstream/`, a full clone of `microsoft/playwright` at the commit +in `DRIVER_VERSION`, created by `python scripts/upstream.py checkout`. The driver was built from it. +Read upstream code there. Do not edit, commit or switch branches in it: the wheel build and +`./scripts/lint.sh` stop when it is not at that commit or has local changes. Before you run upstream +tests in it, it needs `npm ci`, `npm run build` and `npx playwright install`. The wheel build +(`scripts/build_driver.py`) ran the first two there; run them again when `node_modules/` is missing. + +## Published versions + +To run a published version, for example to bisect, create `$SCRATCH_DIR/playwright-python-//` +and run there: + +```bash +python -m venv env && source env/bin/activate # env\Scripts\activate on Windows +pip install playwright== +python -m playwright install +``` + +The fixtures of the repo are not available there. Write the condensed repro as a plain script with +`sync_playwright()` or `async_playwright()` in that directory, with `page.set_content()` in place of +the test server, and run it from there. In `$REPO_DIR`, the source tree shadows the installed package. + +The driver of a release is in `DRIVER_VERSION` at its tag: `git show v:DRIVER_VERSION` +prints the upstream commit it was built from, or, for releases up to 1.64, the `playwright-core` +version of the driver; `npm view playwright-core@ gitHead` prints its commit. Match a +regression in the driver to commits with `git -C $PLAYWRIGHT_DIR log .. -- `. + +## Upstream bugs + +The client talks to the Node driver that is built from `$PLAYWRIGHT_DIR`, and `microsoft/playwright` +and this repo are one team. Reproduce the bug in Python first. Then write the same calls as an +upstream test and run it in `$PLAYWRIGHT_DIR`. The triage skill of `microsoft/playwright`, in +`$PLAYWRIGHT_DIR/.claude/skills/playwright-triage/`, says where such a test goes, which fixtures it +uses and how to run it. The rest of that skill is not for this run. Put the test in a new spec +file, and delete the file when you are done, so that the checkout is clean. + +- The Node test passes: the bug is in the Python client. Fix it here. +- The Node test fails too: the root cause is upstream, and the fix lands in `microsoft/playwright`. + Do not work around it in `playwright/_impl/`. Run `git -C $PLAYWRIGHT_DIR fetch origin main`. + When `git -C $PLAYWRIGHT_DIR log HEAD..origin/main -- ` shows a fix that already landed, + the note says that it reaches this repo with the next roll. Otherwise write the report. +- The Python API lacks a member or an option: the generated API has everything that the upstream + docs in `$PLAYWRIGHT_DIR/docs/src/api` give Python (`* langs:` decides; no `langs` means every + language). The gap is in the docs, so it is an upstream change too. Do not edit a generated file, + and do not add to `scripts/codegen/python_rules.js` to go around the docs. + +Create `$OUTPUT_DIR/upstream/` with: + +- `report.md`: the report for an engineer of `microsoft/playwright` who did not read the Python + issue: what fails, expected versus actual, the versions (the Python package, the commit in + `DRIVER_VERSION`, the browser, the platform), the root cause when you found it, and the condensed + Node test as the repro. +- `report.json`: `{"title": "..."}`, at most 256 characters, for example + `route.fulfill() drops the charset of Content-Type`. + +The comment job appends the title and the body to your comment, for a maintainer to file in +`microsoft/playwright`. The bot never files it. Write `report.json` last, when `report.md` and the +comment are done. Like `proposal.json`, it marks the report as ready. + +## PRs from third parties + +Do not run a PR that changes `setup.py`, `pyproject.toml`, `setup.cfg`, `local-requirements.txt`, +`requirements.txt`, `DRIVER_VERSION`, `NODE_VERSION`, `.pre-commit-config.yaml`, `test.sh`, +`scripts/`, `utils/`, `.github/`, `.azure-pipelines/`, `.claude/`, or a generated file. A PR that +changes a ported test under `tests/{async,sync}/{page,library}`, or adds a mark to one, has faults: +the ports have zero deviations. diff --git a/.github/actions/setup-triage-runner/action.yml b/.github/actions/setup-triage-runner/action.yml new file mode 100644 index 000000000..0e1588634 --- /dev/null +++ b/.github/actions/setup-triage-runner/action.yml @@ -0,0 +1,36 @@ +name: 'Set up the triage runner' +description: > + Prepares the environment that the triage and revision agents work in: the checkout built as CI + builds it, with the browsers and the Copilot CLI installed, and the upstream checkout in .upstream/ + that the driver is built from. Exports PLAYWRIGHT_DIR, the path of that checkout, and CI_ENV_EXEC, + the wrapper from microsoft/playwright that runs a command under xvfb on Linux and at normal I/O + priority on Windows. +runs: + using: composite + steps: + - uses: microsoft/playwright/.github/actions/ci-env@main + - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 + with: + python-version: '3.12' + - uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0 + with: + node-version: '24' + - name: Install the requirements + shell: bash + run: | + python -m pip install --upgrade pip + pip install -r local-requirements.txt + pip install -r requirements.txt + pip install -e . + - name: Check out upstream and build the wheel + shell: bash + run: | + python scripts/upstream.py checkout + python -m build --wheel + echo "PLAYWRIGHT_DIR=${GITHUB_WORKSPACE//\\//}/.upstream" >> "$GITHUB_ENV" + - name: Install the browsers + shell: bash + run: python -m playwright install --with-deps chromium firefox webkit + - name: Install the Copilot CLI + shell: bash + run: npm install -g @github/copilot diff --git a/.github/workflows/revision.yml b/.github/workflows/revision.yml new file mode 100644 index 000000000..7b4ee534b --- /dev/null +++ b/.github/workflows/revision.yml @@ -0,0 +1,132 @@ +# Copilot addresses the maintainers' review comments on a draft PR that the triage workflow opened, +# without write credentials. A fresh job pushes the new commit to the PR's branch using the app, so +# CI runs on it. The steps that are the same for every Playwright repo are actions in +# microsoft/playwright, and the shared guide they carry says how to revise; the prompt below has +# what is specific to this repo. +name: PR revision + +on: + # Apply needs-revision, needs-revision-mac or needs-revision-win; re-apply it to trigger another pass. + # pull_request_target always runs this file from main; the jobs only accept the triage app's own PRs. + pull_request_target: + types: [labeled] + +permissions: {} + +jobs: + prepare: + name: Collect the maintainers' comments + if: >- + github.repository == 'microsoft/playwright-python' && + contains(fromJSON('["needs-revision","needs-revision-mac","needs-revision-win"]'), github.event.label.name) && + github.event.pull_request.state == 'open' && + github.event.pull_request.user.login == 'microsoft-playwright-automation[bot]' && + github.event.pull_request.head.repo.full_name == github.repository + runs-on: ubuntu-latest + permissions: + pull-requests: write + env: + PLATFORM: ${{ fromJSON('{"needs-revision":"linux","needs-revision-mac":"mac","needs-revision-win":"windows"}')[github.event.label.name] }} + outputs: + head: ${{ steps.collect.outputs.head }} + base: ${{ steps.collect.outputs.base }} + branch: ${{ steps.collect.outputs.branch }} + platform: ${{ env.PLATFORM }} + steps: + - name: Collect the PR and the maintainers' comments + id: collect + uses: microsoft/playwright/.github/actions/collect-review@main + with: + pull-request: ${{ github.event.pull_request.number }} + - name: Post the run link + env: + GH_TOKEN: ${{ github.token }} + PR: ${{ github.event.pull_request.number }} + PLATFORM_NAME: ${{ fromJSON('{"linux":"Linux","mac":"macOS","windows":"Windows"}')[env.PLATFORM] }} + RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + run: | + body=$(printf "Hi, I'm the Playwright bot. I started to address the review comments on %s: [agent run](%s)\n" "$PLATFORM_NAME" "$RUN_URL") + gh pr comment "$PR" --body "$body" || echo "::warning::Could not post the run link" + + revise: + name: Revise PR ${{ github.event.pull_request.number }} (${{ needs.prepare.outputs.platform }}) + needs: prepare + runs-on: ${{ fromJSON('{"linux":"playwright-x64-ubuntu24-64-core","mac":"macos-15-xlarge","windows":"playwright-x64-server2025-64-core"}')[needs.prepare.outputs.platform] }} + timeout-minutes: 240 + concurrency: + group: revision-${{ github.event.pull_request.number }} + cancel-in-progress: false + defaults: + run: + shell: bash + env: + PR: ${{ github.event.pull_request.number }} + HEAD_SHA: ${{ needs.prepare.outputs.head }} + BASE_SHA: ${{ needs.prepare.outputs.base }} + TRIAGE_PLATFORM: ${{ needs.prepare.outputs.platform }} + permissions: + copilot-requests: write + outputs: + has_revision: ${{ steps.agent.outputs.has-result }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ needs.prepare.outputs.head }} + persist-credentials: false + - name: Fetch the base and record the directories + run: | + git fetch --no-tags --depth=1 origin "$BASE_SHA" + echo "OUTPUT_DIR=${RUNNER_TEMP//\\//}/revision" >> "$GITHUB_ENV" + echo "REVIEW_DIR=${RUNNER_TEMP//\\//}/review" >> "$GITHUB_ENV" + echo "SCRATCH_DIR=${RUNNER_TEMP//\\//}/scratch" >> "$GITHUB_ENV" + echo "REPO_DIR=${GITHUB_WORKSPACE//\\//}" >> "$GITHUB_ENV" + mkdir -p "${RUNNER_TEMP//\\//}/revision" "${RUNNER_TEMP//\\//}/scratch" + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + name: review + path: ${{ runner.temp }}/review + - name: Set up the runner + uses: ./.github/actions/setup-triage-runner + - name: Revise with Copilot CLI + id: agent + # No GitHub tools and no URL access: the agent reads the PR only from the review the prepare job filtered. + uses: microsoft/playwright/.github/actions/run-agent@main + with: + prompt: | + Follow the shared guide revision.md and this repo's AGENTS.md. Read both in full before starting. $PLAYWRIGHT_DIR is the .upstream/ checkout of microsoft/playwright at the commit in DRIVER_VERSION; do not edit, commit or switch branches in it. + + Pull request: https://github.com/microsoft/playwright-python/pull/${{ github.event.pull_request.number }} + output-dir: ${{ env.OUTPUT_DIR }} + result-file: commit/commit.json + artifact-name: revision + exec: ${{ env.CI_ENV_EXEC }} + + push: + name: Push the revision + needs: [prepare, revise] + # commit.json is written last, so a finished revision is pushed even when the job failed later. + if: ${{ !cancelled() && needs.revise.outputs.has_revision == 'true' }} + runs-on: ubuntu-latest + steps: + - uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0 + id: app-token + with: + app-id: ${{ vars.PLAYWRIGHT_APP_ID }} + private-key: ${{ secrets.PLAYWRIGHT_PRIVATE_KEY }} + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ needs.prepare.outputs.head }} + token: ${{ steps.app-token.outputs.token }} + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + name: revision + path: ${{ runner.temp }}/revision + # Never execute code or scripts from the proposed changes in the app job. + - name: Validate and push + uses: microsoft/playwright/.github/actions/publish-patch@main + with: + token: ${{ steps.app-token.outputs.token }} + dir: ${{ runner.temp }}/revision/commit + patch: revision.patch + metadata: commit.json + branch: ${{ needs.prepare.outputs.branch }} diff --git a/.github/workflows/triage.yml b/.github/workflows/triage.yml new file mode 100644 index 000000000..064e213be --- /dev/null +++ b/.github/workflows/triage.yml @@ -0,0 +1,227 @@ +# Copilot triages and fixes an issue, or works on a task given by hand, without write credentials. +# Fresh jobs post the run link on the issue, open the completed fix as a draft PR using the app, so +# CI runs on it, and comment on the issue. The steps that are the same for every Playwright repo are +# actions in microsoft/playwright. The skill in .claude/skills/playwright-triage/ has what is +# specific to this repo. +name: Issue triage + +on: + # Apply needs-triage, needs-triage-mac or needs-triage-win; re-apply it to trigger another pass. + issues: + types: [labeled] + workflow_dispatch: + inputs: + issue: + description: "Issue number to triage; leave empty to work on the task alone" + type: string + required: false + default: "" + platform: + description: "Platform to triage on" + type: choice + required: true + default: linux + options: + - linux + - mac + - windows + task: + description: "Task to direct the triage, such as what to focus on; the whole job when there is no issue" + type: string + required: false + default: "" + +permissions: {} + +jobs: + start: + name: Post the run link on the issue + # The triage job's condition, for runs that have an issue. + if: >- + github.repository == 'microsoft/playwright-python' && + ((github.event_name == 'workflow_dispatch' && inputs.issue != '') || + contains(fromJSON('["needs-triage","needs-triage-mac","needs-triage-win"]'), github.event.label.name)) + runs-on: ubuntu-latest + permissions: + issues: write + outputs: + comment: ${{ steps.post.outputs.id }} + env: + ISSUE: ${{ github.event.issue.number || inputs.issue }} + PLATFORM_NAME: ${{ fromJSON('{"linux":"Linux","mac":"macOS","windows":"Windows","needs-triage":"Linux","needs-triage-mac":"macOS","needs-triage-win":"Windows"}')[inputs.platform || github.event.label.name] }} + GH_TOKEN: ${{ github.token }} + GH_REPO: ${{ github.repository }} + WORKFLOW_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} + steps: + - name: Post the run link + id: post + run: | + [[ "$ISSUE" =~ ^[1-9][0-9]*$ ]] || { echo "issue must be a positive integer" >&2; exit 1; } + body=$(printf "Hi, I'm the Playwright bot. I started to triage this issue on %s: [agent run](%s)\n" "$PLATFORM_NAME" "$WORKFLOW_URL") + id=$(gh api "repos/$GH_REPO/issues/$ISSUE/comments" -f body="$body" --jq .node_id) + echo "id=$id" >> "$GITHUB_OUTPUT" + + triage: + name: Triage ${{ (github.event.issue.number || inputs.issue) && format('issue {0}', github.event.issue.number || inputs.issue) || 'task' }} (${{ inputs.platform || github.event.label.name }}) + if: >- + github.repository == 'microsoft/playwright-python' && + (github.event_name == 'workflow_dispatch' || + contains(fromJSON('["needs-triage","needs-triage-mac","needs-triage-win"]'), github.event.label.name)) + # Falls back to a runner name for unrelated labels, where the job is skipped anyway. + runs-on: ${{ fromJSON('{"linux":"playwright-x64-ubuntu24-64-core","mac":"macos-15-xlarge","windows":"playwright-x64-server2025-64-core"}')[inputs.platform || fromJSON('{"needs-triage":"linux","needs-triage-mac":"mac","needs-triage-win":"windows"}')[github.event.label.name]] || 'ubuntu-latest' }} + timeout-minutes: 240 + concurrency: + group: triage-${{ github.event.issue.number || inputs.issue || github.run_id }} + cancel-in-progress: false + defaults: + run: + shell: bash + env: + ISSUE: ${{ github.event.issue.number || inputs.issue }} + TRIAGE_PLATFORM: ${{ inputs.platform || fromJSON('{"needs-triage":"linux","needs-triage-mac":"mac","needs-triage-win":"windows"}')[github.event.label.name] }} + TASK: ${{ inputs.task }} + permissions: + copilot-requests: write + outputs: + base: ${{ steps.base.outputs.sha }} + setup: ${{ steps.setup.outcome }} + agent: ${{ steps.agent.outputs.outcome }} + has_fix: ${{ steps.agent.outputs.has-result }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: main + persist-credentials: false + - name: Record the base and the directories + id: base + run: | + if [[ -n "$ISSUE" ]]; then + [[ "$ISSUE" =~ ^[1-9][0-9]*$ ]] || { echo "issue must be a positive integer" >&2; exit 1; } + elif [[ -z "${TASK//[[:space:]]/}" ]]; then + echo "give an issue, a task or both" >&2; exit 1 + fi + echo "sha=$(git rev-parse HEAD)" >> "$GITHUB_OUTPUT" + echo "OUTPUT_DIR=${RUNNER_TEMP//\\//}/triage" >> "$GITHUB_ENV" + echo "SCRATCH_DIR=${RUNNER_TEMP//\\//}/scratch" >> "$GITHUB_ENV" + echo "REPO_DIR=${GITHUB_WORKSPACE//\\//}" >> "$GITHUB_ENV" + mkdir -p "${RUNNER_TEMP//\\//}/triage" "${RUNNER_TEMP//\\//}/scratch" + - name: Set up the runner + id: setup + uses: ./.github/actions/setup-triage-runner + - name: Write the prompt + id: prompt + run: | + delimiter="PROMPT_$RANDOM$RANDOM" + { + echo "prompt<<$delimiter" + echo "Follow .claude/skills/playwright-triage/SKILL.md. Read it, the guide and the AGENTS.md" + echo "it points to in full before starting." + if [[ -n "$ISSUE" ]]; then + printf '\nIssue: https://github.com/%s/issues/%s\n' "$GITHUB_REPOSITORY" "$ISSUE" + fi + if [[ -n "$TASK" ]]; then + printf '\nTask: %s\n' "$TASK" + fi + echo "$delimiter" + } >> "$GITHUB_OUTPUT" + - name: Triage with Copilot CLI + id: agent + uses: microsoft/playwright/.github/actions/run-agent@main + with: + prompt: ${{ steps.prompt.outputs.prompt }} + output-dir: ${{ env.OUTPUT_DIR }} + result-file: fix/proposal.json + artifact-name: triage + github-tools: true + exec: ${{ env.CI_ENV_EXEC }} + - name: Check the upstream bug report + run: | + if [[ -f "$OUTPUT_DIR/upstream/report.json" ]]; then + title=$(node -p 'String(JSON.parse(require("fs").readFileSync(process.argv[1], "utf8")).title ?? "")' "$OUTPUT_DIR/upstream/report.json" || true) + if [[ -z "$title" ]]; then + echo "::warning::upstream/report.json has no title, so the report is not appended to the comment" + else + echo "Upstream bug report: $title" >> "$GITHUB_STEP_SUMMARY" + fi + fi + + publish: + name: Open draft pull request + needs: triage + # proposal.json is written last, so a finished proposal is published even when the job failed later. + if: ${{ !cancelled() && needs.triage.outputs.has_fix == 'true' }} + runs-on: ubuntu-latest + outputs: + pr: ${{ steps.publish.outputs.pull-request }} + steps: + - uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0 + id: app-token + with: + app-id: ${{ vars.PLAYWRIGHT_APP_ID }} + private-key: ${{ secrets.PLAYWRIGHT_PRIVATE_KEY }} + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: ${{ needs.triage.outputs.base }} + token: ${{ steps.app-token.outputs.token }} + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + with: + name: triage + path: ${{ runner.temp }}/triage + # Never execute code or scripts from the proposed changes in the app job. + - name: Validate, push and open draft + id: publish + uses: microsoft/playwright/.github/actions/publish-patch@main + with: + token: ${{ steps.app-token.outputs.token }} + dir: ${{ runner.temp }}/triage/fix + patch: fix.patch + metadata: proposal.json + body-file: pr-body.md + branch: triage/${{ github.event.issue.number || inputs.issue || 'task' }}/${{ github.run_id }}-${{ github.run_attempt }} + commit-body: "${{ (github.event.issue.number || inputs.issue) && format('Fixes: https://github.com/{0}/issues/{1}', github.repository, github.event.issue.number || inputs.issue) || '' }}" + pull-request: true + + comment: + name: Comment on the issue + needs: [start, triage, publish] + # Runs after every triage run that has an issue, including failed ones, and whatever publish did. + if: ${{ !cancelled() && needs.triage.result != 'skipped' && (github.event_name == 'issues' || inputs.issue != '') }} + runs-on: ubuntu-latest + permissions: + issues: write + steps: + - uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2 + # A failed triage run may not have uploaded anything. + continue-on-error: true + with: + name: triage + path: ${{ runner.temp }}/triage + # The bot never files upstream; the comment hands the report to a maintainer. + - name: Describe the upstream bug report + id: upstream + env: + OUTPUT_DIR: ${{ runner.temp }}/triage + run: | + extra="" + if [[ -s "$OUTPUT_DIR/upstream/report.md" && -f "$OUTPUT_DIR/upstream/report.json" ]]; then + title=$(jq -r '.title // empty' "$OUTPUT_DIR/upstream/report.json") + if [[ -n "$title" ]]; then + extra=$(printf 'Upstream bug report, for a maintainer to file in microsoft/playwright:\n\n**Title:** %s\n\n
\nBody\n\n````markdown\n%s\n````\n\n
' "$title" "$(cat "$OUTPUT_DIR/upstream/report.md")") + fi + fi + delimiter="EXTRA_$RANDOM$RANDOM" + { echo "extra<<$delimiter"; echo "$extra"; echo "$delimiter"; } >> "$GITHUB_OUTPUT" + - name: Post the triage comment + uses: microsoft/playwright/.github/actions/triage-comment@main + with: + issue: ${{ github.event.issue.number || inputs.issue }} + dir: ${{ runner.temp }}/triage + footer: Triaged on ${{ fromJSON('{"linux":"Linux","mac":"macOS","windows":"Windows","needs-triage":"Linux","needs-triage-mac":"macOS","needs-triage-win":"Windows"}')[inputs.platform || github.event.label.name] }} by the Playwright bot + job-result: ${{ needs.triage.result }} + setup-outcome: ${{ needs.triage.outputs.setup }} + agent-outcome: ${{ needs.triage.outputs.agent }} + setup-failure-reason: setting up the runner, building the project or downloading the browsers failed + pull-request: ${{ needs.publish.outputs.pr }} + publish-result: ${{ needs.publish.result }} + extra: ${{ steps.upstream.outputs.extra }} + start-comment: ${{ needs.start.outputs.comment }} diff --git a/AGENTS.md b/AGENTS.md index bd3a84377..4d32dc134 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -36,6 +36,10 @@ When `DRIVER_VERSION` changes, for example after a pull, run `tests/async` in separate pytest runs: a sync test fails in a process where an async test ran. +The install is editable: a change under `playwright/` needs no build, the next +`pytest` runs it. Run `mypy playwright` after it. Rebuild the wheel only when +`DRIVER_VERSION` or `NODE_VERSION` changes. + ## The port `playwright/_impl/` is a line-by-line port of the JS client, @@ -105,6 +109,32 @@ left). - Python-only tests (the sync API, asyncio, regressions) go in `tests/{async,sync}/python/`. Keep them few: extend a test before you add one. +### Regression tests + +- A regression test for a bug of the Python client goes in `tests/async/python/` or + `tests/sync/python/`, in the file named after the upstream file of the feature + (`test_page.py`), as one added case in an existing test. Write it once, in the + flavor that the issue uses. When a port carries `xfail(reason="python: ...", + strict=True)` for the bug, remove the mark instead: that port is the regression test. +- Tag the test with the issue in a comment on the first line of its body: + `# Regression test for https://github.com/microsoft/playwright-python/issues/.` +- The fixtures are `page`, `context`, `browser`, `browser_type`, `server`, + `https_server`, `browser_name`, `assetdir` and, in sync, `sync_gather`; + `tests/AGENTS.md` has the whole map. Use `server.set_route(path, handler)`, + `server.PREFIX` and `server.EMPTY_PAGE`. A handler takes a Twisted request: + `setHeader`, `setResponseCode`, `write`, `finish`. +- Annotate every parameter and the return value (`-> None`): mypy and pyright check + the tests. A `Protocol violations` failure is a bug in the client, not in the test. +- Good models: `test_response_headers_should_work` in + `tests/async/python/test_network.py`, and + `test_sync_calls_should_raise_after_driver_exit` in `tests/sync/python/test_sync.py`. +- Run one file with one browser: `pytest tests/async/python/test_page.py --browser + chromium`, then `--browser firefox` and `--browser webkit`. Without `--browser`, + pytest runs all three. `-k ` selects one test, `--headed` runs the browser + headed. The timeout of a test is 30 s; `@pytest.mark.slow` triples it. A change in + `playwright/_impl/` serves both APIs: run the ported files of the feature in both + flavors. + ## Rolls Use the `playwright-roll` skill.