Skip to content

ci(acceptance): add hang-proof Playwright smoke gate#5623

Open
DavertMik wants to merge 1 commit into
4.xfrom
advisor/002-acceptance-smoke-gate
Open

ci(acceptance): add hang-proof Playwright smoke gate#5623
DavertMik wants to merge 1 commit into
4.xfrom
advisor/002-acceptance-smoke-gate

Conversation

@DavertMik

Copy link
Copy Markdown
Contributor

What

Re-establishes an acceptance-level CI gate for 4.x. Today CI runs only unit/rest/runner suites; the only acceptance workflow is acceptance-tests.yml.disabled, which still targets the 3.x branch and a docker-compose WebDriverIO setup. The exact failure class blocking 4.0 (promise-composition hangs / broken error propagation) has zero CI signal.

This adds a hang-proof Playwright smoke job that runs the currently-passing subset on every PR, so the passing set can only grow.

Files

  • test/acceptance/codecept.Playwright.smoke.js — an explicit allowlist config (tests: './{els,session,within}_test.js') of acceptance tests that pass reliably on the 4.x promise core.
  • .github/workflows/acceptance-smoke.ymlpush to 4.x + pull_request to **; timeout-minutes: 20 and an outer timeout 600 (defense-in-depth; exit 124 = hang); installs Chromium, boots the PHP test app, GET-polls health, runs the smoke suite, and uploads test/acceptance/output/ on failure.

Discovery (local, Node 22 / Playwright 1.59 / PHP 8.5)

Each acceptance file run individually under a hard timeout:

File Result
els_test.js PASS (30)
session_test.js PASS (9 + 2 skipped)
within_test.js PASS (12)
config_test.js FAIL (7/1) — one scenario hits a JSON API that returns HTML locally; environment-flaky, not a promise-core failure → excluded
coverage_test.js no @Playwright scenarios → excluded
retryTo_test.js no @Playwright scenarios → excluded

The smoke suite is 52 passed / 2 skipped / 0 failed across three consecutive local runs (~58s each; includes the @Playwright gherkin before_hook.feature via the base config).

Maintenance

The smoke glob is an allowlist: when a promise-core fix makes an excluded file pass, move it into the glob in the same PR. The long-term goal is to delete this config and gate on the full codecept.Playwright.js.

Notes for reviewers

  • No acceptance *_test.js file was edited, skipped, or tagged to make the gate green.
  • The disabled 3.x acceptance workflow is left untouched.
  • Defense in depth against runaway minutes: both timeout-minutes: 20 and the inner timeout 600.

🤖 Generated with Claude Code

Re-establishes acceptance-level CI signal for 4.x, which currently runs only
unit/rest/runner suites while the disabled acceptance workflow still targets
3.x + docker WebDriverIO.

- test/acceptance/codecept.Playwright.smoke.js: an explicit allowlist
  (els/session/within) of acceptance tests that pass reliably on the 4.x
  promise core. 52 passed / 2 skipped / 0 failed across three consecutive
  local runs.
- .github/workflows/acceptance-smoke.yml: runs the smoke suite on every PR
  with both timeout-minutes: 20 and an outer `timeout 600` (exit 124 = hang =
  release blocker), installs Chromium, boots the PHP test app, and uploads
  test/acceptance/output/ on failure.

The allowlist only grows: when a promise-core fix makes an excluded file pass,
move it into the glob in the same PR. Long-term goal is to gate on the full
codecept.Playwright.js and delete this config.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant