Skip to content

Conversation

@jcscottiii
Copy link
Contributor

This commit completes the core integration of Test262 into the WPT runner (wptrunner), enabling the execution of Test262 JavaScript conformance tests within the browser environment. This work builds upon the manifest tooling (PR 1 #56840) and server-side handling (PR 2 #56841) to provide a full end-to-end testing solution for Test262.

Key components introduced and integrated:

  • Client-Side Harness: New JavaScript files (harness-adapter.js, testharness-client.js, testharness.js) under resources/test262/ provide the necessary environment for Test262 tests to execute in an iframe and report results back to the main WPT testharness.
  • Test262 Assertion Libraries (Vendored): For the purpose of enabling these initial smoke tests, the standard Test262 assertion libraries (assert.js and sta.js) are directly placed into third_party/test262/harness/. This provides the immediate dependencies required by the smoke tests. An autoamted vendoring solution, including version tracking via vendored.toml, will be implemented in a subsequent PR focused on automated updates, as described in the RFC.
  • Wptrunner Integration: Modifications to tools/wptrunner/wpttest.py and tools/wptrunner/browsers/*.py enable wptrunner to recognize the test262 test type and execute it using existing testharness executors.
  • Smoke Tests and Metadata: A suite of basic Test262 smoke tests (infrastructure/test262/) and their corresponding expected results (infrastructure/metadata/infrastructure/test262/) are added to verify the correct functioning of the entire integration.
  • Linting Exemption: lint.ignore is updated to exclude the vendored Test262 harness files from linting, respecting their upstream style.

This integration allows browser vendors to run Test262 directly with wptrunner, streamlining conformance testing efforts as specified in the RFC: web-platform-tests/rfcs#229

This commit is the third in a series of smaller PRs split from the larger, original implementation in #55997.

@wpt-pr-bot wpt-pr-bot added infra third_party wptrunner The automated test runner, commonly called through ./wpt run labels Dec 18, 2025
@jcscottiii jcscottiii changed the title feat(runner): Integrate Test262 execution with wptrunner and add smoke tests feat(runner): Integrate Test262 execution with wptrunner and add smoke tests [pt 3/4] Dec 18, 2025
@jcscottiii jcscottiii changed the title feat(runner): Integrate Test262 execution with wptrunner and add smoke tests [pt 3/4] feat(runner): Integrate Test262 execution with wptrunner and add smoke tests [pt 3/5] Dec 18, 2025
@jcscottiii jcscottiii force-pushed the feat/test262-runner-integration branch from 35d9c44 to 3374ad9 Compare December 18, 2025 03:53
@jcscottiii jcscottiii force-pushed the feat/test262-runner-integration branch from 3374ad9 to a2d6c63 Compare December 18, 2025 04:34
…e tests

This commit completes the core integration of Test262 into the WPT
runner (`wptrunner`), enabling the execution of Test262 JavaScript
conformance tests within the browser environment. This work builds upon
the manifest tooling (PR 1 #56840) and server-side handling (PR 2 #56841) to provide
a full end-to-end testing solution for Test262.

Key components introduced and integrated:
- **Client-Side Harness:** New JavaScript files (`harness-adapter.js`,
  `testharness-client.js`, `testharness.js`) under `resources/test262/`
  provide the necessary environment for Test262 tests to execute in an
  `iframe` and report results back to the main WPT testharness.
- **Test262 Assertion Libraries (Vendored):** For the purpose of enabling these
  initial smoke tests, the standard Test262 assertion libraries (`assert.js`
  and `sta.js`) are directly placed into `third_party/test262/harness/`.
  This provides the immediate dependencies required by the smoke tests.
  An autoamted vendoring solution, including version tracking via
  `vendored.toml`, will be implemented in a subsequent PR focused on
  automated updates, as described in the RFC.
- **Wptrunner Integration:** Modifications to `tools/wptrunner/wpttest.py`
  and `tools/wptrunner/browsers/*.py` enable `wptrunner` to recognize the
  `test262` test type and execute it using existing testharness executors.
- **Smoke Tests and Metadata:** A suite of basic Test262 smoke tests
  (`infrastructure/test262/`) and their corresponding expected results
  (`infrastructure/metadata/infrastructure/test262/`) are added to verify
  the correct functioning of the entire integration.
- **Linting Exemption:** `lint.ignore` is updated to exclude the vendored
  Test262 harness files from linting, respecting their upstream style.

This integration allows browser vendors to run Test262 directly with `wptrunner`,
streamlining conformance testing efforts as specified in the RFC:
web-platform-tests/rfcs#229

This commit is the third in a series of smaller PRs split from the larger,
original implementation in #55997.
@jcscottiii jcscottiii force-pushed the feat/test262-serve-handler branch from 16aa645 to d5eb82e Compare December 18, 2025 14:05
@jcscottiii jcscottiii force-pushed the feat/test262-runner-integration branch from a2d6c63 to 258c427 Compare December 18, 2025 14:06
Copy link
Contributor

@DanielRyanSmith DanielRyanSmith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This suggestion came from a Gemini code review:

Harness Adapter Robustness (Critical)

Focus: resources/test262/harness-adapter.js
Critique: Bridging Test262's global-polluting API (e.g., $DONE, Test262Error) to the structured testharness.js API is error-prone.

  • Exception Handling: Ensure the adapter explicitly wraps the execution of the Test262 test code in a try/catch block (or uses window.onerror). Test262 tests often throw bare strings or non-Error objects. The adapter must normalize these into meaningful testharness failure messages, otherwise, the runner might just report a generic "Script error." or timeout.
  • $DONE Semantics: Verify that calling $DONE() (without arguments) signals a PASS, and $DONE(error) signals a FAIL. A common bug is treating any call to $DONE as a completion without checking the argument.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

infra manifest third_party wptrunner The automated test runner, commonly called through ./wpt run

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants