-
Notifications
You must be signed in to change notification settings - Fork 3.6k
feat(runner): Integrate Test262 execution with wptrunner and add smoke tests [pt 3/5] #56842
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
jcscottiii
wants to merge
1
commit into
feat/test262-serve-handler
Choose a base branch
from
feat/test262-runner-integration
base: feat/test262-serve-handler
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
feat(runner): Integrate Test262 execution with wptrunner and add smoke tests [pt 3/5] #56842
jcscottiii
wants to merge
1
commit into
feat/test262-serve-handler
from
feat/test262-runner-integration
+717
−8
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
35d9c44 to
3374ad9
Compare
3374ad9 to
a2d6c63
Compare
…e tests This commit completes the core integration of Test262 into the WPT runner (`wptrunner`), enabling the execution of Test262 JavaScript conformance tests within the browser environment. This work builds upon the manifest tooling (PR 1 #56840) and server-side handling (PR 2 #56841) to provide a full end-to-end testing solution for Test262. Key components introduced and integrated: - **Client-Side Harness:** New JavaScript files (`harness-adapter.js`, `testharness-client.js`, `testharness.js`) under `resources/test262/` provide the necessary environment for Test262 tests to execute in an `iframe` and report results back to the main WPT testharness. - **Test262 Assertion Libraries (Vendored):** For the purpose of enabling these initial smoke tests, the standard Test262 assertion libraries (`assert.js` and `sta.js`) are directly placed into `third_party/test262/harness/`. This provides the immediate dependencies required by the smoke tests. An autoamted vendoring solution, including version tracking via `vendored.toml`, will be implemented in a subsequent PR focused on automated updates, as described in the RFC. - **Wptrunner Integration:** Modifications to `tools/wptrunner/wpttest.py` and `tools/wptrunner/browsers/*.py` enable `wptrunner` to recognize the `test262` test type and execute it using existing testharness executors. - **Smoke Tests and Metadata:** A suite of basic Test262 smoke tests (`infrastructure/test262/`) and their corresponding expected results (`infrastructure/metadata/infrastructure/test262/`) are added to verify the correct functioning of the entire integration. - **Linting Exemption:** `lint.ignore` is updated to exclude the vendored Test262 harness files from linting, respecting their upstream style. This integration allows browser vendors to run Test262 directly with `wptrunner`, streamlining conformance testing efforts as specified in the RFC: web-platform-tests/rfcs#229 This commit is the third in a series of smaller PRs split from the larger, original implementation in #55997.
16aa645 to
d5eb82e
Compare
a2d6c63 to
258c427
Compare
This was referenced Dec 18, 2025
Contributor
DanielRyanSmith
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This suggestion came from a Gemini code review:
Harness Adapter Robustness (Critical)
Focus: resources/test262/harness-adapter.js
Critique: Bridging Test262's global-polluting API (e.g., $DONE, Test262Error) to the structured testharness.js API is error-prone.
- Exception Handling: Ensure the adapter explicitly wraps the execution of the Test262 test code in a
try/catchblock (or useswindow.onerror). Test262 tests often throw bare strings or non-Error objects. The adapter must normalize these into meaningfultestharnessfailure messages, otherwise, the runner might just report a generic "Script error." or timeout. - $DONE Semantics: Verify that calling
$DONE()(without arguments) signals a PASS, and$DONE(error)signals a FAIL. A common bug is treating any call to$DONEas a completion without checking the argument.
DanielRyanSmith
approved these changes
Dec 18, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit completes the core integration of Test262 into the WPT runner (
wptrunner), enabling the execution of Test262 JavaScript conformance tests within the browser environment. This work builds upon the manifest tooling (PR 1 #56840) and server-side handling (PR 2 #56841) to provide a full end-to-end testing solution for Test262.Key components introduced and integrated:
harness-adapter.js,testharness-client.js,testharness.js) underresources/test262/provide the necessary environment for Test262 tests to execute in aniframeand report results back to the main WPT testharness.assert.jsandsta.js) are directly placed intothird_party/test262/harness/. This provides the immediate dependencies required by the smoke tests. An autoamted vendoring solution, including version tracking viavendored.toml, will be implemented in a subsequent PR focused on automated updates, as described in the RFC.tools/wptrunner/wpttest.pyandtools/wptrunner/browsers/*.pyenablewptrunnerto recognize thetest262test type and execute it using existing testharness executors.infrastructure/test262/) and their corresponding expected results (infrastructure/metadata/infrastructure/test262/) are added to verify the correct functioning of the entire integration.lint.ignoreis updated to exclude the vendored Test262 harness files from linting, respecting their upstream style.This integration allows browser vendors to run Test262 directly with
wptrunner, streamlining conformance testing efforts as specified in the RFC: web-platform-tests/rfcs#229This commit is the third in a series of smaller PRs split from the larger, original implementation in #55997.