feat(automation): Add workflow to vendor and update Test262 tests [pt 4/5] #56843
+75
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jcscottiii
changed the title
jcscottiii
force-pushed
the
feat/test262-runner-integration
branch
from
35d9c44 to
3374ad9
Compare
jcscottiii
force-pushed
the
feat/test262-automation
branch
from
6406255 to
04e3166
Compare
jcscottiii
force-pushed
the
feat/test262-runner-integration
branch
from
3374ad9 to
a2d6c63
Compare
jcscottiii
force-pushed
the
feat/test262-automation
branch
from
04e3166 to
e8ec10a
Compare
jcscottiii
force-pushed
the
feat/test262-runner-integration
branch
from
a2d6c63 to
258c427
Compare
This commit introduces a GitHub Actions workflow to automate the
vendoring of the Test262 test suite into `third_party/test262`. This
automation is the final piece of the integration, ensuring that the
tests can be kept up-to-date with the upstream `tc39/test262` repository.
The workflow is configured to run weekly or can be triggered manually.
It performs the following steps:
1. Checks out both the WPT and `tc39/test262` repositories.
2. Selectively copies a subset of the Test262 tests into WPT.
3. Copies the essential Test262 harness files.
4. Creates a `vendored.toml` file at the root of the vendored directory,
recording the specific commit SHA from `tc39/test262` that the
files were sourced from. This ensures traceability, as described
in the RFC.
5. Commits the updated files and opens a pull request with the changes.
Initially, only a subset of Test262 tests related to Interop 2026
features (`Temporal` and `top-level-await`) are being imported. This is
a temporary measure to manage the volume of new test results and to
avoid overloading wpt.fyi, which currently has limitations in processing
the full Test262 result set (see wpt-fyi/issues/4681). The workflow can
be expanded to include more tests in the future.
This work directly supports the integration of Test262 into WPT as detailed
in the RFC: web-platform-tests/rfcs#229
This commit is part of a series of smaller PRs split from the larger,
original implementation in #55997.
jcscottiii
force-pushed
the
feat/test262-automation
branch
from
e8ec10a to
b3adcf7
Compare
This was referenced Dec 18, 2025
DanielRyanSmith
approved these changes
Dec 18, 2025
Comment on lines
+23
to
+60
| run: | | ||
| LATEST_SHA=$(git -C test262-spec rev-parse HEAD) | ||
| echo "Latest remote Test262 SHA: $LATEST_SHA" | ||
|
|
||
| rm -rf wpt/third_party/test262/test/ | ||
| mkdir -p wpt/third_party/test262/test/ | ||
| mkdir -p wpt/third_party/test262/harness/ | ||
|
|
||
| # Selectively copy only Interop 2026 feature tests for now | ||
| # See: https://github.com/web-platform-tests/wpt.fyi/issues/4681 | ||
| # Mapping of web feature to test directory can be found at: https://github.com/tc39/test262/blob/main/WEB_FEATURES.yml | ||
| # Temporal tests | ||
| mkdir -p wpt/third_party/test262/test/built-ins/Date/prototype/toTemporalInstant | ||
| mkdir -p wpt/third_party/test262/test/built-ins/Temporal | ||
| mkdir -p wpt/third_party/test262/test/intl402/Temporal | ||
| cp -r test262-spec/test/built-ins/Date/prototype/toTemporalInstant/* wpt/third_party/test262/test/built-ins/Date/prototype/toTemporalInstant | ||
| cp -r test262-spec/test/built-ins/Temporal/* wpt/third_party/test262/test/built-ins/Temporal | ||
| cp -r test262-spec/test/intl402/Temporal/* wpt/third_party/test262/test/intl402/Temporal | ||
|
|
||
| # Top-level-await tests | ||
| mkdir -p wpt/third_party/test262/test/language/module-code/top-level-await | ||
| cp -r test262-spec/test/language/module-code/top-level-await/* wpt/third_party/test262/test/language/module-code/top-level-await | ||
|
|
||
| # Always sync the harness files | ||
| rsync -a --delete test262-spec/harness/ wpt/third_party/test262/harness/ | ||
| printf "[test262]\nsource = \"https://github.com/tc39/test262\"\nrev = \"${LATEST_SHA}\"\n" > wpt/third_party/test262/vendored.toml | ||
| - name: Commit changes | ||
| id: commit | ||
| continue-on-error: true | ||
| run: | | ||
| cd wpt | ||
| export BRANCH_NAME="$BRANCH_PREFIX-$(date +'%Y%m%d%H%M%S')" | ||
| echo "BRANCH_NAME=$BRANCH_NAME" >> $GITHUB_ENV | ||
| git config user.name "$GIT_AUTHOR_NAME" | ||
| git config user.email "$GIT_AUTHOR_EMAIL" | ||
| git checkout -B $BRANCH_NAME | ||
| git add third_party/test262/ | ||
| git commit -m "$COMMIT_TITLE" |
Contributor
There was a problem hiding this comment.
Optional nit: The scripts are somewhat long and may be subject to change. Would it be better to move some of this to a separate file?
Contributor
DanielRyanSmith
left a comment
DanielRyanSmith
left a comment
There was a problem hiding this comment.
This suggestion came from a Gemini code review:
Focus: The "Selectively copies" step.
Critique:
The work is described as "selectively copying" files from Test262 to WPT.
- The Risk: If a test file is deleted or renamed in the upstream
tc39/test262repository, a simple copy operation (likecp -rorcheckout-index) will not remove the old file from yourthird_party/test262directory. Over time, your vendored directory will accumulate "zombie tests"—files that no longer exist upstream but persist in WPT, potentially causing confusing failures or bloat. - Suggested Fix: You must use a sync method that handles deletions (e.g.,
rsync --deleteor explicitly deleting the target directory before copying). - Constraint: Ensure you do not accidentally delete WPT-specific files (like
README.wpt,DIR_METADATA, orOWNERS) that might live in that directory.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit introduces a GitHub Actions workflow to automate the vendoring of the Test262 test suite into
third_party/test262. This automation is the final piece of the integration, ensuring that the tests can be kept up-to-date with the upstreamtc39/test262repository.The workflow is configured to run weekly or can be triggered manually. It performs the following steps:
tc39/test262repositories.vendored.tomlfile at the root of the vendored directory, recording the specific commit SHA fromtc39/test262that the files were sourced from. This ensures traceability, as described in the RFC.Initially, only a subset of Test262 tests related to Interop 2026 features (
Temporalandtop-level-await) are being imported. This is a temporary measure to manage the volume of new test results and to avoid overloading wpt.fyi, which currently has limitations in processing the full Test262 result set (see wpt-fyi/issues/4681). The workflow can be expanded to include more tests in the future.This work directly supports the integration of Test262 into WPT as detailed in the RFC: web-platform-tests/rfcs#229
This commit is part of a series of smaller PRs split from the larger, original implementation in #55997.