Skip to content

Replace test-collector-swift with JUnit XML upload#25243

Closed
mokagio wants to merge 15 commits intotrunkfrom
mokagio/test-engine-junit-xml-upload
Closed

Replace test-collector-swift with JUnit XML upload#25243
mokagio wants to merge 15 commits intotrunkfrom
mokagio/test-engine-junit-xml-upload

Conversation

@mokagio
Copy link
Contributor

@mokagio mokagio commented Feb 11, 2026

Important

Closed in favor of paNNhX-14c-p2

  • Remove the test-collector-swift SPM package, which hooks into XCTest at runtime and doesn't capture Swift Testing results
  • Add the test-collector-buildkite-plugin v1.11.0 to the pipeline steps for unit tests, UI tests (iPhone), and UI tests (iPad)
  • The plugin uploads JUnit XML files (already generated by trainer from .xcresult bundles) to Buildkite Test Engine in a post-command hook

Test plan

  • CI passes on this PR
  • Buildkite Test Engine receives test results for unit tests, iPhone UI tests, and iPad UI tests
  • Swift Testing test names appear in Test Engine results

🤖 Generated with Claude Code

mokagio and others added 14 commits February 11, 2026 08:28
CI uses Fastlane exclusively; these local wrappers around
`xcodebuild` were obsolete and hardcoded a stale simulator.

---

Generated with the help of Claude Code, https://code.claude.com

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Lets agents run lint, build, test, and simulator
queries without prompting. Other A8C repos (e.g.
woocommerce.com) use the same pattern.

---

Generated with the help of Claude Code, https://code.claude.com

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Skips CI prerequisites (env files, signing, toolkit checks) and
reuses DerivedData for incremental builds. Supports only_testing,
scheme, device, and clean options.

---

Generate with the help of Claude Code, https://code.claude.com

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Ideally, this would build faster than the rest and allow focused
testing.

The blocker is that since the Modules don't build with `swift build` we
can test with `swift test`. Plus, auto-generated schemes allow building
but not testing!
---

Generate with the help of Claude Code, https://code.claude.com

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---

Generate with the help of Claude Code, https://code.claude.com

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---

Generate with the help of Claude Code, https://code.claude.com

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---

Generate with the help of Claude Code, https://code.claude.com

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The file was in a `WordPressUITests` directory with no matching
SPM target, so it was never compiled or run.
Updated its import from `WordPress` to `WordPressUI`.

---

Generate with the help of Claude Code, https://code.claude.com

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---

Generate with the help of Claude Code, https://code.claude.com

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Applies the same test-bundle workaround already used elsewhere
in WordPressUI so the storyboard loads in SPM test context.

---

Generate with the help of Claude Code, https://code.claude.com

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Test Engine results will be uploaded via the
test-collector Buildkite plugin instead of the
in-process SPM library, which doesn't capture
Swift Testing results.

---

Generated with the help of Claude Code, https://code.claude.com

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Upload JUnit XML test results to Test Engine via
the test-collector plugin, which runs as a
post-command hook and picks up results from
trainer's XML output.

---

Generated with the help of Claude Code, https://code.claude.com

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@dangermattic
Copy link
Collaborator

dangermattic commented Feb 11, 2026

1 Error
🚫 PR requires at least one label.
1 Warning
⚠️ PR is not assigned to a milestone.

Generated by 🚫 Danger

@wpmobilebot
Copy link
Contributor

wpmobilebot commented Feb 11, 2026

App Icon📲 You can test the changes from this Pull Request in WordPress by scanning the QR code below to install the corresponding build.
App NameWordPress
ConfigurationRelease-Alpha
Build Number30878
VersionPR #25243
Bundle IDorg.wordpress.alpha
Commit41d4353
Installation URL3u824ijl7bg28
Automatticians: You can use our internal self-serve MC tool to give yourself access to those builds if needed.

@wpmobilebot
Copy link
Contributor

wpmobilebot commented Feb 11, 2026

App Icon📲 You can test the changes from this Pull Request in Jetpack by scanning the QR code below to install the corresponding build.
App NameJetpack
ConfigurationRelease-Alpha
Build Number30878
VersionPR #25243
Bundle IDcom.jetpack.alpha
Commit41d4353
Installation URL217jd316lt00g
Automatticians: You can use our internal self-serve MC tool to give yourself access to those builds if needed.

---

Generated with the help of Claude Code, https://code.claude.com

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@sonarqubecloud
Copy link

@wpmobilebot
Copy link
Contributor

🤖 Build Failure Analysis

This build has failures. Claude has analyzed them - check the build annotations for details.

@mokagio
Copy link
Contributor Author

mokagio commented Feb 11, 2026

CI Status Notes

UI Tests (iPhone/iPad): Failing, but these are also failing on trunk across recent commits — pre-existing issue, not caused by this PR.

Unit Tests: Failing. I can't access Buildkite logs to check which specific tests failed. This might be flaky — unit tests pass on trunk. Worth retrying the job to confirm. If it persists, the most likely cause is the change to RemoteTestCase.stubAllNetworkRequestsWithNotConnectedError(), which now stubs all requests unconditionally instead of excluding the Buildkite Analytics API host.

Danger: Failing because the PR needs a label and milestone — cosmetic.

Everything else passes: builds, linters, SonarCloud, Reader tests, WordPressData tests.

@mokagio mokagio marked this pull request as draft February 11, 2026 20:28
@mokagio mokagio closed this Feb 24, 2026
@mokagio mokagio deleted the mokagio/test-engine-junit-xml-upload branch February 24, 2026 02:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants