Skip to content

ci: build CLI only when it changes, reuse prior release otherwise#36081

Open
wezell wants to merge 8 commits into
mainfrom
issue-36080-cli-build-on-change
Open

ci: build CLI only when it changes, reuse prior release otherwise#36081
wezell wants to merge 8 commits into
mainfrom
issue-36080-cli-build-on-change

Conversation

@wezell

@wezell wezell commented Jun 9, 2026

Copy link
Copy Markdown
Member

What

Performance Optimization: Build/release the dotCMS CLI only when it actually changes, and reuse the prior release otherwise.
Trying to make our automated tests and release actions as fast as possible.

Fixes #36080

Why

The CLI is rebuilt and re-released on every trunk push and every dotCMS release, even when tools/dotcms-cli/** is byte-identical to the prior release. The native build runs an expensive 3-OS GraalVM matrix (including two macOS runners), so we repeatedly pay that cost to ship identical binaries.

The CLI is self-contained under tools/dotcms-cli/** (its only com.dotcms dependency, dotcms-api-data-model, lives in that same subtree), so a path/diff change-key is correct.

Changes

Trunk — cicd_3-trunk.yml

  • build-cli now also requires fromJSON(needs.initialize.outputs.filters).cli == 'true'.
  • deployment passes deploy-cli gated on the same predicate, so it never tries to deploy artifacts that were not built.

Release — cicd_release-cli.yml

  • New detection in precheck: git diff <prev-release-tag>..<this-tag> -- tools/dotcms-cli/ sets CLI_CHANGED. Fail-safe: any uncertainty (manual dispatch, no previous tag) → CLI_CHANGED=true (full build), so a wrong "unchanged" verdict can never ship stale binaries.
  • build, build-cli (native), and release (JReleaser) are gated on CLI_CHANGED == 'true'.
  • Unchanged path republishes the prior release's binaries under the new version across all three channels JReleaser/the pipeline feed:
    • npm (publish-npm-package): repackage the published latest tarball under the new version.
    • GitHub release + Artifactory (new reuse-release job): download dotcms-cli-<prev> assets, re-stamp to the new version, create the dotcms-cli-<new> release, and upload to libs-release-local/com/dotcms/dotcms-cli/<new>/.
  • Removed the Slack announcement from the CLI release cycle.

Notes / caveats

  • The reuse-release job mirrors the channels in tools/dotcms-cli/jreleaser.yml in bash — keep the two in sync if platforms or the Artifactory path change (noted in-code).
  • On the reuse path, JReleaser-generated checksums/signatures are not reproduced (only the .zip/.jar artifacts), and the reused binaries internally report the prior version (inherent to reusing them).
  • These are workflow-logic changes not exercised locally. The reuse-release and npm-reuse steps publish irreversibly and have no dry-run guard; recommend a workflow_dispatch rehearsal before relying on the unchanged path in a real release (can add a dry-run short-circuit if wanted).



Trunk (cicd_3-trunk.yml): gate the native CLI build on the `cli` path
filter and gate deploy-cli on the same predicate, so the 3-OS GraalVM
matrix (incl. two macOS runners) is skipped when tools/dotcms-cli/** is
unchanged.

Release (cicd_release-cli.yml): detect whether the CLI changed since the
previous release tag (fail-safe: build when uncertain) and, when
unchanged, skip the native build + JReleaser and republish the prior
release's binaries under the new version across all channels — npm, the
dotcms-cli-<version> GitHub release, and Artifactory. Remove the Slack
announcement from the CLI release cycle.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude

claude Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Claude finished @wezell's task in 4m 50s —— View job


PR Review: ci: build CLI only when it changes, reuse prior release otherwise

  • Gathering context
  • Reviewing changed files
  • Final assessment

Overall the approach is sound and the fail-safe (unknown/manual → full build) is correct. Several issues are worth a second look before this lands.


Critical / High

1. Divergent "previous version" between channels (npm vs. GitHub/Artifactory)

cicd_release-cli.yml lines 522–525: the npm reuse path calls npm view "${PACKAGE_FULL_NAME}" version to find what to repackage, which returns the latest npm tag. PREV_RELEASE_VERSION (lines 133–136) is derived from git tag semver order. These two can disagree:

  • If the previous release published under the rc npm tag (SNAPSHOT version), latest still points to an older release, so npm would ship binaries from release N-2 while GitHub Release + Artifactory ship from N-1.
  • If npm publishing failed in a prior run, same drift occurs.

The reuse path should use PREV_RELEASE_VERSION for npm too (e.g., derive the normalized npm version from the same git tag), not fall back to whatever npm happens to consider latest.

Fix this →


2. No guard against empty PREV_VERSION in reuse-release

cicd_release-cli.yml line 281: gh release download "dotcms-cli-${PREV_VERSION}" — if PREV_VERSION is empty for any unexpected reason, this executes as gh release download "dotcms-cli-" and produces a confusing API error rather than a clear fail. While the logic guarantees it's populated when CLI_CHANGED=false, the guard costs nothing:

if [[ -z "${PREV_VERSION}" ]]; then
  echo "::error::PREV_RELEASE_VERSION is empty; cannot reuse release"
  exit 1
fi

Fix this →


Medium

3. Unquoted GitHub context injection in precheck

cicd_release-cli.yml line 85:

TAG_NAME=${{ github.event.release.tag_name }}

Unquoted inline expression in a run: block. A tag name with spaces, $(), or ; would execute arbitrary shell. Should be:

TAG_NAME="${{ github.event.release.tag_name }}"

Same applies line 88 (RELEASE_VERSION=${{ github.event.inputs.version }}) and line 161 (HEAD=${{ github.ref_name }}). These are low-exploitation-risk but violate the principle of never injecting context values without quoting.

Fix this →


4. publish-npm-package doesn't block if build fails on the changed path

If build fails: build-cli's !failure() sees the failure and skips. release has needs: [precheck, build-cli]; since build-cli is skipped (not failed), release may attempt to run and fail downloading artifacts that don't exist. Whether failure() transitively sees build's failure through a skipped build-cli is GH Actions version-dependent behavior. Even if release then fails, publish-npm-package's condition only checks release.result in {success, skipped} — a failed release (result=failure) would block npm, so the end state is safe. But the confusing intermediate failure on release could be avoided by adding build to release's needs explicitly, or adding (needs.build.result == 'success' || needs.build.result == 'skipped') to the publish-npm-package condition.


Low / Acknowledged

5. Reused binaries report prior version internally — acknowledged in PR notes. Harmless for most users but dotcms --version would show the old version. Acceptable given the tradeoff, but worth documenting in user-facing release notes.

6. No checksums/signatures on reuse path — acknowledged. Consumers relying on JReleaser-generated checksums will fail verification silently. Low risk for this project's distribution model but worth noting in the GitHub release body (it currently says "Reused binaries…" which is fine).

7. git tag --sort=-version:refname — good fix from the prior commit; semver sort is correct here.


…tion time

Replaces --sort=-creatordate with --sort=-version:refname so PREV_TAG is
determined by semantic version order rather than tag creation timestamp,
which can be misleading when tags are backfilled or re-created.
Extend the existing `dry-run` workflow_dispatch input to cover every
publish/upload action so the unchanged-CLI reuse path can be rehearsed
end-to-end with zero side effects:

- reuse-release: skip `gh release create` and the Artifactory upload,
  logging what would happen instead.
- publish-npm-package: pass `--dry-run` to `npm publish` on both the
  changed path and the reuse path.

JReleaser already honored JRELEASER_DRY_RUN. Updated the input
description to reflect that dry-run now skips npm, GitHub release, and
Artifactory as well.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The CLI integration tests (tools/dotcms-cli, @QuarkusTest + testcontainers
against a live dotCMS) were gated on `cli || backend`, so they ran on nearly
every backend PR — the broad `backend` filter matches almost all of dotCMS/**.

Add a `cli_test` filter = CLI sources (`cli`) + the server REST endpoints the
CLI's @path clients actually consume (assets, folder, site, contenttype,
languages v2, workflow, authentication, user, analytics event) plus the
asset-specific business APIs WebAssetHelper delegates to (fileassets, browser).
Keep `cli` (= tools/dotcms-cli/**) untouched so endpoint-only changes don't
re-trigger the GraalVM native build added in #36081.

- PR + merge-queue now gate CLI tests on `cli_test` instead of `cli || backend`.
- Register `cli_test` in the initialize-phase rewrite (build_test_filters +
  test_filters) so it defaults correctly under change-detection=disabled and
  honors CICD_SKIP_TESTS.
- Safety net: trunk (which previously ran NO CLI tests) now runs them for any
  `backend` change, so a deeper business-layer regression that slips past the
  narrowed PR filter is still caught on main before a release. Nightly continues
  to run them unconditionally.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…36080

- TempFileAPI (rest/api/v1/temp) backs CLI file uploads on the asset push
  path via WebAssetHelper; same rationale as the fileassets/browser deps.
- LTS workflow now gates CLI tests on `cli_test` (CLI sources + consumed
  endpoints) instead of bare `cli`, matching PR/merge-queue behavior so
  endpoint changes on an LTS branch still exercise the CLI suite.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…36080

Address PR review findings:

- publish-npm-package now depends on reuse-release. On the unchanged path
  (CLI_CHANGED=false) reuse-release publishes the GitHub release + Artifactory
  while publish-npm-package republishes to npm; previously they ran in parallel,
  so a reuse-release failure could leave npm published with no GitHub release.
  Adding it to `needs` makes !failure() block npm publish if reuse-release fails;
  on the changed path reuse-release is skipped (not failed) so the job still runs.
- Document, at the PR and merge-queue CLI-test gates, why the trigger is narrow
  (cli_test) and that the trunk CLI run is the compensating control for deep
  backend changes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The clean-up job had no always() guard, so GitHub Actions skipped it whenever
any upstream job failed, leaking the temporary version-update-<ver>-<run_id>
branch. Guard with always() (AUXILIARY_BRANCH is only set when precheck
succeeded, so it still no-ops when there's nothing to clean up). Addresses PR
review follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…hecks

Replace the implicit !failure() gate with explicit
(needs.reuse-release.result == 'success' || 'skipped') &&
(needs.release.result == 'success' || 'skipped') checks so the
mutual-exclusion of the changed/unchanged paths is self-documenting
and a failure in either upstream job reliably blocks the npm publish.

https://claude.ai/code/session_018KKHBasNtegpRZy5G6cCD1

@sfreudenthaler sfreudenthaler left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Acknowledged — won't fix (re: npm view "${PACKAGE_FULL_NAME}" version in the "Reuse previous CLI release and publish" step)

The feedback notes that npm view ... version returns the latest tag version rather than a version derived from PREV_RELEASE_VERSION, and suggests deriving it from the git-based output instead.

This was a deliberate choice, documented in the inline comment above that step:

"We read the source version from npm ('latest') rather than reconstructing it from the git tag, sidestepping the leading-zero normalization applied to release versions above."

Deriving from PREV_RELEASE_VERSION would require replicating the same leading-zero stripping logic from the "Dynamic configuration" step — adding complexity with no practical benefit. This workflow is the sole publisher to the latest npm tag (LTS releases are explicitly excluded, and no other code path publishes there), so the two values are guaranteed to agree under normal operation. The only divergence scenario is out-of-band manual intervention on npm, which is outside the scope of this workflow to defend against.


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI: Safe To Rollback Area : CI/CD PR changes GitHub Actions/workflows Team : Scout

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

CI: only build the CLI when it changes; reuse prior release otherwise

3 participants