Skip to content

Build for Linux in Buildkite + EC2 + Docker#3346

Open
twstokes wants to merge 1 commit intoadd-buildkite-linux-build-groupfrom
twstokes/add-buildkite-linux-build-group
Open

Build for Linux in Buildkite + EC2 + Docker#3346
twstokes wants to merge 1 commit intoadd-buildkite-linux-build-groupfrom
twstokes/add-buildkite-linux-build-group

Conversation

@twstokes
Copy link
Copy Markdown
Contributor

@twstokes twstokes commented May 5, 2026

AINFRA-2352

Related issues

How AI was used in this PR

AI-assisted throughout. Claude inspected the existing .buildkite/pipeline.yml, surveyed the buildkite-ci Terraform repo to map available queues, and cross-referenced pipelines in other projects to identify the right pattern. A self-review pass added the Linux ECONNRESET workaround, dynamic Node image derivation from .nvmrc, and the shell/apt hardening below.

Proposed Changes

Adds a `📦 Build for Linux` group to `.buildkite/pipeline.yml`, mirroring the existing Mac and Windows groups: input gate (skipped on `trunk` and tagged release builds), group dependency on the input, and a 2-arch matrix (`x64`, `arm64`) that runs `npm run make:linux-{{matrix}}` and uploads the resulting `.deb`.

  • Build runs inside a Node Docker container. Amazon Linux is RPM-based; `dpkg` isn't in its default repos and `fakeroot` is only available via EPEL. Both are hard prerequisites of `electron-forge`'s `MakerDeb`. Wrapping the step in a Debian container — using the Buildkite Docker plugin, an established A8C pattern — sidesteps the AMI mismatch cleanly.

  • Image is derived from `.nvmrc`. `shared-pipeline-vars` reads `.nvmrc` and exports `NODE_DOCKER_IMAGE="node:${VERSION}-bookworm"`, which `pipeline.yml` references as `$NODE_DOCKER_IMAGE`. Bumping `.nvmrc` moves the Linux container in lockstep with the NVM-installed Node on Mac/Windows agents, so `$NVM_PLUGIN` isn't needed on this step.

  • `git remote set-url origin https://...` inside the container. The host buildkite-agent clones over SSH and `prepare-dev-build-version.mjs` calls `git fetch --tags --force`. The container has no `~/.ssh/known_hosts` or deploy key, so SSH hangs on the host-key prompt. Studio is a public repo, so swapping to anonymous HTTPS for the in-container fetch is the simplest fix and avoids exposing the agent's deploy key inside the container.

  • Inline shell hardening. `set -euo pipefail` at the top of the command; `apt-get -o Acquire::Retries=3 update` to ride out flaky Debian mirrors; `npm ci --maxsockets 1` to avoid the [BUG] Getting constant Client network socket disconnected before secure TLS connection was established npm/cli#4652 ECONNRESETs that `install-node-dependencies.sh` already documents on Linux.

  • Adds `DOCKER_PLUGIN="docker#5.13.0"` and `NODE_DOCKER_IMAGE` to `.buildkite/shared-pipeline-vars`.

Known follow-up

The Linux step currently does not restore the Buildkite npm cache that `install-node-dependencies.sh` provides on Mac/Windows. The toolkit's cache helpers run on the host, so wiring them around a Docker step needs either two pipeline steps (host install → container build) or a persistent host volume mounted into the container — both larger than the scope here. Tracked as a TODO in `pipeline.yml`.

Testing Instructions

  1. Open this PR (or rebuild on Buildkite if a build already exists).
  2. Confirm the `🚦 Build for Linux?` input step appears alongside the existing Mac/Windows input gates.
  3. Click "yes" — the matrix should fan out to two jobs: `🔨 Linux Dev Build - x64` and `🔨 Linux Dev Build - arm64`.
  4. Both should pick up an agent on the `default` queue, pull the `node:<.nvmrc>-bookworm` image, install `fakeroot`, run `npm ci`, and produce a `.deb` artifact:
    • `apps/studio/out/make/deb/x64/studio__amd64.deb`
    • `apps/studio/out/make/deb/arm64/studio__arm64.deb`
  5. Download each artifact and run `dpkg-deb -I ` to confirm metadata, and `dpkg-deb -x /tmp/extracted && file /tmp/extracted/opt/Studio/studio` to confirm the binary architecture matches.

Note: `Distribute Dev Builds` (`.buildkite/pipeline.yml:205`) is intentionally not yet wired to depend on the Linux group — Linux distribution via Fastlane is out of scope for this PR. The `.deb` artifacts are reachable directly from the Buildkite build page.

Pre-merge Checklist

  • Have you checked for TypeScript, React or other console errors?
  • Linux x64 dev build succeeds end-to-end on Buildkite.
  • Linux arm64 dev build succeeds end-to-end on Buildkite.
  • Resulting `.deb` files install cleanly on a target Debian/Ubuntu system (smoke test).

@twstokes twstokes changed the base branch from trunk to add-buildkite-linux-build-group May 5, 2026 17:04
@twstokes twstokes force-pushed the twstokes/add-buildkite-linux-build-group branch from 5616e21 to f759f38 Compare May 5, 2026 17:18
@twstokes twstokes changed the title Twstokes/add buildkite linux build group Run Linux builds in Buildkite + EC2 + Docker May 5, 2026
@twstokes twstokes changed the title Run Linux builds in Buildkite + EC2 + Docker Build for Linux in Buildkite + EC2 + Docker May 5, 2026
@twstokes
Copy link
Copy Markdown
Contributor Author

twstokes commented May 5, 2026

Tested on an ARM64 Debian VM:

Screenshot 2026-05-05 at 13 30 08

@twstokes twstokes force-pushed the twstokes/add-buildkite-linux-build-group branch 2 times, most recently from 7003adc to 1c0ef93 Compare May 5, 2026 18:46
@twstokes twstokes force-pushed the twstokes/add-buildkite-linux-build-group branch from 1c0ef93 to 792f331 Compare May 5, 2026 18:49
@twstokes twstokes requested review from iangmaia and mokagio May 5, 2026 19:00
@twstokes twstokes marked this pull request as ready for review May 5, 2026 19:00
Comment thread .buildkite/pipeline.yml
Comment on lines +176 to +181
# `--maxsockets 1` works around npm/cli#4652 (ECONNRESETs on Linux);
# see install-node-dependencies.sh for the original rationale.
# TODO: restore the Buildkite npm cache used on Mac/Windows. The
# toolkit's cache helpers live on the host, so wiring them around a
# Docker step needs extra plumbing or a persistent agent volume.
npm ci --unsafe-perm --no-audit --no-progress --maxsockets 1
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, we could add some platform introspection in the install script, to opt-out of the caching flow. Just to keep the npm ci options usage consistent.

@twstokes twstokes requested a review from ivan-ottinger May 6, 2026 00:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants