fix(deploy): build binaries on the runner to stop droplet OOM (exit 255)#27
Merged
Conversation
…n box The deploy SSHed into the ~458MB droplet and ran `go build` there. The Go linker's peak memory OOM-killed the build — and with it the sshd serving the deploy session — surfacing as "Connection closed by remote host" (exit 255). It was flaky because it tracked momentary memory pressure from the co-resident ergo/forgejo/tor/podman/agentbbs processes (run #25 passed, #26 failed on near-identical code). Build both binaries on the 16GB GitHub runner instead (pure-Go, modernc sqlite, so CGO_ENABLED=0 static cross-build), scp them to the droplet, and run setup.sh with SKIP_BUILD=1 so the box never compiles. Arch is detected from the droplet so amd64/arm64 both work. setup.sh now also skips the Go toolchain download when SKIP_BUILD=1. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
vu1nz Security Review0 finding(s) in PR #? No security issues found. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The deploy run for #26 failed at the build step with:
exit 255+ "Connection closed by remote host" is an SSH-level disconnect: the deploy SSHes into the ~458MB droplet and runsgo buildthere, but the Go linker's peak memory OOM-kills the build — and the kernel takes out the sshd serving the deploy session with it. It's flaky because it tracks momentary memory pressure from the co-resident ergo / forgejo / tor / podman / live-agentbbs processes (run #25 passed, #26 failed 6 min later on near-identical code).setup.shalready anticipated this (SKIP_BUILDenv, "tiny droplets can't compile") but the deploy never used it.Fix
modernc.org/sqlite, no cgo), soCGO_ENABLED=0produces a static, portable binary.uname -m→amd64/arm64) so the cross-build matches.setup.shwithSKIP_BUILD=1— the box never compiles, removing the OOM failure mode entirely.setup.shnow also skips the Go toolchain download whenSKIP_BUILD=1.setup.shstill does itsgit reset --hard origin/<branch>so config/assets/service definitions stay in sync; only the compile step is offloaded.Notes / follow-up
import "C"); the actual cross-build runs on the runner viaactions/setup-go+go-version-file: go.mod(1.26.x). The sandbox here only has Go 1.22, so the compile itself is validated by this workflow run.setup.shwith the defaultSKIP_BUILD=0and would compile on-box. Out of scope here; can be switched to a "fetch prebuilt binary from CI artifacts" model in a follow-up if its on-box builds also prove flaky.🤖 Generated with Claude Code