perf: switch app rootfs from ext4 to erofs with LZ4 compression#94
Merged
hiroTamada merged 5 commits intomainfrom Feb 13, 2026
Merged
perf: switch app rootfs from ext4 to erofs with LZ4 compression#94hiroTamada merged 5 commits intomainfrom
hiroTamada merged 5 commits intomainfrom
Conversation
App rootfs disks are read-only in the VM (mounted as the lower layer of an overlayfs), so a read-only filesystem is a better fit. erofs with LZ4 compression produces 60% smaller disk images (77MB vs 193MB for node:20-alpine) and is faster to create than ext4. Changes: - Set DefaultImageFormat to FormatErofs (disk.go) - Auto-detect filesystem type when mounting /dev/vda so both erofs and ext4 images work (mount.go) - Add unit test comparing ext4 vs erofs export (disk_test.go) Verified: VM kernel (ch-6.12.8-kernel-1.4) has CONFIG_EROFS_FS=y built-in. E2E tested: build → erofs conversion → VM boot → app execution all working. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- convertToErofs falls back to ext4 if mkfs.erofs is not installed - buildImage now logs errors to stderr (was only storing in metadata) - waitForImageReady propagates the actual error from image metadata instead of generic "image conversion failed" Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| // Check if mkfs.erofs is available | ||
| if _, err := exec.LookPath("mkfs.erofs"); err != nil { | ||
| fmt.Fprintf(os.Stderr, "Warning: mkfs.erofs not found, falling back to ext4\n") | ||
| return convertToExt4(rootfsDir, diskPath) |
There was a problem hiding this comment.
Silent erofs-to-ext4 fallback masks missing tool in production
Medium Severity
convertToErofs silently falls back to convertToExt4 when mkfs.erofs is missing, writing only a warning to stderr. Since DefaultImageFormat was switched to FormatErofs in this same commit, a production host without mkfs.erofs will silently produce ext4 images while callers believe they created erofs images. The test TestExportRootfsFormats would also silently pass by testing ext4 twice, never actually validating erofs output.
Additional Locations (1)
Create the erofs disk inside the builder VM after the BuildKit push completes, eliminating the slow host-side umoci unpack + mkfs.erofs conversion pipeline. After buildctl pushes the image to the registry, the builder agent: 1. Pulls the manifest and layer blobs from the local registry 2. Extracts layers into a tmpfs (handles gzip/zstd decompression) 3. Processes OCI whiteout files 4. Runs mkfs.erofs with LZ4 compression 5. Writes the erofs file to the source volume The host then mounts the source volume, copies the erofs disk, and registers it directly as a ready image via RegisterPrebuiltImage. This skips waitForImageReady entirely. Every step has a graceful fallback - if any part fails, the existing push → unpack → convert pipeline is used instead. E2E benchmark (node:20-alpine test app, warm caches): - Baseline: ~11,020ms total (~468ms post-build conversion) - Optimized: ~8,341ms total (conversion bypassed) Also adds erofs-utils and zstd to the builder Dockerfile, resizes source volumes to 512MB minimum for erofs workspace, and includes an e2e benchmark script. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Process OCI whiteout files after each layer extraction instead of in a single pass after all layers. The previous approach could incorrectly delete files added by later layers when an earlier layer contained whiteouts targeting the same paths. Add 4096-byte sector alignment to prebuilt erofs disks in RegisterPrebuiltImage, matching the alignment applied by convertToErofs in the normal pipeline. Required by hypervisors for block device I/O. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.
Reject absolute paths and .. components in the erofs disk path received from the builder VM via vsock before using it in filepath.Join. Defense in depth — the VM is the security boundary, but untrusted input from it should still be validated on the host side. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
sjmiller609
approved these changes
Feb 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
/dev/vdain the VM init so both erofs (new default) and ext4 (legacy) images boot correctlyDetails
App rootfs disks are read-only in the VM — they're mounted as the lower layer of an overlayfs. A read-only compressed filesystem (erofs) is a natural fit:
The VM kernel (
ch-6.12.8-kernel-1.4) hasCONFIG_EROFS_FS=ybuilt-in with full compression support (LZ4, ZIP, LZMA).Files changed
lib/images/disk.go—DefaultImageFormatchanged fromFormatExt4toFormatErofslib/system/init/mount.go— omit-tflag when mounting/dev/vdaso kernel auto-detects fs type; backwards compatible with existing ext4 imageslib/images/disk_test.go— new test exercising both formatsTest plan
TestExportRootfsFormats(ext4 + erofs both produce valid, sector-aligned output)erofs test OK)erofs: (device vda): mounted with root inode @ nid 37.CONFIG_EROFS_FS=yinch_defconfig🤖 Generated with Claude Code
Note
Medium Risk
Changes the default on-disk rootfs format and introduces a new image registration path that bypasses the existing conversion pipeline, so boot/mount compatibility and registry/layer-extraction edge cases could impact build readiness or runtime.
Overview
Switches the default OCI rootfs export format from
ext4toerofswith LZ4 compression, adding a fallback toext4whenmkfs.erofsis unavailable and updating VM init mounting to auto-detect the filesystem type for backward compatibility.Adds a prebuilt-
erofsfast path for source builds: the builder agent pulls the freshly pushed image from the internal registry, extracts layers (handling whiteouts), runsmkfs.erofs, and returns the disk path in the build result; the host then extracts that disk from the source volume and registers it as a ready image via a newimages.Manager.RegisterPrebuiltImage, skipping the existing push→unpack→convert pipeline when successful.Includes supporting changes: builder image now installs
erofs-utils/zstd, Docker builder image builds use--network=host, improved error reporting when image conversion fails, a new unit test coveringext4vserofsexport/sector alignment, and an added E2E benchmark script for measuring the optimization.Written by Cursor Bugbot for commit b0bc0ba. This will update automatically on new commits. Configure here.