Skip to content

Live bootstrap to guix#578

Draft
vxtls wants to merge 274 commits into
fosslinux:masterfrom
vxtls:live-bootstrap-to-guix
Draft

Live bootstrap to guix#578
vxtls wants to merge 274 commits into
fosslinux:masterfrom
vxtls:live-bootstrap-to-guix

Conversation

@vxtls
Copy link
Copy Markdown

@vxtls vxtls commented Mar 15, 2026

No description provided.

vxtls added 30 commits February 22, 2026 17:12
feat(steps-guix): add libgcrypt-1.12.1 default build with gcc-detected host and pkg-config path
feat(steps-guix): add guile-gcrypt-0.5.0 with dynamic libgcrypt prefix and ld library path
@Googulator
Copy link
Copy Markdown
Collaborator

Googulator commented Apr 19, 2026

Currently running a bare metal test with this diff on top of the latest commit:

$ git diff
diff --git a/steps-guix/improve/guix-daemon-and-pull.sh b/steps-guix/improve/guix-daemon-and-pull.sh
index c7625474..f61b5d9f 100644
--- a/steps-guix/improve/guix-daemon-and-pull.sh
+++ b/steps-guix/improve/guix-daemon-and-pull.sh
@@ -218,6 +218,9 @@ mkdir -p /proc /sys /dev "${guix_localstate_dir}/daemon-socket" /var/lib/guix /r
 mount | grep ' on /proc ' >/dev/null 2>&1 || mount -t proc proc /proc
 mount | grep ' on /sys ' >/dev/null 2>&1 || mount -t sysfs sysfs /sys
 mount | grep ' on /dev ' >/dev/null 2>&1 || mount -t devtmpfs devtmpfs /dev
+# tmpfs must be unmounted to avoid overfilling memory
+mount | grep ' on /tmp ' >/dev/null 2>&1 && umount /tmp
+test -f /swapfile && swapon /swapfile
 if ! mount | grep ' on /dev/pts ' >/dev/null 2>&1; then
     mkdir -p /dev/pts
     mount -t devpts devpts /dev/pts
diff --git a/steps-guix/jump/linux64.sh b/steps-guix/jump/linux64.sh
index 1bb6ba93..f942c2a1 100644
--- a/steps-guix/jump/linux64.sh
+++ b/steps-guix/jump/linux64.sh
@@ -37,4 +37,8 @@ else
         --append="console=ttyS0 root=/dev/sda1 init=/init rw rootwait consoleblank=0"
 fi
 quiesce_filesystem_for_kexec
-kexec -e
+if [ "${BARE_METAL}" = True ]; then
+    echo b > /proc/sysrq-trigger || true
+else
+    kexec -e
+fi

I disabled kexec for bare metal, because I couldn't get the framebuffer to work reliably after kexecing from 32-bit 4.14-openela to 64-bit 6.12-gnu. The other change activates the swapfile, and unmounts /tmp before starting the Guix bootstrap - this was needed to avoid going OOM on my 8GiB RAM bootstrap rig.

@vxtls
Copy link
Copy Markdown
Author

vxtls commented Apr 19, 2026

I’m also thinking about running a bare-metal setup. Is using Coreboot + SeaBIOS on a modern motherboard a viable option?

@Googulator
Copy link
Copy Markdown
Collaborator

Googulator commented Apr 19, 2026

If you can run Coreboot on your board, it should be, especially since qemu's default BIOS implementation is SeaBIOS.

Note that I had to edit that patch in that previous comment, as I got Bash's conditional syntax wrong (it's if...fi, not if...endif).

@vxtls
Copy link
Copy Markdown
Author

vxtls commented Apr 19, 2026

Great, I’ll add the patch shortly.
I’m currently experimenting with an offline ISO image (I’ll keep the default build using substitutes in the project), but I’ll include a configuration file that allows for building an image suitable for offline installation. I’m running into some issues at the moment.

@vxtls
Copy link
Copy Markdown
Author

vxtls commented Apr 20, 2026

There is currently no reliable method for including Closure in an ISO image and performing a guided installation.

@Googulator
Copy link
Copy Markdown
Collaborator

The new patch is malformed: @@ -211,11 +211,8 @@ should be @@ -211,11 +211,4 @@.

With that fixed, I had a test failure in wmin69qfszbjz0mflj2sya8sm2r5c7bs-glib-2.83.3.drv (GLib 2.83.3), that went away on retry. Maybe we should disable tests for it?

@Googulator
Copy link
Copy Markdown
Collaborator

OK, after lots of wrangling...

I have an ISO.

More to come tomorrow. I've just came home, and am quite tired. And it did took quite some wrangling to get here.

@Googulator
Copy link
Copy Markdown
Collaborator

So, finally got around to writing it all up.

  1. I encountered the system Guile leakage issue again. Adding . /var/guix/profiles/per-user/root/current-guix/etc/profile before calling guix system image fixed this, but exposed a new issue: environment variables on the host system point to 32-bit libraries, incompatible with the 64-bit Guile in the current Guix profile. To fix this, the guix system image needs to be appropriately prefixed: env -i PATH="$PATH" guix system image ...
  2. Since https://guix.gnu.org/en/blog/2025/privilege-escalation-vulnerabilities-2025/, Guix requires slirp4netns and a working /dev/net/tun to run custom fixed output derivations (i.e. download scripts). These aren't currently bootstrapped before building Guix. I have been able to build slirp4netns using Guix itself (i.e. running guix system image in a guix shell that includes the slirp4netns package), but simply creating /dev/net/tun using mknod proved insufficient - likely some kernel config change is needed. I suggest either using the kernel config from Guix's own kernel package (which is known to work well for this use case), or just reverting the 6 commits for the Guix daemon running in live-bootstrap (but not the actual Guix channel repo!). For now, I used --disable-chroot as a workaround for the affected packages - unfortunately this causes other packages to break, so I had to keep switching chroot on and off manually during the build.
  3. Guix tests would fail, so I disabled them. Looking through the build log, there are 3 test failures:
    3.1. tests/channels.scm:501 (latest-channel-instances, missing introduction for "guix") fails because of the hack to disable channel authentication. We should either disable / XFAIL this test, or add authentication information to the channel repository we create.
    3.2. tests/graph.scm:207 ("bad DAG") seems to depend on having real %bootstrap-inputs (that is, the original bootstrap binaries downloaded from Guix's servers) - disable or patch it.
    3.3. tests/guix-environment fails because it's testing if the 32-bit bootstrap-guile has a %host-type beginning with "i686". In a seemingly musl-specific issue, our bootstrap-guile ends up with a %host-type value of "x86_64-pc-linux-muslx32", as the build environment seemingly detects that the kernel is 64-bit under a 32-bit userland, and Guix then thinks this means it's inappropriately picking a 64-bit bootstrap-guile for a 32-bit build, and fails the test. Ideally, we should fix this in bootstrap-guile itself.

And then, some bad news:

  1. The generated ISO self-identifies as "Guix 1.5.0rc1", unlike the official ISO, which says "Guix 1.5.0" - this seems to be a Guix tarball packaging error or quirk; probably we are missing a few commits from the official Guix Git repository that actually bring the included Guix package up to the real version 1.5.0.
  2. The generated ISO takes a long time to boot (at least in a Gen 1 Hyper-V VM, where I tried it), and then fails the installation during the disk partitioning stage. I haven't tried a manual installation using guix system, only the semi-graphical installer.

@vxtls
Copy link
Copy Markdown
Author

vxtls commented May 4, 2026

Could you tell in detail why the disk partitioning failed? I'm trying to use a non-default ISO for the setup, specifically, a custom ISO that defines as few components as possible, and I'm adding the --no-substitutes option during the final installation (because by default, the Guix installation image downloads external binaries, whereas I want to achieve a truly source-only installation).

@Googulator
Copy link
Copy Markdown
Collaborator

dump.2026-04-28.10.12.04.tar.gz

This is the dump I was able to obtain from the failed installation. Looks like maybe the target device isn't being found.

With that said, I suspect the failure might be due to some packages being built with chroot off, resulting in contamination from the live-bootstrap host environment.

@Googulator
Copy link
Copy Markdown
Collaborator

I tried the Manual partitioning option in Setup, only to get an empty list of potential disks to partition.

@vxtls
Copy link
Copy Markdown
Author

vxtls commented May 4, 2026

what lsblk show in terminal?

@Googulator
Copy link
Copy Markdown
Collaborator

Looks like I was dumb - the mounted virtual disk had a malformed partition table on it, which confused Setup. A parted mklabel gpt later, guided partitioning succeeds.

@Googulator
Copy link
Copy Markdown
Collaborator

Googulator commented May 4, 2026

image

Looks like it's looking for the local channel repository inside the ISO. Cloning it should work, though.

EDIT: no git in the image, but scp -rq worked. Installation proceeding normally (using upstream substitutes this time, but I reckon it should work without them, as well).

EDIT 2: it's not actually using substitutes; while it did look for them, what it's actually downloading is all source code, because the bootstrap changes made the upstream substitutes not match. Using the L-B system as a substitute server remains a possibility.

@vxtls
Copy link
Copy Markdown
Author

vxtls commented May 7, 2026

Compiling Scheme modules...
make  check-TESTS check-local
make[3]: Entering directory '/tmp/guix-build-guix-1.5.0rc1.drv-0/source'
make[4]: Entering directory '/tmp/guix-build-guix-1.5.0rc1.drv-0/source'
PASS: tests/accounts.scm
PASS: tests/base16.scm
PASS: tests/base32.scm
PASS: tests/base64.scm
PASS: tests/boot-parameters.scm
PASS: tests/bournish.scm
SKIP: tests/builders.scm
SKIP: tests/build-emacs-utils.scm
FAIL: tests/build-utils.scm
PASS: tests/cache.scm
FAIL: tests/challenge.scm
SKIP: tests/channels.scm
PASS: tests/combinators.scm
SKIP: tests/containers.scm
SKIP: tests/cpio.scm
PASS: tests/cve.scm
SKIP: tests/debug-link.scm
make[4]: *** [Makefile:7302: tests/derivations.log] Error 1
make[4]: Leaving directory '/tmp/guix-build-guix-1.5.0rc1.drv-0/source'
make[3]: *** [Makefile:7285: check-TESTS] Error 2
make[3]: Leaving directory '/tmp/guix-build-guix-1.5.0rc1.drv-0/source'
make[2]: *** [Makefile:7533: check-am] Error 2
make[2]: Leaving directory '/tmp/guix-build-guix-1.5.0rc1.drv-0/source'
make[1]: *** [Makefile:7036: check-recursive] Error 1
make[1]: Leaving directory '/tmp/guix-build-guix-1.5.0rc1.drv-0/source'
make: *** [Makefile:7535: check] Error 2

Test suite failed, dumping logs.
error: in phase 'check': uncaught exception:
%exception #<&invoke-error program: "make" arguments: ("check") exit-status: 2 term-signal: #f stop-signal: #f>
phase `check' failed after 27.4 seconds
command "make" "check" failed with status 2
build process 12 exited with status 256
bash-5.3#

@Googulator
Copy link
Copy Markdown
Collaborator

Not sure if it was the exact same one, but I did occasionally see a strange failure in the check phase of guix-1.5.0rc1, that wasn't a test failure (which should move on to other tests), but something that instantly interrupted execution of the test suite. It's nondeterministic, and simply retrying helps.

@vxtls
Copy link
Copy Markdown
Author

vxtls commented May 7, 2026

It's nondeterministic, and simply retrying helps.

I suppose part of the concept of reproducible builds is consistency in behavior. I think that if we have time, we should look into the specific cause.

As for tests that fail consistently, what I want to do is fix those failing tests. First, I’ll look into why they’re failing, whether it’s because the hashes don’t match or some other reason.

@vxtls
Copy link
Copy Markdown
Author

vxtls commented May 13, 2026

3. 3.3. tests/guix-environment fails because it's testing if the 32-bit bootstrap-guile has a %host-type beginning with "i686". In a seemingly musl-specific issue, our bootstrap-guile ends up with a %host-type value of "x86_64-pc-linux-muslx32", as the build environment seemingly detects that the kernel is 64-bit under a 32-bit userland, and Guix then thinks this means it's inappropriately picking a 64-bit bootstrap-guile for a 32-bit build, and fails the test. Ideally, we should fix this in bootstrap-guile itself.

fixed in 2a58e2c
testing now

Comment thread steps-guix/guix-1.5.0/patches/accept-local-bootstrap-guile-host-type.patch Outdated
-s i686-linux \
- -- guile -c '(exit (string-prefix? "i686" %host-type))'
+ -- guile -c '(exit (string=? "x86_64-pc-linux-muslx32" %host-type))'
+ -- guile -c '(exit (string-prefix? "x86_64" %host-type))'
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This amounts to effectively disabling the test, as it's meant precisely to check that on a multiarch-capable machine, the correct seed is selected based on the choice of environment (i.e. a 32-bit seed when explicitly building 32-bit on 64-bit hardware).

IMO the right fix is to patch Guile 2.0.9 so it reports the correct "i686-pc-linux-musl" architecture.

@vxtls
Copy link
Copy Markdown
Author

vxtls commented May 26, 2026

I'm currently trying to get Guix through the test, but I've encountered some problems.

building /gnu/store/8z9y6z06rnkq46slbl5prvqjyfcv13hr-guix-1.5.0rc1.drv...
/ 'check' phase�@ build-log 8966 4096
�██████▏ 100.0%
warning: rewriting hashes in `/tmp/guix-tests/store/265w5sn5ksc1cjpxhk574dpd13h1y67x-world'; cross fingers
The following build is still in progress:
  /gnu/store/8z9y6z06rnkq46slbl5prvqjyfcv13hr-guix-1.5.0rc1.drv

The following build is still in progress:
  /gnu/store/8z9y6z06rnkq46slbl5prvqjyfcv13hr-guix-1.5.0rc1.drv

hint: Using 'master' as the name for the initial branch. This default branch name
hint: will change to "main" in Git 3.0. To configure the initial branch name
hint: to use in all of your new repositories, which will suppress this warning,
hint: call:
hint:
hint:   git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint:   git branch -m <name>
hint:
hint: Disable this message with "git config set advice.defaultBranchName false"
Initialized empty Git repository in /tmp/guix-build-guix-1.5.0rc1.drv-0/guix-directory.aQ2jKi/.git/
[master (root-commit) ddca58d] f776c182477307b56fa1b3b6320a8c21995671df265a9c778e3ccbfc6abae089
 2 files changed, 2 insertions(+)
 create mode 100644 a.txt
 create mode 100644 b.scm
guix system: error: open-file: No such file or directory: "/tmp/guix-tests/store/lf3fxb53zncxykvi7z75wf4kppf1p441-git-download.drv"

FULL LOG
10803-ice-9/boot-9.scm:1685:16: In procedure raise-exception:
10804-In procedure getaddrinfo: Servname not supported for ai_socktype
10805:builder for `/tmp/guix-tests/store/4p5fa78bq36capkl8l2sas05f3kh0s63-git-download.drv' failed with exit code 1
10806:@ build-failed /tmp/guix-tests/store/4p5fa78bq36capkl8l2sas05f3kh0s63-git-download.drv - 1 builder for `/tmp/guix-tests/store/4p5fa78bq36capkl8l2sas05f3kh0s63-git-download.drv' failed with exit code 1
10807-hint: Using 'master' as the name for the initial branch. This default branch name
10808-hint: will change to "main" in Git 3.0. To configure the initial branch name
10809-hint: to use in all of your new repositories, which will suppress this warning,
10810-hint: call:
10811-hint:
10812-hint:     git config --global init.defaultBranch <name>
10813-hint:
10814-hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
10815-hint: 'development'. The just-created branch can be renamed via this command:
10816-hint:
10817-hint:     git branch -m <name>
10818-hint:
10819-hint: Disable this message with "git config set advice.defaultBranchName false"
10820-Initialized empty Git repository in /tmp/guix-build-guix-1.5.0rc1.drv-0/guix-directory.F5S7bs/.git/
10821:@ build-started /tmp/guix-tests/store/6g45bx453jyz9dgirmb272aj7jnm4har-git-download.drv - x86_64-linux /tmp/guix-tests/var/log/guix/drvs/6g//45bx453jyz9dgirmb272aj7jnm4har-git-download.drv.gz 8792
10822:Initialized empty Git repository in /tmp/guix-tests/store/39s3ykljjd7q3yp7wbsdnmdsdk9yns75-git-download/.git/
10823-fatal: detected dubious ownership in repository at '/tmp/guix-build-guix-1.5.0rc1.drv-0/guix-directory.F5S7bs/.git/nonexistent'
10824-To add an exception for this directory, call:
10825-
10826-  git config --global --add safe.directory /tmp/guix-build-guix-1.5.0rc1.drv-0/guix-directory.F5S7bs/.git/nonexistent
10827-fatal: Could not read from remote repository.
10828-
10829-Please make sure you have the correct access rights
10830-and the repository exists.
10831-Failed to do a shallow fetch; retrying a full fetch...
10832-fatal: detected dubious ownership in repository at '/tmp/guix-build-guix-1.5.0rc1.drv-0/guix-directory.F5S7bs/.git/nonexistent'
10833-To add an exception for this directory, call:
10834-
10835-  git config --global --add safe.directory /tmp/guix-build-guix-1.5.0rc1.drv-0/guix-directory.F5S7bs/.git/nonexistent
10836-fatal: Could not read from remote repository.
10837-
10838-Please make sure you have the correct access rights
10839-and the repository exists.
10840-git-fetch: '/gnu/store/rj4rixfm0fxhfamzcahwadcivwzf43b1-git-minimal-2.52.0/bin/git fetch -- origin' failed with exit code 128
10841-Trying content-addressed mirror at bordeaux.guix.gnu.org...
10842-Unable to fetch from bordeaux.guix.gnu.org, getaddrinfo-error: (-8)
10843-Trying content-addressed mirror at ci.guix.gnu.org...
10844-Unable to fetch from ci.guix.gnu.org, getaddrinfo-error: (-8)
10845-Trying content-addressed mirror at bordeaux.guix.gnu.org...
10846-Unable to fetch from bordeaux.guix.gnu.org, getaddrinfo-error: (-8)
10847-Trying content-addressed mirror at ci.guix.gnu.org...
10848-Unable to fetch from ci.guix.gnu.org, getaddrinfo-error: (-8)
10849-Trying to download from Software Heritage...
10850-Backtrace:
10851-In ice-9/boot-9.scm:
10852-  1752:10 14 (with-exception-handler _ _ #:unwind? _ # _)
10853-In unknown file:
10854-          13 (apply-smob/0 #<thunk 7ffff77352a0>)
10855-In ice-9/boot-9.scm:
10856-    724:2 12 (call-with-prompt _ _ #<procedure default-prompt-handle…>)
10857-In ice-9/eval.scm:
10858-    619:8 11 (_ #(#(#<directory (guile-user) 7ffff773ac80>)))
10859-In guix/ui.scm:
10860-   2399:7 10 (run-guix . _)
10861-  2362:10  9 (run-guix-command _ . _)
10862-In ice-9/boot-9.scm:
10863-  1752:10  8 (with-exception-handler _ _ #:unwind? _ # _)
10864-In guix/scripts/perform-download.scm:
10865:    228:8  7 (perform-git-download _ _ #:print-build-trace? _)
10866-In guix/build/git.scm:
10867-   134:22  6 (git-fetch-with-fallback "git://localhost:45573/nonexi…" …)
10868-In guix/swh.scm:
10869-    471:2  5 (swh-download-directory-by-nar-hash #vu8(227 176 196 …) …)
10870-   271:22  4 (call "https://archive.softwareheritage.org/api/1/exti…" …)
10871-In web/client.scm:
10872-   482:29  3 (http-request "https://archive.softwareheritage.org/ap…" …)
10873-   338:10  2 (open-socket-for-uri _ #:verify-certificate? _)
10874-In unknown file:
10875-           1 (getaddrinfo "archive.softwareheritage.org" "https" 0 # …)
10876-In ice-9/boot-9.scm:
10877-  1685:16  0 (raise-exception _ #:continuable? _)
10878-
10879-ice-9/boot-9.scm:1685:16: In procedure raise-exception:
10880-In procedure getaddrinfo: Servname not supported for ai_socktype

@vxtls
Copy link
Copy Markdown
Author

vxtls commented May 26, 2026

Here's how some previous issues were resolved: 1. Architecture check: Forced the expected i686/x86_64 to be unified to x86_64. This was just a workaround, not a formal fix. 2. Some random failures occurred because the guix defined in package-manager.scm used the official URL, causing our local bootstrap seeds to not be actually used, resulting in failures. The solution was to cp /var/lib/guix/local-channels/guix/gnu/packages/package-management.scm /usr/share/guile/site/3.0/gnu/packages/package-management.scm. This allows us to overwrite the actual scm with our modified package-management.scm. This modification is relatively simple and an effective fix.

@vxtls
Copy link
Copy Markdown
Author

vxtls commented May 26, 2026

2. Since https://guix.gnu.org/en/blog/2025/privilege-escalation-vulnerabilities-2025/, Guix requires slirp4netns and a working /dev/net/tun to run custom fixed output derivations (i.e. download scripts). These aren't currently bootstrapped before building Guix. I have been able to build slirp4netns using Guix itself (i.e. running guix system image in a guix shell that includes the slirp4netns package), but simply creating /dev/net/tun using mknod proved insufficient - likely some kernel config change is needed. I suggest either using the kernel config from Guix's own kernel package (which is known to work well for this use case), or just reverting the 6 commits for the Guix daemon running in live-bootstrap (but not the actual Guix channel repo!). For now, I used --disable-chroot as a workaround for the affected packages - unfortunately this causes other packages to break, so I had to keep switching chroot on and off manually during the build.

I dont know if this related to #578 (comment) , seems a network related issue? (seems not, not directly related, what would be the potential issue?)

Also, regarding the issue you mentioned with Guix 1.5.0rc1, this is because the download available on the official GNU Guix website is Guix 1.5.0rc1, not the stable release that was released on January 23, 2026.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants