diff --git a/CLAUDE.md b/CLAUDE.md index 0c3fa0f9..74a43ff6 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -8,7 +8,7 @@ This document provides guidelines and context for working with Claude Code on th **Repository**: https://github.com/hyperpolymath/echidna **Version**: 2.3.0 (see `CHANGELOG.md`; release-tag publication tracked in `docs/handover/TODO.md`) -**License**: MPL-2.0 +**License**: AGPL-3.0-or-later **Architecture overview**: [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) **Canonical prover count**: [`docs/PROVER_COUNT.md`](docs/PROVER_COUNT.md) (128 total, 12 core) **Environment variables**: [`docs/ENV-VARS.md`](docs/ENV-VARS.md) @@ -215,10 +215,10 @@ cargo fmt --check # Format check - **RSR/CCCP Compliance Required** - follow Rhodium Standard Repository guidelines - **Justfile PRIMARY** - never use Make or other build systems - **Podman not Docker** - always use Podman for containers -- **License**: MPL-2.0 (not AGPL, not dual MIT/Palimpsest) +- **License**: AGPL-3.0-or-later (migrated from MPL-2.0 in #112, 2026-05-27) - **Author**: Jonathan D.A. Jewell --- -**Last Updated**: 2026-03-23 +**Last Updated**: 2026-05-30 **Maintained By**: Jonathan D.A. Jewell diff --git a/Justfile b/Justfile index 74ddc200..ae135a02 100644 --- a/Justfile +++ b/Justfile @@ -534,6 +534,43 @@ bench-chapel-mrr: cd src/chapel && chpl -o bench_mrr bench_mrr.chpl && \ ./bench_mrr --verbose=false --timeout=10 +# Rebuild Chapel 2.8.0 from source with CHPL_LIB_PIC=pic so that +# `chpl --library --dynamic` can produce a shared-library form of the +# metalayer. The apt deb ships only the `lib_pic-none` runtime +# variant. ~25-35 min wall on a 4-core x86_64. ~5 GB disk under +# ~/.cache/echidna/chapel-pic. Procedure + tradeoffs: +# docs/decisions/2026-05-30-chapel-pic-rebuild.md. NOT a CI step. +chapel-pic-from-source: + #!/usr/bin/env bash + set -euo pipefail + CACHE_DIR="${ECHIDNA_CHAPEL_PIC_CACHE:-$HOME/.cache/echidna/chapel-pic}" + CHPL_VERSION="${CHPL_VERSION:-2.8.0}" + mkdir -p "$CACHE_DIR" + cd "$CACHE_DIR" + if [ ! -f "chapel-${CHPL_VERSION}.tar.gz" ]; then + echo "Downloading chapel ${CHPL_VERSION}..." + curl -fL "https://github.com/chapel-lang/chapel/releases/download/${CHPL_VERSION}/chapel-${CHPL_VERSION}.tar.gz" \ + -o "chapel-${CHPL_VERSION}.tar.gz" + fi + if [ ! -d "chapel-${CHPL_VERSION}" ]; then + tar xf "chapel-${CHPL_VERSION}.tar.gz" + fi + cd "chapel-${CHPL_VERSION}" + export CHPL_LIB_PIC=pic + export CHPL_LLVM=bundled + export CHPL_HOME="$PWD" + source util/setchplenv.bash + echo "Building chapel runtime with CHPL_LIB_PIC=pic (~30 min)..." + make -j"$(nproc)" + echo "Verifying lib_pic-pic variant landed..." + if find lib -name 'lib_pic-pic' -type d | head -1 | grep -q .; then + echo "OK: PIC runtime built at $CHPL_HOME" + echo "Use: PATH=$CHPL_HOME/bin/linux64-x86_64:\$PATH chpl --library --dynamic ..." + else + echo "ERROR: lib_pic-pic variant not found after build" >&2 + exit 1 + fi + # Build Zig FFI bridge for Chapel (prerequisite for --features chapel) build-chapel-ffi: @echo "Building Zig FFI bridge..." diff --git a/docs/decisions/2026-05-30-chapel-pic-rebuild.md b/docs/decisions/2026-05-30-chapel-pic-rebuild.md new file mode 100644 index 00000000..16acde34 --- /dev/null +++ b/docs/decisions/2026-05-30-chapel-pic-rebuild.md @@ -0,0 +1,148 @@ + + +# 2026-05-30 — Chapel PIC rebuild from source for `--library --dynamic` + +ADR-style addendum to [2026-05-30-chapel-rehabilitation.md][rehab]. +Documents the procedure and constraints for replacing the apt-shipped +Chapel runtime with a position-independent variant so the metalayer can +be linked as a shared library instead of a static archive. + +[rehab]: ./2026-05-30-chapel-rehabilitation.md + +## Status + +Documented. Procedure is `just chapel-pic-from-source`; **not** invoked +from any CI workflow because the build is ~30 min wall and ~5 GB disk, +which would dominate the Chapel CI budget for negligible runtime gain +until L2.5 multi-locale work needs the shared-library form. + +## Context + +The rehabilitation PR (#146) chose `--library --static` for +`chapel-build` because the apt-distributed Chapel deb only ships one +runtime variant — `lib_pic-none`: + +``` +/usr/lib/chapel/2.8/runtime/lib/linux64/gnu/x86_64/loc-flat/ + comm-none/tasks-qthreads/launch-smp/tmr-generic/unwind-system/ + mem-jemalloc/atomics-cstdlib/lib_pic-none/san-none/libchpllaunch.a +``` + +`chpl --library --dynamic` requires a `lib_pic-pic/` variant in the +same path tree; without it, the link of `libchpl.a` into a `.so` fails +with: + +``` +relocation R_X86_64_TPOFF32 against symbol `chpl_task_root_uniqueId' +can not be used when making a shared object; recompile with -fPIC +``` + +The same constraint applies to the conda-forge `chapel` package, which +also ships only the non-PIC runtime variant as of 2.8.0. + +The only way to obtain a PIC runtime is to build Chapel from the +release source with `CHPL_LIB_PIC=pic` set at `make` time. The variant +ends up at the parallel `lib_pic-pic/` path and is auto-discovered by +`chpl --library --dynamic` at link time. + +## Decision + +Ship a documented, reproducible Justfile recipe that performs the +source rebuild on demand, but do **not** invoke it from CI and do **not** +change the canonical `chapel-build` recipe. Local developers and the +eventual L2.5 multi-locale workflow can opt in via `just chapel-pic-from-source`. + +Rationale: + +- The PIC rebuild is a one-shot ~30 min operation per CI runner, which + cannot be amortised across PRs without a registry-pushed container + image — see follow-up below. +- The static-library form (`--library --static`) is correct for the + current Rust-link path; Rust binaries already embed the Chapel + runtime by linking against `libechidna_chapel.a`. Switching to + `.so` linkage gains separability of the Chapel runtime but adds + packaging complexity (now two artifacts to ship and version). +- The PIC form is only **required** for L2.5 (multi-locale Chapel + dispatch under PGAS), where the runtime needs to be loadable from + multiple Chapel-distinct executables. That work is gated on L1 + Cap'n Proto, so the PIC runtime is a precondition we can satisfy + asynchronously when L2.5 starts. + +## Procedure + +```bash +just chapel-pic-from-source +``` + +Equivalent to: + +```bash +mkdir -p ~/.cache/echidna/chapel-pic && cd ~/.cache/echidna/chapel-pic + +# Source — 2.8.0 matches the local development install. Adjust if +# CI moves to a newer point release. +curl -fL https://github.com/chapel-lang/chapel/releases/download/2.8.0/chapel-2.8.0.tar.gz \ + -o chapel-2.8.0.tar.gz +tar xf chapel-2.8.0.tar.gz +cd chapel-2.8.0 + +# PIC runtime variant — the single env var that drives the additional +# `lib_pic-pic/` subtree under `runtime/lib/.../`. LLVM is the default +# back end on 2.8+; the apt deb also uses LLVM, so we match. +export CHPL_LIB_PIC=pic +export CHPL_LLVM=bundled # match apt deb +export CHPL_HOME=$PWD +source util/setchplenv.bash + +# Build. This step is ~25-35 min wall on a 4-core x86_64 box. +make -j$(nproc) + +# Verify the PIC variant landed. +find lib -name 'lib_pic-pic' -type d | head -1 +``` + +After this completes, `chpl` from `$CHPL_HOME/bin/linux64-x86_64/chpl` +can produce shared-library output: + +```bash +$CHPL_HOME/bin/linux64-x86_64/chpl \ + --library --dynamic -I ../zig_ffi \ + -o libechidna_chapel parallel_proof_search.chpl chapel_ffi_exports.chpl +``` + +## Tradeoffs + +| Aspect | `--library --static` (current default) | `--library --dynamic` (PIC rebuild) | +|---|---|---| +| Build time, cold | ~10 s | ~30 min (one-time) + ~10 s | +| Build time, warm | ~10 s | ~10 s | +| Disk footprint | runtime in libchpl.a (~30 MB) | runtime in libchpl.so (~30 MB) + source tree (~5 GB) | +| Rust link | static archive into executable | dynamic load at runtime | +| Multi-process sharing | each process embeds runtime | runtime shared across processes | +| L2.5 multi-locale | does not support | required | + +## Follow-up + +A registry-pushed Containerfile that pre-builds the PIC runtime would +amortise the 30 min cost across CI runs, but requires: + +- A `Containerfile.chapel-pic` under `.containerization/`. +- A registry destination (ghcr.io/hyperpolymath/echidna-chapel-pic:2.8.0). +- A CI job to build + push on chapel version bumps. + +Tracked as a Wave-3 follow-up issue; the current PR does **not** add +the container path because it would duplicate the in-flight L1 Cap'n +Proto schema work and risk image-tag drift if Chapel point-releases +between now and L2.5 starting. + +## Verification + +The recipe is not executed in CI for the reasons above. Local +verification (one-time, by anyone who wants `--dynamic` linkage) is: + +```bash +just chapel-pic-from-source +# Confirm: +find ~/.cache/echidna/chapel-pic/chapel-2.8.0/lib -name 'lib_pic-pic' -type d +# Expected output: at least one path containing `lib_pic-pic/`. +``` diff --git a/docs/decisions/2026-05-30-chapel-rehabilitation.md b/docs/decisions/2026-05-30-chapel-rehabilitation.md index 6ea5f79f..64cfca17 100644 --- a/docs/decisions/2026-05-30-chapel-rehabilitation.md +++ b/docs/decisions/2026-05-30-chapel-rehabilitation.md @@ -106,9 +106,12 @@ Wave 1 (this PR, echidna only): - `Justfile` recipes: `chapel-build`, `chapel-smoke`, `chapel-test`. Wave 2 (separate PR, after Wave 1 green-on-main for ≥7 days): -- Rebuild Chapel CI image with `CHPL_LIB_PIC=pic` runtime; revert +- Rebuild Chapel from source with `CHPL_LIB_PIC=pic` runtime; revert the metalayer build to `--dynamic`; flip `rust-chapel-real` to - strict. + strict. Recipe + tradeoffs: + [2026-05-30-chapel-pic-rebuild.md](./2026-05-30-chapel-pic-rebuild.md). + CI flip deferred — the ~30 min source build dominates the chapel-ci + budget; a registry-pushed container image is the L2.5 prerequisite. - Add the cancel-token thread through `tryProver` and switch the Chapel-side default to the speculative search. - Wire the `proven` and `docudactyl` parallel-dispatch tracks diff --git a/docs/wiki/FAQ.md b/docs/wiki/FAQ.md index 36208bf7..11ebbec3 100644 --- a/docs/wiki/FAQ.md +++ b/docs/wiki/FAQ.md @@ -35,9 +35,9 @@ Canonical sources of truth: - [`.machine_readable/6a2/STATE.a2ml`](https://github.com/hyperpolymath/echidna/blob/main/.machine_readable/6a2/STATE.a2ml) for current state - [`docs/ROADMAP.md`](https://github.com/hyperpolymath/echidna/blob/main/docs/ROADMAP.md) for direction -## Why MPL-2.0 and not MIT or AGPL? +## Why AGPL-3.0-or-later? -Practical balance: weak copyleft at the file level (modifications to MPL'd files must be open) without copyleft-by-linking (so downstream commercial use is straightforward). The project migrated from a dual MIT/Palimpsest-0.6 licence in 2026; `NOTICE` and `LICENSE` reflect the current state. +Estate-wide policy decision (co-dev alignment). The project migrated from MPL-2.0 to AGPL-3.0-or-later on 2026-05-27 (#112); prior to MPL-2.0 it had been dual MIT/Palimpsest-0.6. `NOTICE` and `LICENSE` reflect the current state. ## How do I report a security issue?