fix(aspect+e2e): make tests truthful + ums-mcp Mutex + install Elixir (unblocks #149)#150
Merged
Merged
Conversation
🏁 path-claims benchCommit NumbersHost-dependent — compare deltas across commits, not absolute values. |
🔍 Hypatia Security ScanFindings: 251 issues detected
View findings[
{
"reason": "Stale AI session file -- delete",
"type": "stale",
"file": "GEMINI.md",
"action": "delete",
"rule_module": "root_hygiene",
"severity": "medium"
},
{
"reason": "Action hyperpolymath/standards/.github/workflows/governance-reusable.yml@main needs attention",
"type": "unpinned_action",
"file": "governance.yml",
"action": "pin_sha",
"rule_module": "workflow_audit",
"severity": "high"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/cartridges/sanctify-mcp/adapter/mod.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/cartridges/academic-workflow-mcp/adapter/mod.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/cartridges/fireflag-mcp/adapter/mod.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/cartridges/ephapax-mcp/adapter/mod.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/cartridges/bofig-mcp/adapter/mod.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/cartridges/hesiod-mcp/adapter/mod.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/mcp-bridge/main.d.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "believe_me undermines formal verification (1 occurrences, CWE-704)",
"type": "believe_me",
"file": "/home/runner/work/boj-server/boj-server/src/abi/Boj/SafeHTTP.idr",
"action": "flag",
"rule_module": "code_safety",
"severity": "critical"
}
]Powered by Hypatia Neurosymbolic CI/CD Intelligence |
hyperpolymath
added a commit
that referenced
this pull request
May 25, 2026
Two follow-up failures on #150's first CI run, each fixed at source: 1. **Zig FFI Tests** — failed with "no build.zig file found" for orchestrator-lsp-mcp. Root cause: the scope step's `git diff --name-only -- 'cartridges/**'` pulls a cartridge into the test set as soon as ANY file under it changes — including cartridge.json. My PR added a `status` field to 15 cartridges' cartridge.json, which pulled orchestrator-lsp-mcp into scope, but that cart's build.zig lives at `ffi/zig/build.zig` (deeper nesting) so the workflow's `cd cartridges/$cart/ffi && zig build test` fell over. Tightened the pathspec to `cartridges/*/ffi/**` so only ffi/-relevant diffs scope a cart in — matches the workflow's own `on.paths` filter, which was already `cartridges/**/ffi/**` (the pathspec and the diff filter were inconsistent before). 2. **E2E — Server did not start within 10 seconds** — Elixir backend was *found* (the setup-beam fix from the first commit worked), but `mix run --no-halt` on a cold runner spends most of the 10s window compiling deps. Two changes: (a) added `mix compile` as a workflow step so deps land in `_build/` before the test starts the server; (b) bumped the script's wait window from 10s (50 × 200ms) to 60s (300 × 200ms) so a slow CI cold start doesn't false-fail. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
🔍 Hypatia Security ScanFindings: 251 issues detected
View findings[
{
"reason": "Stale AI session file -- delete",
"type": "stale",
"file": "GEMINI.md",
"action": "delete",
"rule_module": "root_hygiene",
"severity": "medium"
},
{
"reason": "Action hyperpolymath/standards/.github/workflows/governance-reusable.yml@main needs attention",
"type": "unpinned_action",
"file": "governance.yml",
"action": "pin_sha",
"rule_module": "workflow_audit",
"severity": "high"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/cartridges/sanctify-mcp/adapter/mod.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/cartridges/academic-workflow-mcp/adapter/mod.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/cartridges/fireflag-mcp/adapter/mod.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/cartridges/ephapax-mcp/adapter/mod.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/cartridges/bofig-mcp/adapter/mod.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/cartridges/hesiod-mcp/adapter/mod.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/mcp-bridge/main.d.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "believe_me undermines formal verification (1 occurrences, CWE-704)",
"type": "believe_me",
"file": "/home/runner/work/boj-server/boj-server/src/abi/Boj/SafeHTTP.idr",
"action": "flag",
"rule_module": "code_safety",
"severity": "critical"
}
]Powered by Hypatia Neurosymbolic CI/CD Intelligence |
🔍 Hypatia Security ScanFindings: 251 issues detected
View findings[
{
"reason": "Stale AI session file -- delete",
"type": "stale",
"file": "GEMINI.md",
"action": "delete",
"rule_module": "root_hygiene",
"severity": "medium"
},
{
"reason": "Action hyperpolymath/standards/.github/workflows/governance-reusable.yml@main needs attention",
"type": "unpinned_action",
"file": "governance.yml",
"action": "pin_sha",
"rule_module": "workflow_audit",
"severity": "high"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/cartridges/sanctify-mcp/adapter/mod.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/cartridges/academic-workflow-mcp/adapter/mod.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/cartridges/fireflag-mcp/adapter/mod.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/cartridges/ephapax-mcp/adapter/mod.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/cartridges/bofig-mcp/adapter/mod.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/cartridges/hesiod-mcp/adapter/mod.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/mcp-bridge/main.d.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "believe_me undermines formal verification (1 occurrences, CWE-704)",
"type": "believe_me",
"file": "/home/runner/work/boj-server/boj-server/src/abi/Boj/SafeHTTP.idr",
"action": "flag",
"rule_module": "code_safety",
"severity": "critical"
}
]Powered by Hypatia Neurosymbolic CI/CD Intelligence |
… elixir Two failing checks on PR #149 (and on every PR/main run since at least 2026-05-20) had four distinct root causes. Each fixed at source: ## Aspect — Thread Safety + ABI Contract + SPDX (27 → 0 failures) 1. **Comment-stripping filters were broken.** Aspect 2 (Idris2 banned patterns) used `grep -v '^\s*--'` and `grep -v '^\s*|||'` to skip line-comments and `|||` doc-comments — but `grep -rn` output is `path:lineno:content`, so the line never starts with `--` or `|||`; it starts with the path. The filters silently let every commented match through, producing two false-positive failures (Admitted in `cartridges/fleet-mcp/.../SafeFleet.idr` doc-comment + trailing `Echidnabot — ... (Admitted, sorry)` comment). Fixed by anchoring the filters at `:[[:space:]]*--` etc., factored into one `strip_comments_and_docstrings` helper that also handles trailing `-- … <pat>` comment matches. 2. **`believe_me` check didn't exempt class-J axioms.** `src/abi/Boj/ SafetyLemmas.idr` declares 5 documented class-J `believe_me` primitives (`charEqSound`, `charEqSym`, `unpackLength`, `appendLengthSum`, `substrLengthBound`) — see PROOF-NEEDS.md / ADR- 008. Added a `PROOF_EXEMPT` regex so the test passes on documented axioms while still failing on any new `believe_me` elsewhere. 3. **Aspect 1 Mutex check was over-aggressive.** It failed any .zig file with `pub export fn` + zero `Mutex` references — including purely-functional FFI like `cartridges/burble-admin-mcp/ffi/ burble_admin_ffi.zig` (3 exports, ZERO file-scope globals — table lookups + arithmetic over i32). 9 false-positive failures. The right invariant: only fail when there's ALSO file-scope mutable global state (`^(pub )?var <ident>`). Refined accordingly. Now reports purely-functional FFI with a clear pass message. 4. **Aspect 4 lacked a stub/ffi_only status.** 15 cartridges failed "incomplete layers (ABI=false ...)" — but ~10 of them are manifest-only stubs (cartridge.json declares the API surface, no abi/ or ffi/ yet) and ~5 are intentionally proof-free observability/glue (boj-health, claude-ai-mcp, lang-mcp, orchestrator-lsp-mcp, toolchain-mcp). Added a `"status"` field to `cartridge.json` (`complete` (default) / `stub` / `ffi_only`); Aspect 4 honours it and reports `(N complete, M stub, K ffi_only)` so the categories stay visible. 5. **ums-mcp had a real bug.** 15 C-ABI exports operating on a global `var sessions: [MAX_SESSIONS]SessionSlot` array, no Mutex. The filter fixes above narrow Aspect 1 to true positives, and this was the one left over. Added `var sessions_mu: std.Thread.Mutex` and `sessions_mu.lock(); defer sessions_mu.unlock();` to all 14 sessions-touching exports. `ums_can_transition` is a pure function (enum→enum) and stays lock-free. Mirrors the 007-mcp pattern (`g_state_mu` in `cartridges/007-mcp/ffi/oo7_mcp_ffi.zig:79`). `cd cartridges/ums-mcp/ffi && zig build` passes. After all five fixes: 115 passed / 0 failed / 1 warning (was 87/27/1). The one warning (`federation.zig` `catch unreachable` patterns) was already pre-existing — out of scope here. ## E2E — Full REST + MCP Bridge (failing since 2026-05-20) `tests/e2e_full.sh` requires `mix` to start the Elixir backend, but `.github/workflows/e2e.yml` never installed Elixir/OTP. Added an `erlef/setup-beam@v1.18.2` step (Elixir 1.18 + OTP 27 — matches the estate convention used in every other repo's hypatia-scan.yml) plus a `mix deps.get` step before the test runs. ## Foundational follow-up (NOT in this PR) Same gap as r-g-t-v#89 and absolute-zero#42: `main` branch protection has no `required_status_checks` block, which is how three workflows (E2E, OpenSSF Scorecard Enforcer, Instant Sync) have been failing on main for days without blocking merges. Hypatia PR #316 ships the BH001/BH002/BH003 rules that detect this class estate-wide. ## Test plan - [x] `bash tests/aspect_tests.sh` — 115/0/1 (was 87/27/1) - [x] `cd cartridges/ums-mcp/ffi && zig build` — clean - [x] All cartridge.json files still valid JSON - [x] e2e.yml YAML parses; step ordering correct (setup-beam before build-FFI / run-e2e) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two follow-up failures on #150's first CI run, each fixed at source: 1. **Zig FFI Tests** — failed with "no build.zig file found" for orchestrator-lsp-mcp. Root cause: the scope step's `git diff --name-only -- 'cartridges/**'` pulls a cartridge into the test set as soon as ANY file under it changes — including cartridge.json. My PR added a `status` field to 15 cartridges' cartridge.json, which pulled orchestrator-lsp-mcp into scope, but that cart's build.zig lives at `ffi/zig/build.zig` (deeper nesting) so the workflow's `cd cartridges/$cart/ffi && zig build test` fell over. Tightened the pathspec to `cartridges/*/ffi/**` so only ffi/-relevant diffs scope a cart in — matches the workflow's own `on.paths` filter, which was already `cartridges/**/ffi/**` (the pathspec and the diff filter were inconsistent before). 2. **E2E — Server did not start within 10 seconds** — Elixir backend was *found* (the setup-beam fix from the first commit worked), but `mix run --no-halt` on a cold runner spends most of the 10s window compiling deps. Two changes: (a) added `mix compile` as a workflow step so deps land in `_build/` before the test starts the server; (b) bumped the script's wait window from 10s (50 × 200ms) to 60s (300 × 200ms) so a slow CI cold start doesn't false-fail. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
mcp-bridge/main.js imports nickel-validator.js, which does an
`existsSync(...)` on the .ncl contract file. That requires Deno
read permission. The shebang in main.js correctly lists
`--allow-read`, but `deno run <file>` ignores shebangs — only the
flags on the CLI invocation matter.
Reproduced locally:
$ echo '{"jsonrpc":"2.0","id":1,"method":"initialize",...}' \
| deno run --allow-net --allow-env mcp-bridge/main.js
error: Uncaught (in promise) NotCapable: Requires read access to
"/.../cartridges/local-coord-mcp/schemas/coord-messages-contracts.ncl"
Adding --allow-read to MCP_RUNNER. Fixes 3 of the 17 E2E feature
failures surfaced once my setup-beam fix got the server actually
running:
- MCP initialize (was: expected '"result"', got '{}')
- MCP tools/list (was: expected '"tools"', got '{}')
- MCP tools/call boj_health (was: expected '"result"', got '{}')
The remaining 14 failures (feedback/order paths) are a test-vs-router
mismatch — the test calls e.g. `POST /cartridges/feedback-mcp/load`
but the router only exposes `POST /cartridge/:name/invoke`
(singular, no `/load` step — cartridges auto-load via Catalog). That
needs real product-side work: either add the missing routes
(`/cartridges/:name/load`, `/cartridges/:name/invoke`, `/order`) or
rewrite tests to use existing routes. Filing as a separate concern;
out of scope for this baseline-CI-rot PR.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
8a411bf to
b8a36dc
Compare
🔍 Hypatia Security ScanFindings: 251 issues detected
View findings[
{
"reason": "Stale AI session file -- delete",
"type": "stale",
"file": "GEMINI.md",
"action": "delete",
"rule_module": "root_hygiene",
"severity": "medium"
},
{
"reason": "Action hyperpolymath/standards/.github/workflows/governance-reusable.yml@main needs attention",
"type": "unpinned_action",
"file": "governance.yml",
"action": "pin_sha",
"rule_module": "workflow_audit",
"severity": "high"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/cartridges/sanctify-mcp/adapter/mod.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/cartridges/academic-workflow-mcp/adapter/mod.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/cartridges/fireflag-mcp/adapter/mod.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/cartridges/ephapax-mcp/adapter/mod.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/cartridges/bofig-mcp/adapter/mod.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/cartridges/hesiod-mcp/adapter/mod.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "TypeScript file detected -- banned language",
"type": "banned_language_file",
"file": "/home/runner/work/boj-server/boj-server/mcp-bridge/main.d.ts",
"action": "flag",
"rule_module": "cicd_rules",
"severity": "critical"
},
{
"reason": "believe_me undermines formal verification (1 occurrences, CWE-704)",
"type": "believe_me",
"file": "/home/runner/work/boj-server/boj-server/src/abi/Boj/SafeHTTP.idr",
"action": "flag",
"rule_module": "code_safety",
"severity": "critical"
}
]Powered by Hypatia Neurosymbolic CI/CD Intelligence |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two failing checks (Aspect 27 fails + E2E 1 fail) had four distinct root causes. Each fixed at source:
Aspect — Thread Safety + ABI Contract + SPDX (27 → 0 fails)
^\s*--/^\s*|||never matched becausegrep -rnoutput starts withpath:lineno:, not content:[[:space:]]*--/:[[:space:]]*|||+ add trailing-- … <pat>comment filter, factored intostrip_comments_and_docstringshelperbelieve_mecheck didn't exempt the 5 documented class-J axioms insrc/abi/Boj/SafetyLemmas.idrPROOF_EXEMPTregex; passes on documented axioms, fails on any new use elsewhereburble_admin_ffi.zig— 3 exports, zero file-scope globals — table lookups overi32)^(pub )?var <ident>file-scope global state. Purely-functional exports now get an explicit pass message"status": "stub" | "ffi_only" | "complete"(defaultcomplete) field tocartridge.json. Aspect 4 honours it and reports(N complete, M stub, K ffi_only)so categories stay visible in CI logscartridges/ums-mcp/ffi/ums_ffi.zighas 15 C-ABI exports operating on a globalvar sessions: [MAX_SESSIONS]SessionSlot, no Mutex. Once the filter fixes narrow Aspect 1 to true positives, this is the one that remains.var sessions_mu: std.Thread.Mutexandsessions_mu.lock(); defer sessions_mu.unlock();to all 14 sessions-touching exports.ums_can_transitionis pure (enum→enum) and stays lock-free. Mirrorsg_state_muincartridges/007-mcp/ffi/oo7_mcp_ffi.zig:79.zig buildpassesAfter all five: 115 passed / 0 failed / 1 warning (was 87/27/1). The one warning (
federation.zigcatch unreachable) was pre-existing — out of scope here.E2E — Full REST + MCP Bridge (failing since 2026-05-20)
tests/e2e_full.shrequiresmixto start the Elixir backend, but.github/workflows/e2e.ymlnever installed Elixir/OTP. Added anerlef/setup-beam@v1.18.2step (Elixir 1.18 + OTP 27 — matches the estate convention used in every repo's hypatia-scan.yml) plus amix deps.getstep before the test runs.Test plan
bash tests/aspect_tests.sh— 115/0/1 (was 87/27/1)cd cartridges/ums-mcp/ffi && zig build— cleancartridge.jsonfiles still valid JSONe2e.ymlYAML parses; step ordering correct (setup-beam before build-FFI / run-e2e)Foundational follow-up (NOT in this PR — flagged)
Same gap as
r-g-t-v#89andabsolute-zero#42:mainbranch protection has norequired_status_checksblock, so red-CI PRs can merge despite three workflows (E2E + Aspect + Bench, OpenSSF Scorecard Enforcer, Instant Sync) being persistently red on main. Hypatia PR #316 ships BH001/BH002/BH003 to detect this class estate-wide; the BH001 rule fires on exactly this repo.🤖 Generated with Claude Code