Skip to content

rule: detect phantom required status-check contexts in branch protection #339

@hyperpolymath

Description

@hyperpolymath

Context

Audit of 603 open PRs across the estate revealed many repos have stale required-status-check contexts in their branch protection that can never go green because:

  1. The check name was renamed (e.g. Hypatia neurosymbolic scanHypatia Neurosymbolic Analysis on 2026-05-19)
  2. The check is dispatched externally and doesn't fire on PRs (e.g. Deposit findings for gitbot-fleet, panic-attack assail)

otpiser#11 was blocked for hours despite all required actual checks being green, because of three such phantom contexts on otpiser/main. Dropping them (via gh api -X DELETE repos/o/r/branches/main/protection/required_status_checks/contexts) unblocked the PR immediately.

Symptom

gh pr view N --json mergeStateStatus shows BLOCKED despite failing: [].

Rule

Add to lib/rules/branch_protection.ex:

@known_phantom_contexts [
  "Hypatia neurosymbolic scan",                  # renamed 2026-05-19
  "Deposit findings for gitbot-fleet",           # external dispatch
  "panic-attack assail"                          # external dispatch
]

def check_phantom_required_contexts(branch_protection, recent_runs, _opts \\ []) do
  required = branch_protection["required_status_checks"]["contexts"] || []
  observed_check_names = MapSet.new(Enum.map(recent_runs, & &1["name"]))

  phantoms = Enum.filter(required, fn ctx ->
    ctx in @known_phantom_contexts or
      (not MapSet.member?(observed_check_names, ctx) and
       runs_older_than_30d?(recent_runs))
  end)

  if phantoms != [] do
    {:fail, %{
      rule: :phantom_required_contexts,
      severity: :high,
      message: "Required contexts that never fire on PRs",
      contexts: phantoms,
      fix: "gh api -X DELETE repos/<o>/<r>/branches/main/protection/required_status_checks/contexts -f 'contexts[]=...'"
    }}
  else
    :ok
  end
end

Estate sweep

After this rule lands, run a one-pass audit across all 600+ estate repos and produce a per-repo phantom-cleanup PR (or batch admin-API patch).

Filed from session sweeping otpiser#11 + estate-wide drift.

Metadata

Metadata

Assignees

No one assigned

    Labels

    cicdCI/CD pipeline, GitHub Actions, workflows, rulesets, releasesenhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions