ci: auto-approve integration tests for internal PRs#98
ci: auto-approve integration tests for internal PRs#98danielmeppiel wants to merge 4 commits intomainfrom
Conversation
Fork PRs still require manual environment approval to prevent untrusted artifacts from accessing secrets. Internal PRs (same repo) skip the gate since contributors already have write access. This eliminates manual approval overhead for the common case while preserving the security gate for external contributions.
There was a problem hiding this comment.
Pull request overview
This PR implements auto-approval for integration tests on internal PRs while maintaining the manual approval security gate for fork PRs. It addresses the workflow friction where repo collaborators with write access must manually approve environment protection rules for their own PRs, which provides no security benefit since they can already push directly to main.
Changes:
- Split the single
approvejob intoapprove-fork(requires manual environment approval) andapprove-internal(auto-approved) - Update downstream job dependencies to run if either approval path succeeds using
always()conditional logic - Update workflow comments to reflect the dual approval path architecture
The test harness in 4 integration test files hardcoded .venv/bin/apm
as the primary binary, falling back to PATH only when the venv didn't
exist. In CI (ci-integration.yml), the PR binary artifact is placed on
PATH via symlink, but uv sync also creates .venv/bin/apm from main's
source. The tests always found the venv binary first, effectively
testing main's code instead of the PR's.
Invert the priority: prefer shutil.which('apm') (finds PATH binary in
CI, or venv binary locally since venv activates onto PATH), falling
back to the hardcoded .venv path only when not on PATH.
Affected files:
- test_ado_e2e.py
- test_mixed_deps.py
- test_skill_compile.py
- test_skill_install.py
| # Fallback to local dev venv | ||
| apm_path = Path(__file__).parent.parent.parent / ".venv" / "bin" / "apm" | ||
|
|
There was a problem hiding this comment.
Missing final fallback to "apm" when venv path doesn't exist. The other three test files modified in this PR (test_skill_install.py, test_skill_compile.py, test_mixed_deps.py) all include a final return "apm" statement after checking the venv path. This ensures the test can still attempt to find apm on the system PATH if neither the CI artifact nor the venv binary are available.
Without this fallback, if the venv doesn't exist and shutil.which("apm") returns None, the code will use a Path object instead of a string for apm_path, which could cause unexpected behavior when constructing the command string.
| # Fallback to local dev venv | |
| apm_path = Path(__file__).parent.parent.parent / ".venv" / "bin" / "apm" | |
| # Fallback to local dev venv, then final fallback to "apm" | |
| venv_apm = Path(__file__).parent.parent.parent / ".venv" / "bin" / "apm" | |
| if venv_apm.exists(): | |
| apm_path = str(venv_apm) | |
| else: | |
| # If the venv binary is not available, still attempt to use "apm" on PATH | |
| apm_path = "apm" |
Problem
Every PR — including internal ones from repo collaborators — triggers the
Integration Tests (PR)workflow which requires manual environment approval before running. This creates unnecessary friction:Additionally, 4 integration test files hardcoded
.venv/bin/apmas the binary path, bypassing the PR artifact binary that CI places on PATH. This meant integration tests were silently testing main's source code instead of the PR's binary, defeating the purpose of fork-safe artifact testing.Changes
1. Auto-approve internal PRs (
ci-integration.yml)Split the
approvejob into two:approve-forkhead_repository != github.repository)approve-internalDownstream jobs (
smoke-test→integration-tests→release-validation) run if either approval job succeeds (the other is skipped).2. Fix integration test binary resolution (
test_ado_e2e.py,test_mixed_deps.py,test_skill_compile.py,test_skill_install.py)Inverted binary lookup priority in 4 test files: prefer
shutil.which("apm")(finds the PR binary on PATH in CI) over the hardcoded.venv/bin/apm(main's source install). Falls back to venv path for local development where PATH includes the venv.Without this fix: integration tests for ADO, mixed deps, skills, and skill compilation run against main's source code, not the PR's artifact — regressions go undetected.
3. Upgrade CodeQL Action (
codeql.yml)Updated CodeQL Action from v3 to v4 (Node.js 16 deprecation).
Security model (unchanged for forks)
integration-testsenvironment protection rulesmain(never PR code) for test scriptsmainif neededValidation
'azure-devops'→'github'source label regression). Reverted → all 9 ADO tests pass.