Skip to content

Add real-pilot finding classification harness#64

Draft
InfoSecHack wants to merge 1 commit into
mainfrom
codex/real-pilot-finding-classification
Draft

Add real-pilot finding classification harness#64
InfoSecHack wants to merge 1 commit into
mainfrom
codex/real-pilot-finding-classification

Conversation

@InfoSecHack
Copy link
Copy Markdown
Owner

Summary

  • Add scripts/classify_real_pilot_findings.py to turn local scenario.json + findings.json plus optional reviewer labels into sanitized review artifacts.
  • Add focused tests covering repo-output refusal, label matching, ambiguous prefixes, classification validation, redaction, summaries, no fake scoring fields, and useful evidence summaries.
  • Preserve the bounded pilot model: no live AWS, no Terraform, no raw artifacts committed, and reviewer judgment instead of score/pass-fail labels.

Current pilot dry run

Generated /tmp/iamscope-real-pilot-dev-001-review from the local pilot artifacts:

  • findings: 18
  • verdicts: 15 validated, 3 inconclusive
  • patterns: 15 cross_account_trust, 3 admin_reachability
  • label status: 18 unlabeled
  • output account/ARN scans: clean

Validation

  • python -m pytest -q tests/test_real_pilot_finding_classification.py: 10 passed
  • Local pilot classifier run against /tmp/iamscope-real-pilot-dev-001: passed
  • Generated output account/ARN scans: clean
  • Repo account/ARN hygiene scans: clean
  • Terraform/raw artifact scan: clean
  • ./scripts/check.sh: passed
  • ./scripts/test_fast.sh: 2009 passed
  • git diff --check: passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant