Skip to content

Fix eval workflows#1228

Merged
aaronpowell merged 2 commits intogithub:stagedfrom
JanKrivanek:dev/jankrivanek/fix-eval-workflows
Mar 30, 2026
Merged

Fix eval workflows#1228
aaronpowell merged 2 commits intogithub:stagedfrom
JanKrivanek:dev/jankrivanek/fix-eval-workflows

Conversation

@JanKrivanek
Copy link
Copy Markdown
Contributor

Motivation

Fixing the trigger and limits for eval pipelines from
#1195

Copilot AI review requested due to automatic review settings March 30, 2026 17:43
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adjusts the skill evaluation workflows introduced in PR #1195 by fixing the PR trigger behavior for the PR-gate validator and preventing nightly report creation from failing due to GitHub body-size limits.

Changes:

  • Update the PR-gate workflow to run on pull_request_target and explicitly checkout the PR head SHA.
  • Add report body truncation logic to keep Skill Quality Report discussions/issues under GitHub’s body limit.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
.github/workflows/skill-check.yml Changes trigger to pull_request_target and checks out PR head SHA to run validation and post PR comments.
.github/workflows/skill-quality-report.yml Truncates generated report body before posting to Discussion/Issue to avoid exceeding GitHub limits.
Comments suppressed due to low confidence (1)

.github/workflows/skill-check.yml:26

  • Because this workflow checks out untrusted PR code (head SHA) under pull_request_target, any later shell execution that incorporates repo-controlled paths/content becomes high-risk. In this job, the validator command is assembled from git diff-derived paths and executed as a shell string, which can be vulnerable to command injection with crafted filenames. If you keep pull_request_target, ensure subsequent steps build commands using argument arrays / proper quoting (avoid executing concatenated strings) and consider persist-credentials: false on checkout to further reduce token exposure.
      - name: Checkout PR code
        uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
        with:
          ref: ${{ github.event.pull_request.head.sha }}
          fetch-depth: 0

- skill-check.yml: Revert to pull_request trigger (read-only token).
  Remove PR comment posting; upload results as artifact instead.
- skill-check-comment.yml: New workflow_run-triggered workflow that
  downloads the artifact and posts/updates the PR comment with
  write permissions, without ever checking out PR code.
- skill-quality-report.yml: Replace character-based truncation with
  byte-based (Buffer.byteLength) limit. Shrink <details> sections
  structurally before falling back to hard byte-trim, keeping
  markdown rendering intact.
@aaronpowell aaronpowell merged commit 1c60024 into github:staged Mar 30, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants