FEAT: Forked PR coverage comment workflow #375

gargsaumya · 2025-12-16T09:40:39Z

Work Item / Issue Reference

AB#40910

GitHub Issue: #<ISSUE_NUMBER>

Summary

Testing the workflow_run solution for posting coverage comments on forked PRs.

Copilot

Pull request overview

This PR implements a workflow_run solution to enable posting code coverage comments on pull requests from forked repositories, which normally face token permission restrictions. The implementation splits the coverage reporting into two workflows: one that runs on pull_request events and collects coverage data, and another that runs on workflow_run completion events with elevated permissions to post comments.

Key Changes:

Added artifact-based data passing between the coverage workflow and a new comment-posting workflow
Created a new workflow triggered by workflow_run events to handle comment posting with proper permissions
Added test print statements to main.py to validate the PR workflow

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 6 comments.

File	Description
.github/workflows/pr-code-coverage.yml	Added steps to save coverage data as JSON artifact and conditionally skip PR comments for forked repos
.github/workflows/post-coverage-comment.yml	New workflow that downloads coverage artifacts from completed workflow runs and posts PR comments with proper permissions
main.py	Added test print statements to validate the workflow (test changes only)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.github/workflows/pr-code-coverage.yml

Copilot · 2025-12-16T09:51:01Z

.github/workflows/post-coverage-comment.yml

+      - name: Download coverage data
+        uses: actions/download-artifact@v4
+        with:
+          name: coverage-comment-data
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+          run-id: ${{ github.event.workflow_run.id }}
+


The workflow assumes that artifacts uploaded in a workflow_run context will be available, but there's no error handling if the artifact download fails for other reasons (network issues, artifact expiration, etc.). Consider adding a conditional check after the download step to verify the artifact was successfully downloaded before proceeding to read the file.

Copilot · 2025-12-16T09:51:02Z

.github/workflows/post-coverage-comment.yml

+    if: >
+      github.event.workflow_run.event == 'pull_request' &&
+      github.event.workflow_run.conclusion == 'success'


The condition only checks if the workflow_run concluded successfully, but it doesn't verify that the coverage artifact was actually uploaded. If the pr-code-coverage workflow succeeds but skips the artifact upload step (e.g., due to the conditional on line 446 of pr-code-coverage.yml), this workflow will still run and fail when trying to download a non-existent artifact. Consider adding a check for the specific job or step status that uploads the artifact.

yes, can we have a step here which explicilty checks if artifact is present or not?

Copilot · 2025-12-16T09:51:02Z

.github/workflows/pr-code-coverage.yml

+          }
+          EOF
+          cat coverage-comment-data/pr-info.json
+


The step does not validate the JSON output before uploading it as an artifact. If the jq command fails or produces invalid JSON due to special characters in the environment variables, the artifact will contain malformed JSON that will cause the downstream workflow to fail. Consider adding a validation step using 'jq . coverage-comment-data/pr-info.json' after generating the file to ensure it's valid JSON before uploading.

Suggested change

- name: Validate coverage comment JSON

run: jq . coverage-comment-data/pr-info.json

valid case, can add this

Copilot · 2025-12-16T09:51:02Z

.github/workflows/post-coverage-comment.yml

+          echo "LOW_COVERAGE_FILES<<EOF" >> $GITHUB_ENV
+          echo "$LOW_COV_FILES" >> $GITHUB_ENV
+          echo "EOF" >> $GITHUB_ENV


When extracting multiline values and using heredoc syntax to set environment variables, the values should be properly quoted to prevent shell interpretation issues. The current implementation could fail if the JSON-extracted values contain special shell characters. Consider wrapping the echo statements in quotes, e.g., 'echo "$LOW_COV_FILES"' instead of 'echo $LOW_COV_FILES'.

Copilot · 2025-12-16T09:51:03Z

.github/workflows/post-coverage-comment.yml

+        uses: actions/download-artifact@v4
+        with:
+          name: coverage-comment-data
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+          run-id: ${{ github.event.workflow_run.id }}


The actions/download-artifact@v4 action does not support the 'run-id' parameter for downloading artifacts from another workflow run. This parameter was available in v3 but was removed in v4. To download artifacts from a workflow_run event, you need to use the GitHub API directly or use a third-party action like dawidd6/action-download-artifact that supports cross-workflow artifact downloads.

Suggested change

uses: actions/download-artifact@v4

with:

name: coverage-comment-data

github-token: ${{ secrets.GITHUB_TOKEN }}

run-id: ${{ github.event.workflow_run.id }}

uses: dawidd6/action-download-artifact@v2

with:

workflow: PR Code Coverage

run_id: ${{ github.event.workflow_run.id }}

name: coverage-comment-data

github_token: ${{ secrets.GITHUB_TOKEN }}

bewithgaurav

I have added comments related to logic optimizations, restructuring and code comments. requesting changes on this, please reach out if you need any clarification 👍

bewithgaurav · 2025-12-22T10:15:32Z

main.py

 # Clean one-liner: set level and output mode together
 setup_logging(output="both")
-
+print("Logging is set up.")


these changes are unrelated I guess, could you pls remove?

bewithgaurav · 2025-12-22T11:15:19Z

.github/workflows/post-coverage-comment.yml

+name: Post Coverage Comment
+
+on:
+  workflow_run:


we need a detailed explanation over here on why we created a new yml which ticks on workflow_run - this will be better for future reference and keeping context on the research

below comment is from my understanding

# This workflow posts coverage comments for forked PRs. # # Why a separate workflow? # GitHub restricts write permissions for `pull_request` events from forks # (security measure to prevent malicious code from modifying the target repo). # The `workflow_run` event runs code from the main branch with full permissions, # allowing us to post comments on forked PRs safely. # # Data flow: pr-code-coverage.yml saves metrics as artifact → this workflow reads artifact → posts comment

bewithgaurav · 2025-12-22T11:17:25Z

.github/workflows/post-coverage-comment.yml

can we change the file's name to forked-pr-coverage-comment.yml since it Posts comments for forked PRs.. there is no other 'post' activity that's being done here

bewithgaurav · 2025-12-22T11:19:35Z

.github/workflows/post-coverage-comment.yml

+    if: >
+      github.event.workflow_run.event == 'pull_request' &&
+      github.event.workflow_run.conclusion == 'success'


yes, can we have a step here which explicilty checks if artifact is present or not?

bewithgaurav · 2025-12-22T11:20:54Z

.github/workflows/pr-code-coverage.yml

+        with:
+          name: coverage-comment-data
+          path: coverage-comment-data/
+          retention-days: 1


1-day retention may cause issues if the workflow_run event is delayed or if investigation is needed later.
make this to a week (7 days) perhaps? just check the memory limit etc. - since its a normal JSON/txt data it should be fine

bewithgaurav · 2025-12-22T11:21:22Z

.github/workflows/pr-code-coverage.yml

+          }
+          EOF
+          cat coverage-comment-data/pr-info.json
+


valid case, can add this

bewithgaurav · 2025-12-22T11:25:17Z

.github/workflows/post-coverage-comment.yml

+          number: ${{ env.PR_NUMBER }}
+          header: Code Coverage Report
+          message: |
+            # 📊 Code Coverage Report


the template is repeated two times across workflows, to streamline this you can use a composite GH action, below is a suggested approach:

Create a local composite action that both workflows call:

.github/ ├── actions/ │ └── post-coverage-comment/ │ └── action.yml # Shared comment logic ├── workflows/ │ ├── pr-code-coverage.yml │ └── forked-pr-coverage-comment.yml

.github/actions/post-coverage-comment/action.yml:

name: 'Post Coverage Comment' description: 'Posts a coverage report comment to a PR' inputs: pr_number: required: true coverage_percentage: required: true covered_lines: required: true total_lines: required: true patch_coverage_pct: required: true low_coverage_files: required: true patch_coverage_summary: required: true ado_url: required: true runs: using: 'composite' steps: - uses: marocchino/sticky-pull-request-comment@v2 with: number: ${{ inputs.pr_number }} header: Code Coverage Report message: | # 📊 Code Coverage Report <table> <tr> <td align="center" width="200"> ### 🔥 Diff Coverage ### **${{ inputs.patch_coverage_pct }}** ... </td> </tr> </table>

Usage in both workflows:

- uses: ./.github/actions/post-coverage-comment with: pr_number: ${{ github.event.pull_request.number }} coverage_percentage: ${{ env.COVERAGE_PERCENTAGE }} # ... etc

Pros: Simple, no external dependencies, version controlled with repo
Cons: Need to checkout repo first in workflow_run

bewithgaurav · 2025-12-22T11:33:32Z

.github/workflows/pr-code-coverage.yml

+          cat coverage-comment-data/pr-info.json
+
+      - name: Upload coverage comment data
+        uses: actions/upload-artifact@v4


the artifact is uploaded for all PRs, but post-coverage-comment.yml runs on every workflow_run completion
this means same-repo PRs will again run the comment flow - one posted directly by pr-code-coverage.yml, and another by post-coverage-comment.yml - although this is a sticky comment no duplicate comments will be posted.
However, we dont need to kick post-coverage/forked-pr-code-coverage flow for normal PRs
to avoid this, lets only upload the artifact for forked PRs - else this will be a problem:
This ensures post-coverage-comment.yml only has data to work with when it's actually needed (forked PRs).

gargsaumya added 2 commits December 16, 2025 09:47

testing code coverage failure

cdb94b2

testing

b2a1ac6

Copilot AI review requested due to automatic review settings December 16, 2025 09:40

Copilot started reviewing on behalf of gargsaumya December 16, 2025 09:45 View session

gargsaumya changed the title ~~Test: Forked PR coverage comment workflow~~ FEAT: Forked PR coverage comment workflow Dec 16, 2025

Copilot AI reviewed Dec 16, 2025

View reviewed changes

testing

7e50a98

gargsaumya force-pushed the saumya/code-cov-test branch from 9011198 to 7e50a98 Compare December 17, 2025 10:48

bewithgaurav requested changes Dec 22, 2025

View reviewed changes

addressed review comments

d126932

gargsaumya force-pushed the saumya/code-cov-test branch from 4b724e0 to d126932 Compare December 23, 2025 07:40

Merge branch 'main' into saumya/code-cov-test

b78baf5



	- name: Validate coverage comment JSON
	run: jq . coverage-comment-data/pr-info.json

FEAT: Forked PR coverage comment workflow #375

Are you sure you want to change the base?

FEAT: Forked PR coverage comment workflow #375

Uh oh!

Conversation

gargsaumya commented Dec 16, 2025 • edited by azure-boards bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Work Item / Issue Reference

Summary

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

Uh oh!

bewithgaurav left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

gargsaumya commented Dec 16, 2025 •

edited by azure-boards bot

Loading