Skip to content

Nightly report: post details as follow-up comments instead of truncating #1239

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
aaronpowell merged 1 commit into from
Mar 31, 2026
Merged

Nightly report: post details as follow-up comments instead of truncating #1239

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
194 changes: 112 additions & 82 deletions .github/workflows/skill-quality-report.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -206,93 +206,95 @@ jobs:

const title = `Skill Quality Report — ${today}`;

const body = [
`# ${title}`,
'',
`**${skillDirs.length} skills** and **${agentFiles.length} agents** scanned.`,
'',
`| Severity | Count |`,
`|----------|-------|`,
const annotatedSkills = annotateWithAuthors(skillsOutput, 'skill');
const annotatedAgents = annotateWithAuthors(agentsOutput, 'agent');

// ── Body size management ──────────────────────────────
// GitHub body limit is ~65536 UTF-8 bytes for both
// Discussions and Issues. When the full report fits, we
// inline everything. When it doesn't, the body gets a
// compact summary and the verbose sections are written to
// separate files that get posted as follow-up comments.
const MAX_BYTES = 65000; // leave margin

function makeDetailsBlock(heading, summary, content) {
return [
`## ${heading}`, '',
'<details>',
`<summary>${summary}</summary>`, '',
'```', content, '```', '',
'</details>',
].join('\n');
}

const summaryLines = [
`# ${title}`, '',
`**${skillDirs.length} skills** and **${agentFiles.length} agents** scanned.`, '',
'| Severity | Count |',
'|----------|-------|',
`| ⛔ Errors | ${errorCount} |`,
`| ⚠️ Warnings | ${warningCount} |`,
`| ℹ️ Advisories | ${advisoryCount} |`,
'',
'---',
'',
'## Skills',
'',
'<details>',
'<summary>Full skill-validator output for skills</summary>',
'',
'```',
annotateWithAuthors(skillsOutput, 'skill'),
'```',
'',
'</details>',
'',
'## Agents',
'',
'<details>',
'<summary>Full skill-validator output for agents</summary>',
'',
'```',
annotateWithAuthors(agentsOutput, 'agent'),
'```',
'',
'</details>',
'',
`| ℹ️ Advisories | ${advisoryCount} |`, '',
'---',
'',
`_Generated by the [Skill Validator nightly scan](https://github.com/${context.repo.owner}/${context.repo.repo}/actions/workflows/skill-quality-report.yml)._`,
].join('\n');

core.setOutput('title', title);
core.setOutput('body_file', 'report-body.md');

// GitHub Issues/Discussions enforce a body size limit on the
// UTF-8 payload (~65536 bytes). Use byte-based limits and prefer
// shrinking verbose <details> sections to keep markdown valid.
const MAX_BODY_BYTES = 65000; // leave some margin

function shrinkDetailsSections(markdown) {
return markdown.replace(
/<details([\s\S]*?)>[\s\S]*?<\/details>/g,
(match, attrs) => {
const placeholder = '\n<summary>Details truncated</summary>\n\n' +
"> Full output was truncated to fit GitHub's body size limit. " +
'See the workflow run for complete output.\n';
return `<details${attrs}>${placeholder}</details>`;
];
const footer = `\n---\n\n_Generated by the [Skill Validator nightly scan](https://github.com/${context.repo.owner}/${context.repo.repo}/actions/workflows/skill-quality-report.yml)._`;

const skillsBlock = makeDetailsBlock('Skills', 'Full skill-validator output for skills', annotatedSkills);
const agentsBlock = makeDetailsBlock('Agents', 'Full skill-validator output for agents', annotatedAgents);

// Try full inline body first
const fullBody = summaryLines.join('\n') + '\n\n' + skillsBlock + '\n\n' + agentsBlock + footer;

const commentParts = []; // overflow comment files

let finalBody;
if (Buffer.byteLength(fullBody, 'utf8') <= MAX_BYTES) {
finalBody = fullBody;
} else {
// Details won't fit inline — move them to follow-up comments
const bodyNote = '\n\n> **Note:** Detailed output is posted in the comments below (too large for the discussion body).\n';
finalBody = summaryLines.join('\n') + bodyNote + footer;

// Split each section into ≤65 KB chunks
function chunkContent(label, content) {
const prefix = `## ${label}\n\n\`\`\`\n`;
const suffix = '\n```';
const overhead = Buffer.byteLength(prefix + suffix, 'utf8');
const budget = MAX_BYTES - overhead;

const buf = Buffer.from(content, 'utf8');
if (buf.length <= budget) {
return [prefix + content + suffix];
}
);
}
const parts = [];
let offset = 0;
let partNum = 1;
while (offset < buf.length) {
const slice = buf.slice(offset, offset + budget).toString('utf8');
// Remove trailing replacement char from mid-codepoint cut
const clean = slice.replace(/\uFFFD$/, '');
const hdr = `## ${label} (part ${partNum})\n\n\`\`\`\n`;
parts.push(hdr + clean + suffix);
Comment on lines +259 to +277
Copy link

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

chunkContent() computes budget using prefix/suffix overhead, but when splitting it actually uses a larger header (hdr includes (part N)). That means each chunk can exceed MAX_BYTES by the extra header bytes, risking GraphQL failures due to comment body size. Compute the per-part budget using the actual hdr + suffix byte length (or reserve for the worst-case header size) before slicing.

This issue also appears on line 358 of the same file.

Copilot uses AI. Check for mistakes.
offset += Buffer.byteLength(clean, 'utf8');
partNum++;
}
return parts;
}

function trimToByteLimit(str, maxBytes) {
const buf = Buffer.from(str, 'utf8');
if (buf.length <= maxBytes) return str;
// Slice bytes and decode, which safely handles multi-byte chars
return buf.slice(0, maxBytes).toString('utf8').replace(/\uFFFD$/, '');
commentParts.push(...chunkContent('Skills', annotatedSkills));
commentParts.push(...chunkContent('Agents', annotatedAgents));
}

const truncNote = '\n\n> **Note:** Output was truncated to fit GitHub\'s body size limit. See the [workflow run](https://github.com/' + context.repo.owner + '/' + context.repo.repo + '/actions/workflows/skill-quality-report.yml) for full output.\n';
const truncNoteBytes = Buffer.byteLength(truncNote, 'utf8');

let finalBody = body;

if (Buffer.byteLength(finalBody, 'utf8') > MAX_BODY_BYTES) {
// First try: collapse <details> sections to reduce size
finalBody = shrinkDetailsSections(finalBody);
}
core.setOutput('title', title);
core.setOutput('body_file', 'report-body.md');

if (Buffer.byteLength(finalBody, 'utf8') > MAX_BODY_BYTES) {
// Last resort: hard byte-trim + truncation note
finalBody = trimToByteLimit(finalBody, MAX_BODY_BYTES - truncNoteBytes);
}
fs.writeFileSync('report-body.md', finalBody);

if (Buffer.byteLength(finalBody, 'utf8') < Buffer.byteLength(body, 'utf8')) {
finalBody += truncNote;
// Write overflow comment parts as numbered files
for (let i = 0; i < commentParts.length; i++) {
fs.writeFileSync(`report-comment-${i}.md`, commentParts[i]);
}

fs.writeFileSync('report-body.md', finalBody);
core.setOutput('comment_count', String(commentParts.length));

# ── Create Discussion (preferred) or Issue (fallback) ────────
- name: Create Discussion
Expand All @@ -304,6 +306,7 @@ jobs:
const fs = require('fs');
const title = '${{ steps.report.outputs.title }}'.replace(/'/g, "\\'");
const body = fs.readFileSync('report-body.md', 'utf8');
const commentCount = parseInt('${{ steps.report.outputs.comment_count }}' || '0', 10);

// Find the "Skill Quality Reports" category
const categoriesResult = await github.graphql(`
Expand Down Expand Up @@ -331,15 +334,15 @@ jobs:
return;
}

await github.graphql(`
const result = await github.graphql(`
mutation($repoId: ID!, $categoryId: ID!, $title: String!, $body: String!) {
createDiscussion(input: {
repositoryId: $repoId,
categoryId: $categoryId,
title: $title,
body: $body
}) {
discussion { url }
discussion { id url }
}
}
`, {
Expand All @@ -349,7 +352,24 @@ jobs:
body: body,
});

console.log('Discussion created successfully.');
const discussionId = result.createDiscussion.discussion.id;
console.log(`Discussion created: ${result.createDiscussion.discussion.url}`);

// Post overflow detail comments
for (let i = 0; i < commentCount; i++) {
const commentBody = fs.readFileSync(`report-comment-${i}.md`, 'utf8');
await github.graphql(`
mutation($discussionId: ID!, $body: String!) {
addDiscussionComment(input: {
discussionId: $discussionId,
body: $body
}) {
comment { id }
}
}
`, { discussionId, body: commentBody });
console.log(`Posted detail comment ${i + 1}/${commentCount}`);
Comment on lines +359 to +371
Copy link

Copilot AI Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The step is continue-on-error: true and the script posts follow-up comments after creating the discussion. If any addDiscussionComment call fails (size/rate limits/transient error), the step will fail and the fallback issue step will run even though the discussion was already created, leading to duplicate report posts. Consider catching errors during comment posting (and logging) so the step can still succeed once the discussion exists, or otherwise gate the fallback on whether the discussion was created.

Suggested change
for (let i = 0; i < commentCount; i++) {
const commentBody = fs.readFileSync(`report-comment-${i}.md`, 'utf8');
await github.graphql(`
mutation($discussionId: ID!, $body: String!) {
addDiscussionComment(input: {
discussionId: $discussionId,
body: $body
}) {
comment { id }
}
}
`, { discussionId, body: commentBody });
console.log(`Posted detail comment ${i + 1}/${commentCount}`);
try {
for (let i = 0; i < commentCount; i++) {
const commentBody = fs.readFileSync(`report-comment-${i}.md`, 'utf8');
await github.graphql(`
mutation($discussionId: ID!, $body: String!) {
addDiscussionComment(input: {
discussionId: $discussionId,
body: $body
}) {
comment { id }
}
}
`, { discussionId, body: commentBody });
console.log(`Posted detail comment ${i + 1}/${commentCount}`);
}
} catch (error) {
core.warning(`One or more discussion comments failed to post: ${error.message || error}`);
console.error(error);

Copilot uses AI. Check for mistakes.
}

- name: Fallback — Create Issue
if: steps.create-discussion.outcome == 'failure'
Expand All @@ -358,7 +378,17 @@ jobs:
run: |
# Create label if it doesn't exist (ignore errors if it already exists)
gh label create "skill-quality" --description "Automated skill quality reports" --color "d4c5f9" 2>/dev/null || true
gh issue create \
ISSUE_URL=$(gh issue create \
--title "${{ steps.report.outputs.title }}" \
--body-file report-body.md \
--label "skill-quality"
--label "skill-quality")
echo "Created issue: $ISSUE_URL"

# Post overflow detail comments on the issue
COMMENT_COUNT=${{ steps.report.outputs.comment_count }}
for i in $(seq 0 $(( ${COMMENT_COUNT:-0} - 1 ))); do
if [ -f "report-comment-${i}.md" ]; then
gh issue comment "$ISSUE_URL" --body-file "report-comment-${i}.md"
echo "Posted detail comment $((i+1))/${COMMENT_COUNT}"
fi
done
Loading