From fa04d3fc98d4a9f1e8c10f78b6946637614f49ba Mon Sep 17 00:00:00 2001 From: Tadas Labudis Date: Tue, 12 May 2026 08:01:30 +0100 Subject: [PATCH 1/8] Improve issue triage template with structured triage flow - Add set-issue-type and set-issue-field safe outputs - Restructure flow: gather context, quality check, triage, apply - Add quality gate for incomplete issues (ask for missing info) - Add issue type assignment (skip if already set) - Add issue field assignment (skip if already set) - Add duplicate detection with confidence levels (duplicate vs related) - Add structured triage report format with assessment table - Add label safety warning (labels can trigger automation) - Add explicit guardrails: don't invent context, don't overwrite metadata - Clarify comment audience is maintainers, not issue authors - Keep web search, debugging strategies, sub-task breakdown - Update docs to reflect new capabilities Inspired by patterns from the GitHub Issues triage agent. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- docs/issue-triage.md | 18 ++--- workflows/issue-triage.md | 138 ++++++++++++++++++++++++++------------ 2 files changed, 104 insertions(+), 52 deletions(-) diff --git a/docs/issue-triage.md b/docs/issue-triage.md index 6b0bc266..2f4ddf77 100644 --- a/docs/issue-triage.md +++ b/docs/issue-triage.md @@ -4,7 +4,7 @@ **Automatically triage issues when they are created or reopened** -The [Issue Triage workflow](../workflows/issue-triage.md?plain=1) runs when issues are created or reopened to analyze content, check related items, categorize, add labels, and post triage comments. +The [Issue Triage workflow](../workflows/issue-triage.md?plain=1) runs when issues are created or reopened to analyze content, check related items, set issue type and fields, add labels, detect duplicates, and post structured triage reports. ## Installation @@ -22,11 +22,11 @@ This walks you through adding the workflow to your repository. ```mermaid graph LR - A[Issue Created/Reopened] --> B[Analyze Content] - B --> C[Check Related Items] - C --> D[Categorize Issue] - D --> E[Add Labels] - E --> F[Post Triage Comment] + A[Issue Created/Reopened] --> B[Gather Context] + B --> C[Spam & Quality Check] + C --> D[Triage: Type, Labels, Fields] + D --> E[Detect Duplicates] + E --> F[Post Triage Report] ``` The workflow may search for relevant documentation, error messages, or similar issues online to assist with triage. @@ -41,8 +41,8 @@ This workflow requires no configuration and works out of the box. You can custom After editing run `gh aw compile` to update the workflow and commit all changes to the default branch. -### Human in the Loop +### Human in the loop -- Review triage comments for accuracy -- Validate label assignments and priority assessments +- Review triage reports for accuracy +- Validate label, type, and field assignments - Override or adjust triage decisions when needed diff --git a/workflows/issue-triage.md b/workflows/issue-triage.md index 28c96ce9..b5abc8a2 100644 --- a/workflows/issue-triage.md +++ b/workflows/issue-triage.md @@ -1,10 +1,10 @@ --- description: | Intelligent issue triage assistant that processes new and reopened issues. - Analyzes issue content, selects appropriate labels, detects spam, gathers context - from similar issues, and provides analysis notes including debugging strategies, - reproduction steps, and resource links. Helps maintainers quickly understand and - prioritize incoming issues. + Analyzes issue content, detects spam and incomplete reports, selects appropriate + labels, sets issue type and fields, detects duplicates, and provides structured + triage reports with debugging strategies and resource links. Helps maintainers + quickly understand and prioritize incoming issues. on: issues: @@ -23,6 +23,10 @@ safe-outputs: add-labels: max: 5 add-comment: + set-issue-type: + max: 1 + set-issue-field: + max: 5 tools: web-fetch: @@ -37,54 +41,102 @@ timeout-minutes: 10 -You're a triage assistant for GitHub issues. Your task is to analyze issue #${{ github.event.issue.number }} and perform some initial triage tasks related to that issue. +You are a triage assistant for GitHub issues. Your task is to analyze issue #${{ github.event.issue.number }}, categorize it with the right metadata, and help maintainers act quickly. Your triage comments are written for maintainers reviewing the triage, not for the issue author. + +Do not make assumptions beyond what the issue content supports. Do not invent missing context. + +## Step 1: Gather context + +1. Retrieve the issue content using the `get_issue` tool. +2. Fetch any comments on the issue using the `get_issue_comments` tool. +3. Fetch the list of labels available in this repository using `gh label list`. +4. Search for similar issues using the `search_issues` tool. + +## Step 2: Spam and quality check + +**Spam and invalid issues:** If the issue is obviously spam, bot-generated, gibberish, or a test issue, add a one-sentence comment explaining why and exit the workflow. Do not apply metadata to these issues. + +**Incomplete issues:** If the issue lacks enough detail for meaningful triage, add a comment that politely asks the author to provide the missing information: +- For bugs: steps to reproduce, expected vs actual behavior, logs/errors, environment details. +- For other issue types: equivalent details that would make the report actionable. + +Be specific about what is missing and why it is needed. Do not attempt to apply labels, type, or fields to incomplete issues. + +If the issue has sufficient detail, proceed to Step 3. + +## Step 3: Triage + +### 3a: Set issue type + +- If the issue already has a type set, do not change it. +- Otherwise, determine the single best issue type (e.g., Bug, Feature, Task). +- If no type is clearly supported by the issue content, leave it unset and note what is missing. + +### 3b: Select labels + +- Be cautious with labels; they can trigger automation in many repositories. +- Choose labels that accurately reflect the issue's nature from the repository's available labels. +- Select priority labels if you can determine urgency (high-priority, med-priority, low-priority). +- Consider platform labels (android, ios) if applicable. +- Do not apply labels that do not exist in the repository. +- If no labels are clearly applicable, do not apply any. +- It is better to under-label than to speculatively add labels. + +### 3c: Set issue fields + +- If the issue already has a value set for a field, do not overwrite it. +- Only set fields when you can confidently determine the value from the issue content. +- Do not make up field values. + +### 3d: Detect duplicates and related issues + +- Review the similar issues found in Step 1. +- Classify matches as: + - **Duplicate** (high confidence): the issue describes the same problem as an existing open issue. Include up to 3. + - **Related**: similar domain or adjacent problem, but not a duplicate. Include up to 3. +- If no similar issues are found, state that explicitly in your report. + +### 3e: Additional analysis + +- Write notes, debugging strategies, and/or reproduction steps relevant to the issue. +- Search the web for relevant documentation, error messages, or known solutions if applicable. +- Suggest resources or links that might help resolve the issue. +- If appropriate, break the issue down into sub-tasks with a checklist. + +## Step 4: Apply results + +Apply all triage results: +- Use `set_issue_type` to set the issue type (if determined). +- Use `update_issue` to apply labels. +- Use `set_issue_field` to set any issue fields (if determined). +- Add an issue comment with your triage report using the format below. -1. Select appropriate labels for the issue from the provided list. +## Comment format -2. Retrieve the issue content using the `get_issue` tool. If the issue is obviously spam, or generated by bot, or something else that is not an actual issue to be worked on, then add an issue comment to the issue with a one-sentence analysis and exit the workflow. +Use this structure for the triage comment. Use collapsed sections to keep it tidy. -3. Next, use the GitHub tools to gather additional context about the issue: +```markdown +## 🎯 Triage report - - Fetch the list of labels available in this repository. Use 'gh label list' bash command to fetch the labels. This will give you the labels you can use for triaging issues. - - Fetch any comments on the issue using the `get_issue_comments` tool - - Find similar issues if needed using the `search_issues` tool - - List the issues to see other open issues in the repository using the `list_issues` tool +{2-3 sentence summary to help a maintainer quickly grasp the issue.} -4. Analyze the issue content, considering: +### 📊 Assessment - - The issue title and description - - The type of issue (bug report, feature request, question, etc.) - - Technical areas mentioned - - Severity or priority indicators - - User impact - - Components affected +| Dimension | Value | Reasoning | +|---|---|---| +| **Type** | [value or "unchanged"] | [brief] | +| **Labels** | [values or "none"] | [brief] | +| **[Field name]** | [value] | [brief] | -5. Write notes, ideas, nudges, resource links, debugging strategies and/or reproduction steps for the team to consider relevant to the issue. +### 🔗 Similar issues -6. Select appropriate labels from the available labels list provided above: +- issue-url (duplicate/related) — [brief explanation] - - Choose labels that accurately reflect the issue's nature - - Be specific but comprehensive - - Select priority labels if you can determine urgency (high-priority, med-priority, or low-priority) - - Consider platform labels (android, ios) if applicable - - Search for similar issues, and if you find similar issues consider using a "duplicate" label if appropriate. Only do so if the issue is a duplicate of another OPEN issue. - - Only select labels from the provided list above - - It's okay to not add any labels if none are clearly applicable +
💡 Notes and suggestions -7. Apply the selected labels: +{Debugging strategies, reproduction steps, resource links, sub-task checklists, nudges for the team.} - - Use the `update_issue` tool to apply the labels to the issue - - DO NOT communicate directly with users - - If no labels are clearly applicable, do not apply any labels +
+``` -8. If very confident, add an issue comment to the issue with your analysis: - - Start with "🎯 Agentic Issue Triage" - - Provide a brief summary of the issue - - Mention any relevant details that might help the team understand the issue better - - Include any debugging strategies or reproduction steps if applicable - - Suggest resources or links that might be helpful for resolving the issue or learning skills related to the issue or the particular area of the codebase affected by it - - Mention any nudges or ideas that could help the team in addressing the issue - - If you have possible reproduction steps, include them in the comment - - If you have any debugging strategies, include them in the comment - - If appropriate break the issue down to sub-tasks and write a checklist of things to do. - - Use collapsed-by-default sections in the GitHub markdown to keep the comment tidy. Collapse all sections except the short main summary at the top. +If no similar issues were found, omit the "Similar issues" section. If there are no notes to add, omit the collapsed section. From 953c036e47b5699ae34f361aefdbe86cb12234aa Mon Sep 17 00:00:00 2001 From: Tadas Labudis Date: Tue, 12 May 2026 08:03:13 +0100 Subject: [PATCH 2/8] Remove set-issue-field safe output (not yet supported by compiler) Keep field-related instructions in the prompt body so the agent can still recommend field values in the triage report. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- workflows/issue-triage.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/workflows/issue-triage.md b/workflows/issue-triage.md index b5abc8a2..fb622786 100644 --- a/workflows/issue-triage.md +++ b/workflows/issue-triage.md @@ -25,8 +25,6 @@ safe-outputs: add-comment: set-issue-type: max: 1 - set-issue-field: - max: 5 tools: web-fetch: From 9582067e80348d7a8b53360c79a0052ebddebeb9 Mon Sep 17 00:00:00 2001 From: Tadas Labudis Date: Tue, 12 May 2026 08:22:17 +0100 Subject: [PATCH 3/8] Add duplicate label action, remove unsupported set_issue_field - Instruct agent to apply duplicate label when high-confidence duplicate is detected and the label exists in the repo - Remove set_issue_field from Step 4 (compiler does not support it yet) - Keep field guidance in Step 3c for comment-only recommendations Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- workflows/issue-triage.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/workflows/issue-triage.md b/workflows/issue-triage.md index fb622786..a4afebdc 100644 --- a/workflows/issue-triage.md +++ b/workflows/issue-triage.md @@ -92,6 +92,7 @@ If the issue has sufficient detail, proceed to Step 3. - Classify matches as: - **Duplicate** (high confidence): the issue describes the same problem as an existing open issue. Include up to 3. - **Related**: similar domain or adjacent problem, but not a duplicate. Include up to 3. +- If a high-confidence duplicate is found and the repository has a `duplicate` label, apply it. - If no similar issues are found, state that explicitly in your report. ### 3e: Additional analysis @@ -106,7 +107,6 @@ If the issue has sufficient detail, proceed to Step 3. Apply all triage results: - Use `set_issue_type` to set the issue type (if determined). - Use `update_issue` to apply labels. -- Use `set_issue_field` to set any issue fields (if determined). - Add an issue comment with your triage report using the format below. ## Comment format From 56913369f2bd7ee35c29b1ff6beb75db827ba4e2 Mon Sep 17 00:00:00 2001 From: Tadas Labudis Date: Tue, 12 May 2026 08:35:39 +0100 Subject: [PATCH 4/8] Add auto-close spam, coding agent suitability, labels toolset - Add close-issue safe output to auto-close spam as not_planned - Apply invalid/spam label to spam issues - Apply needs-info/question label to incomplete issues - Add coding agent suitability assessment (Suitable/Needs more info/Not suitable) - Replace gh label list bash with labels toolset for richer metadata - Update Step 4 with close_issue action Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- workflows/issue-triage.md | 28 +++++++++++++++++++++++----- 1 file changed, 23 insertions(+), 5 deletions(-) diff --git a/workflows/issue-triage.md b/workflows/issue-triage.md index a4afebdc..1c9ea348 100644 --- a/workflows/issue-triage.md +++ b/workflows/issue-triage.md @@ -25,11 +25,15 @@ safe-outputs: add-comment: set-issue-type: max: 1 + close-issue: + target: "triggering" + state-reason: "not_planned" + max: 1 tools: web-fetch: github: - toolsets: [issues] + toolsets: [issues, labels] min-integrity: none # This workflow is allowed to examine and comment on any issues timeout-minutes: 10 @@ -47,18 +51,23 @@ Do not make assumptions beyond what the issue content supports. Do not invent mi 1. Retrieve the issue content using the `get_issue` tool. 2. Fetch any comments on the issue using the `get_issue_comments` tool. -3. Fetch the list of labels available in this repository using `gh label list`. +3. Fetch the list of labels available in this repository using the `list_labels` tool. 4. Search for similar issues using the `search_issues` tool. ## Step 2: Spam and quality check -**Spam and invalid issues:** If the issue is obviously spam, bot-generated, gibberish, or a test issue, add a one-sentence comment explaining why and exit the workflow. Do not apply metadata to these issues. +**Spam and invalid issues:** If the issue is obviously spam, bot-generated, gibberish, or a test issue: +- Add a one-sentence comment explaining why. +- Apply the `invalid` or `spam` label if one exists in the repository. +- Close the issue as "not planned." +- Do not apply any other metadata. Exit the workflow. **Incomplete issues:** If the issue lacks enough detail for meaningful triage, add a comment that politely asks the author to provide the missing information: - For bugs: steps to reproduce, expected vs actual behavior, logs/errors, environment details. - For other issue types: equivalent details that would make the report actionable. +- Apply a `needs-info` or `question` label if one exists in the repository. -Be specific about what is missing and why it is needed. Do not attempt to apply labels, type, or fields to incomplete issues. +Be specific about what is missing and why it is needed. Do not attempt to apply type or other labels to incomplete issues. If the issue has sufficient detail, proceed to Step 3. @@ -95,7 +104,14 @@ If the issue has sufficient detail, proceed to Step 3. - If a high-confidence duplicate is found and the repository has a `duplicate` label, apply it. - If no similar issues are found, state that explicitly in your report. -### 3e: Additional analysis +### 3e: Assess coding agent suitability + +Assess whether the issue is suitable for automated coding agent assignment: +- **Suitable**: clear requirements, sufficient context, well-defined success criteria, self-contained scope. +- **Needs more info**: potentially suitable but missing details needed to start. +- **Not suitable**: requires investigation, design decisions, extensive coordination, or policy/architectural choices. + +### 3f: Additional analysis - Write notes, debugging strategies, and/or reproduction steps relevant to the issue. - Search the web for relevant documentation, error messages, or known solutions if applicable. @@ -107,6 +123,7 @@ If the issue has sufficient detail, proceed to Step 3. Apply all triage results: - Use `set_issue_type` to set the issue type (if determined). - Use `update_issue` to apply labels. +- Use `close_issue` to close the issue if it is spam (state reason: "not planned"). - Add an issue comment with your triage report using the format below. ## Comment format @@ -125,6 +142,7 @@ Use this structure for the triage comment. Use collapsed sections to keep it tid | **Type** | [value or "unchanged"] | [brief] | | **Labels** | [values or "none"] | [brief] | | **[Field name]** | [value] | [brief] | +| **Coding agent** | [Suitable / Needs more info / Not suitable] | [brief] | ### 🔗 Similar issues From 5673dede4782f84e2344ea56a34fd6b45289d0c8 Mon Sep 17 00:00:00 2001 From: Tadas Labudis Date: Tue, 12 May 2026 08:40:46 +0100 Subject: [PATCH 5/8] Remove issue fields references (not supported yet) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- docs/issue-triage.md | 4 ++-- workflows/issue-triage.md | 10 ++-------- 2 files changed, 4 insertions(+), 10 deletions(-) diff --git a/docs/issue-triage.md b/docs/issue-triage.md index 2f4ddf77..de6ac38f 100644 --- a/docs/issue-triage.md +++ b/docs/issue-triage.md @@ -4,7 +4,7 @@ **Automatically triage issues when they are created or reopened** -The [Issue Triage workflow](../workflows/issue-triage.md?plain=1) runs when issues are created or reopened to analyze content, check related items, set issue type and fields, add labels, detect duplicates, and post structured triage reports. +The [Issue Triage workflow](../workflows/issue-triage.md?plain=1) runs when issues are created or reopened to analyze content, check related items, set issue type, add labels, detect duplicates, and post structured triage reports. ## Installation @@ -24,7 +24,7 @@ This walks you through adding the workflow to your repository. graph LR A[Issue Created/Reopened] --> B[Gather Context] B --> C[Spam & Quality Check] - C --> D[Triage: Type, Labels, Fields] + C --> D[Triage: Type, Labels] D --> E[Detect Duplicates] E --> F[Post Triage Report] ``` diff --git a/workflows/issue-triage.md b/workflows/issue-triage.md index 1c9ea348..f088e94d 100644 --- a/workflows/issue-triage.md +++ b/workflows/issue-triage.md @@ -2,7 +2,7 @@ description: | Intelligent issue triage assistant that processes new and reopened issues. Analyzes issue content, detects spam and incomplete reports, selects appropriate - labels, sets issue type and fields, detects duplicates, and provides structured + labels, sets issue type, detects duplicates, and provides structured triage reports with debugging strategies and resource links. Helps maintainers quickly understand and prioritize incoming issues. @@ -89,13 +89,7 @@ If the issue has sufficient detail, proceed to Step 3. - If no labels are clearly applicable, do not apply any. - It is better to under-label than to speculatively add labels. -### 3c: Set issue fields - -- If the issue already has a value set for a field, do not overwrite it. -- Only set fields when you can confidently determine the value from the issue content. -- Do not make up field values. - -### 3d: Detect duplicates and related issues +### 3c: Detect duplicates and related issues - Review the similar issues found in Step 1. - Classify matches as: From fc5d3030419fb3ac0d9920456c966c1d6447dab1 Mon Sep 17 00:00:00 2001 From: Tadas Labudis Date: Tue, 12 May 2026 08:43:41 +0100 Subject: [PATCH 6/8] Remove field name row from assessment table Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- workflows/issue-triage.md | 1 - 1 file changed, 1 deletion(-) diff --git a/workflows/issue-triage.md b/workflows/issue-triage.md index f088e94d..26073989 100644 --- a/workflows/issue-triage.md +++ b/workflows/issue-triage.md @@ -135,7 +135,6 @@ Use this structure for the triage comment. Use collapsed sections to keep it tid |---|---|---| | **Type** | [value or "unchanged"] | [brief] | | **Labels** | [values or "none"] | [brief] | -| **[Field name]** | [value] | [brief] | | **Coding agent** | [Suitable / Needs more info / Not suitable] | [brief] | ### 🔗 Similar issues From 27317886814688c84fd6993302b0091015b25b3d Mon Sep 17 00:00:00 2001 From: Tadas Labudis Date: Tue, 12 May 2026 12:12:29 +0100 Subject: [PATCH 7/8] Fix spam double-comment: single close comment, no triage report Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- workflows/issue-triage.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/workflows/issue-triage.md b/workflows/issue-triage.md index 26073989..df954c0b 100644 --- a/workflows/issue-triage.md +++ b/workflows/issue-triage.md @@ -57,10 +57,9 @@ Do not make assumptions beyond what the issue content supports. Do not invent mi ## Step 2: Spam and quality check **Spam and invalid issues:** If the issue is obviously spam, bot-generated, gibberish, or a test issue: -- Add a one-sentence comment explaining why. - Apply the `invalid` or `spam` label if one exists in the repository. -- Close the issue as "not planned." -- Do not apply any other metadata. Exit the workflow. +- Close the issue as "not planned" with a one-sentence reason (e.g., "Closing as spam."). No triage report, no assessment table. +- Do not apply any other metadata. **Stop here; do not continue to Steps 3 or 4.** **Incomplete issues:** If the issue lacks enough detail for meaningful triage, add a comment that politely asks the author to provide the missing information: - For bugs: steps to reproduce, expected vs actual behavior, logs/errors, environment details. From 30306c05a57eb29a7edc73e49f8d76b9cddfd318 Mon Sep 17 00:00:00 2001 From: Don Syme Date: Tue, 12 May 2026 14:55:42 +0100 Subject: [PATCH 8/8] Apply suggestion from @reneexeener Co-authored-by: Renee Xu --- workflows/issue-triage.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/workflows/issue-triage.md b/workflows/issue-triage.md index df954c0b..94667744 100644 --- a/workflows/issue-triage.md +++ b/workflows/issue-triage.md @@ -51,7 +51,7 @@ Do not make assumptions beyond what the issue content supports. Do not invent mi 1. Retrieve the issue content using the `get_issue` tool. 2. Fetch any comments on the issue using the `get_issue_comments` tool. -3. Fetch the list of labels available in this repository using the `list_labels` tool. +3. Fetch the list of labels available in this repository using the `list_label` tool. 4. Search for similar issues using the `search_issues` tool. ## Step 2: Spam and quality check