feat: add security-review skill for AI-powered codebase vulnerability scanning#1210
Closed
Mrigank005 wants to merge 2 commits intogithub:stagedfrom
Closed
feat: add security-review skill for AI-powered codebase vulnerability scanning#1210Mrigank005 wants to merge 2 commits intogithub:stagedfrom
Mrigank005 wants to merge 2 commits intogithub:stagedfrom
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new set of Copilot plugin content (agents/skills/reference material) and standardizes plugin manifests, including registering a new security-review skill in the skills index.
Changes:
- Standardize many
plugin.jsonentries (remove trailing slashes; in several plugins switchagentsto point at the whole./agentsdirectory). - Add multiple new agents/skills and reference docs across several plugins.
- Update
docs/README.skills.mdto include the newsecurity-reviewskill entry.
Reviewed changes
Copilot reviewed 116 out of 312 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| plugins/java-development/.github/plugin/plugin.json | Normalizes skill paths (removes trailing slashes). |
| plugins/go-mcp-development/agents/go-mcp-expert.md | Adds a Go MCP expert agent definition. |
| plugins/go-mcp-development/.github/plugin/plugin.json | Changes agent registration to ./agents; normalizes skill path. |
| plugins/gem-team/agents/gem-reviewer.md | Adds “reviewer” agent definition and workflow. |
| plugins/gem-team/agents/gem-implementer.md | Adds “implementer” agent definition and workflow. |
| plugins/gem-team/agents/gem-documentation-writer.md | Adds “documentation writer” agent definition and workflow. |
| plugins/gem-team/agents/gem-devops.md | Adds “devops” agent definition and workflow. |
| plugins/gem-team/agents/gem-browser-tester.md | Adds “browser tester” agent definition and workflow. |
| plugins/gem-team/.github/plugin/plugin.json | Changes agent registration to ./agents. |
| plugins/frontend-web-dev/skills/playwright-generate-test/SKILL.md | Adds Playwright test generation skill. |
| plugins/frontend-web-dev/skills/playwright-explore-website/SKILL.md | Adds Playwright website exploration skill. |
| plugins/frontend-web-dev/agents/electron-angular-native.md | Adds Electron/Angular/native code review agent instructions. |
| plugins/frontend-web-dev/.github/plugin/plugin.json | Changes agent registration to ./agents; normalizes skill paths. |
| plugins/flowstudio-power-automate/skills/flowstudio-power-automate-mcp/references/connection-references.md | Adds reference documentation for connection references. |
| plugins/flowstudio-power-automate/skills/flowstudio-power-automate-mcp/references/action-types.md | Adds action types lookup reference. |
| plugins/flowstudio-power-automate/skills/flowstudio-power-automate-mcp/references/MCP-BOOTSTRAP.md | Adds MCP bootstrap quick reference. |
| plugins/flowstudio-power-automate/skills/flowstudio-power-automate-debug/references/debug-workflow.md | Adds debugging workflow reference. |
| plugins/flowstudio-power-automate/skills/flowstudio-power-automate-debug/references/common-errors.md | Adds common error reference for debugging. |
| plugins/flowstudio-power-automate/skills/flowstudio-power-automate-build/references/trigger-types.md | Adds trigger templates reference. |
| plugins/flowstudio-power-automate/skills/flowstudio-power-automate-build/references/flow-schema.md | Adds flow schema reference. |
| plugins/flowstudio-power-automate/skills/flowstudio-power-automate-build/references/build-patterns.md | Adds build templates reference. |
| plugins/flowstudio-power-automate/.github/plugin/plugin.json | Normalizes skill paths (removes trailing slashes). |
| plugins/fastah-ip-geo-tools/skills/geofeed-tuner/references/snippets-python3.md | Adds Python snippets/reference for geofeed tuning. |
| plugins/fastah-ip-geo-tools/skills/geofeed-tuner/assets/small-territories.json | Adds supporting asset list for geofeed tuning. |
| plugins/fastah-ip-geo-tools/skills/geofeed-tuner/assets/example/01-user-input-rfc8805-feed.csv | Adds example input CSV asset. |
| plugins/fastah-ip-geo-tools/.github/plugin/plugin.json | Normalizes skill path (removes trailing slash). |
| plugins/edge-ai-tasks/.github/plugin/plugin.json | Changes agent registration to ./agents. |
| plugins/doublecheck/skills/doublecheck/assets/verification-report-template.md | Adds verification report template asset. |
| plugins/doublecheck/agents/doublecheck.md | Adds doublecheck verification agent definition. |
| plugins/doublecheck/.github/plugin/plugin.json | Changes agent registration to ./agents; normalizes skill path. |
| plugins/devops-oncall/skills/multi-stage-dockerfile/SKILL.md | Adds multi-stage Dockerfile guidance skill. |
| plugins/devops-oncall/agents/azure-principal-architect.md | Adds Azure principal architect agent definition. |
| plugins/devops-oncall/.github/plugin/plugin.json | Changes agent registration to ./agents; normalizes skill paths. |
| plugins/dataverse-sdk-for-python/skills/dataverse-python-usecase-builder/SKILL.md | Adds Dataverse use-case builder skill. |
| plugins/dataverse-sdk-for-python/skills/dataverse-python-quickstart/SKILL.md | Adds Dataverse Python quickstart skill. |
| plugins/dataverse-sdk-for-python/skills/dataverse-python-production-code/SKILL.md | Adds Dataverse production-code patterns skill. |
| plugins/dataverse-sdk-for-python/skills/dataverse-python-advanced-patterns/SKILL.md | Adds Dataverse advanced patterns skill. |
| plugins/dataverse-sdk-for-python/.github/plugin/plugin.json | Normalizes skill paths (removes trailing slashes). |
| plugins/database-data-management/skills/postgresql-code-review/SKILL.md | Adds PostgreSQL-specific code review skill. |
| plugins/database-data-management/agents/postgresql-dba.md | Adds PostgreSQL DBA agent definition. |
| plugins/database-data-management/agents/ms-sql-dba.md | Adds MS SQL DBA agent definition. |
| plugins/database-data-management/.github/plugin/plugin.json | Changes agent registration to ./agents; normalizes skill paths. |
| plugins/csharp-mcp-development/skills/csharp-mcp-server-generator/SKILL.md | Adds C# MCP server generator skill. |
| plugins/csharp-mcp-development/agents/csharp-mcp-expert.md | Adds C# MCP expert agent definition. |
| plugins/csharp-mcp-development/.github/plugin/plugin.json | Changes agent registration to ./agents; normalizes skill path. |
| plugins/csharp-dotnet-development/skills/dotnet-upgrade/SKILL.md | Adds .NET upgrade guidance skill. |
| plugins/csharp-dotnet-development/skills/dotnet-best-practices/SKILL.md | Adds .NET best practices skill. |
| plugins/csharp-dotnet-development/skills/csharp-xunit/SKILL.md | Adds xUnit testing best practices skill. |
| plugins/csharp-dotnet-development/skills/csharp-tunit/SKILL.md | Adds TUnit testing best practices skill. |
| plugins/csharp-dotnet-development/skills/csharp-nunit/SKILL.md | Adds NUnit testing best practices skill. |
| plugins/csharp-dotnet-development/skills/csharp-async/SKILL.md | Adds C# async best practices skill. |
| plugins/csharp-dotnet-development/skills/aspnet-minimal-api-openapi/SKILL.md | Adds Minimal API + OpenAPI guidance skill. |
| plugins/csharp-dotnet-development/agents/expert-dotnet-software-engineer.md | Adds expert .NET engineer agent definition. |
| plugins/csharp-dotnet-development/.github/plugin/plugin.json | Changes agent registration to ./agents; normalizes skill paths. |
| plugins/copilot-sdk/.github/plugin/plugin.json | Normalizes skill path (removes trailing slash). |
| plugins/context-engineering/skills/what-context-needed/SKILL.md | Adds “what context needed” skill. |
| plugins/context-engineering/skills/refactor-plan/SKILL.md | Adds refactor planning skill. |
| plugins/context-engineering/skills/context-map/SKILL.md | Adds context mapping skill. |
| plugins/context-engineering/agents/context-architect.md | Adds context architect agent definition. |
| plugins/context-engineering/.github/plugin/plugin.json | Changes agent registration to ./agents; normalizes skill paths. |
| plugins/clojure-interactive-programming/skills/remember-interactive-programming/SKILL.md | Adds interactive-programming reminder skill. |
| plugins/clojure-interactive-programming/agents/clojure-interactive-programming.md | Adds Clojure interactive programming agent definition. |
| plugins/clojure-interactive-programming/.github/plugin/plugin.json | Changes agent registration to ./agents; normalizes skill path. |
| plugins/cast-imaging/agents/cast-imaging-structural-quality-advisor.md | Adds CAST Imaging structural quality advisor agent. |
| plugins/cast-imaging/agents/cast-imaging-software-discovery.md | Adds CAST Imaging software discovery agent. |
| plugins/cast-imaging/agents/cast-imaging-impact-analysis.md | Adds CAST Imaging impact analysis agent. |
| plugins/cast-imaging/.github/plugin/plugin.json | Changes agent registration to ./agents. |
| plugins/azure-cloud-development/skills/azure-pricing/references/SERVICE-NAMES.md | Adds Azure pricing service-name reference. |
| plugins/azure-cloud-development/skills/azure-pricing/references/REGIONS.md | Adds Azure pricing regions reference. |
| plugins/azure-cloud-development/skills/azure-pricing/references/COST-ESTIMATOR.md | Adds Azure cost estimation formulas reference. |
| plugins/azure-cloud-development/skills/azure-pricing/references/COPILOT-STUDIO-RATES.md | Adds Copilot Studio rates snapshot reference. |
| plugins/azure-cloud-development/skills/azure-pricing/SKILL.md | Adds Azure pricing skill definition. |
| plugins/azure-cloud-development/agents/terraform-azure-planning.md | Adds Terraform planning agent definition. |
| plugins/azure-cloud-development/agents/terraform-azure-implement.md | Adds Terraform implementation agent definition. |
| plugins/azure-cloud-development/agents/azure-verified-modules-terraform.md | Adds AVM Terraform agent definition. |
| plugins/azure-cloud-development/agents/azure-verified-modules-bicep.md | Adds AVM Bicep agent definition. |
| plugins/azure-cloud-development/agents/azure-saas-architect.md | Adds Azure SaaS architect agent definition. |
| plugins/azure-cloud-development/agents/azure-principal-architect.md | Adds Azure principal architect agent definition. |
| plugins/azure-cloud-development/agents/azure-logic-apps-expert.md | Adds Azure Logic Apps expert agent definition. |
| plugins/azure-cloud-development/.github/plugin/plugin.json | Changes agent registration to ./agents; normalizes skill paths. |
| plugins/awesome-copilot/skills/suggest-awesome-github-copilot-skills/SKILL.md | Adds skill for suggesting skills from awesome-copilot. |
| plugins/awesome-copilot/skills/suggest-awesome-github-copilot-instructions/SKILL.md | Adds skill for suggesting instruction files from awesome-copilot. |
| plugins/awesome-copilot/skills/suggest-awesome-github-copilot-agents/SKILL.md | Adds skill for suggesting agents from awesome-copilot. |
| plugins/awesome-copilot/agents/meta-agentic-project-scaffold.md | Adds meta agent for pulling content from awesome-copilot. |
| plugins/awesome-copilot/.github/plugin/plugin.json | Changes agent registration to ./agents; normalizes skill paths. |
| plugins/automate-this/.github/plugin/plugin.json | Normalizes skill path (removes trailing slash). |
| docs/README.skills.md | Registers the new security-review skill in the skills index. |
| | [scaffolding-oracle-to-postgres-migration-test-project](../skills/scaffolding-oracle-to-postgres-migration-test-project/SKILL.md) | Scaffolds an xUnit integration test project for validating Oracle-to-PostgreSQL database migration behavior in .NET solutions. Creates the test project, transaction-rollback base class, and seed data manager. Use when setting up test infrastructure before writing migration integration tests, or when a test project is needed for Oracle-to-PostgreSQL validation. | None | | ||
| | [scoutqa-test](../skills/scoutqa-test/SKILL.md) | This skill should be used when the user asks to "test this website", "run exploratory testing", "check for accessibility issues", "verify the login flow works", "find bugs on this page", or requests automated QA testing. Triggers on web application testing scenarios including smoke tests, accessibility audits, e-commerce flows, and user flow validation using ScoutQA CLI. Use this skill proactively after implementing web application features to verify they work correctly. | None | | ||
| | [secret-scanning](../skills/secret-scanning/SKILL.md) | Guide for configuring and managing GitHub secret scanning, push protection, custom patterns, and secret alert remediation. For pre-commit secret scanning in AI coding agents via the GitHub MCP Server, this skill references the Advanced Security plugin (`advanced-security@copilot-plugins`). Use this skill when enabling secret scanning, setting up push protection, defining custom patterns, triaging alerts, resolving blocked pushes, or when an agent needs to scan code for secrets before committing. | `references/alerts-and-remediation.md`<br />`references/custom-patterns.md`<br />`references/push-protection.md` | | ||
| | [security-review](../skills/security-review/SKILL.md) | AI-powered codebase security scanner that reasons about code like a security researcher — tracing data flows, understanding component interactions, and catching vulnerabilities that pattern-matching tools miss. Use this skill when asked to scan code for security vulnerabilities, find bugs, check for SQL injection, XSS, command injection, exposed API keys, hardcoded secrets, insecure dependencies, access control issues, or any request like "is my code secure?", "review for security issues", "audit this codebase", or "check for vulnerabilities". Covers injection flaws, authentication and access control bugs, secrets exposure, weak cryptography, insecure dependencies, and business logic issues across JavaScript, TypeScript, Python, Java, PHP, Go, Ruby, and Rust. | `references/language-patterns.md`<br />`references/report-format.md`<br />`references/secret-patterns.md`<br />`references/vuln-categories.md`<br />`references/vulnerable-packages.md` | |
There was a problem hiding this comment.
This PR advertises adding a new security-review skill (and this README entry links to ../skills/security-review/SKILL.md plus multiple reference files), but the provided diffs do not include the actual skills/security-review/ folder or the referenced files. If those files are missing from the PR, this entry will be a broken link and the skill won’t be distributable; add the new skill folder + referenced files, or remove/adjust this README entry to match what’s actually being added.
| - Use try/catch blocks around await expressions | ||
| - Avoid swallowing exceptions in async methods | ||
| - Use `ConfigureAwait(false)` when appropriate to prevent deadlocks in library code | ||
| - Propagate exceptions with `Task.FromException()` instead of throwing in async Task returning methods |
There was a problem hiding this comment.
This is incorrect guidance for async methods: inside an async Task/async Task<T> method you should throw; and let the async state machine capture the exception into the returned Task. Task.FromException(...) is for non-async code paths that need to return a faulted Task without using async. Suggest updating this bullet to reflect standard TAP guidance (throw in async methods; use Task.FromException only when you are not using async).
Suggested change
| - Propagate exceptions with `Task.FromException()` instead of throwing in async Task returning methods | |
| - In async `Task`/`Task<T>` methods, throw exceptions normally; use `Task.FromException(...)` only in non-async methods that must return a faulted `Task` |
Comment on lines
+115
to
+116
|
|
||
| --- |
There was a problem hiding this comment.
This trailing --- can be confused with a front-matter delimiter by tooling that scans markdown, and it doesn’t appear to serve a purpose at the end of the skill file. Recommend removing it (or replacing with an explicit markdown horizontal rule plus surrounding context) to avoid accidental parsing/formatting issues.
Suggested change
| --- |
Comment on lines
20
to
22
| "agents": [ | ||
| "./agents/azure-principal-architect.md", | ||
| "./agents/azure-saas-architect.md", | ||
| "./agents/azure-logic-apps-expert.md", | ||
| "./agents/azure-verified-modules-bicep.md", | ||
| "./agents/azure-verified-modules-terraform.md", | ||
| "./agents/terraform-azure-planning.md", | ||
| "./agents/terraform-azure-implement.md" | ||
| "./agents" | ||
| ], |
There was a problem hiding this comment.
Switching agents from explicit file paths to the whole ./agents directory changes what gets packaged/loaded. If the plugin loader expects explicit markdown files, this will prevent agents from registering; if it supports directories, it may inadvertently load non-agent markdown files added later (README, notes, drafts), creating unexpected surface area. Prefer listing the agent files explicitly (as before), or confirm/encode the intended behavior (e.g., only *.md under agents/ that meet agent front-matter requirements are loaded) and add a short comment in the plugin schema/docs if directory paths are officially supported.
| @@ -0,0 +1,60 @@ | |||
| --- | |||
| description: 'An agent that helps plan and execute multi-file changes by identifying relevant context and dependencies' | |||
| model: 'GPT-5' | |||
There was a problem hiding this comment.
The model: 'GPT-5' identifier may not be recognized by the target runtime/config (other agents in this PR use GPT-4.1 or gpt-4). If model identifiers must match a supported allow-list, this will fail at invocation time. Consider switching to a known-supported model string used elsewhere in the repo, or document the required model naming convention for agent front matter.
Suggested change
| model: 'GPT-5' | |
| model: 'GPT-4.1' |
| --- | ||
| description: "Work with PostgreSQL databases using the PostgreSQL extension." | ||
| name: "PostgreSQL Database Administrator" | ||
| tools: ["codebase", "edit/editFiles", "githubRepo", "extensions", "runCommands", "database", "pgsql_bulkLoadCsv", "pgsql_connect", "pgsql_describeCsv", "pgsql_disconnect", "pgsql_listDatabases", "pgsql_listServers", "pgsql_modifyDatabase", "pgsql_open_script", "pgsql_query", "pgsql_visualizeSchema"] |
There was a problem hiding this comment.
This agent’s directive says to inspect/manage the database and “do not look into the codebase”, but the declared tool set includes codebase and editing tools (edit/editFiles). That’s a mismatch that can lead to the agent doing the wrong thing. Recommended: remove codebase/editing tools from the tools list for this agent (or soften/clarify the directive if codebase inspection is actually allowed).
Suggested change
| tools: ["codebase", "edit/editFiles", "githubRepo", "extensions", "runCommands", "database", "pgsql_bulkLoadCsv", "pgsql_connect", "pgsql_describeCsv", "pgsql_disconnect", "pgsql_listDatabases", "pgsql_listServers", "pgsql_modifyDatabase", "pgsql_open_script", "pgsql_query", "pgsql_visualizeSchema"] | |
| tools: ["githubRepo", "extensions", "runCommands", "database", "pgsql_bulkLoadCsv", "pgsql_connect", "pgsql_describeCsv", "pgsql_disconnect", "pgsql_listDatabases", "pgsql_listServers", "pgsql_modifyDatabase", "pgsql_open_script", "pgsql_query", "pgsql_visualizeSchema"] |
|
|
||
| # PostgreSQL Database Administrator | ||
|
|
||
| Before running any tools, use #extensions to ensure that `ms-ossdata.vscode-pgsql` is installed and enabled. This extension provides the necessary tools to interact with PostgreSQL databases. If it is not installed, ask the user to install it before continuing. |
There was a problem hiding this comment.
This agent’s directive says to inspect/manage the database and “do not look into the codebase”, but the declared tool set includes codebase and editing tools (edit/editFiles). That’s a mismatch that can lead to the agent doing the wrong thing. Recommended: remove codebase/editing tools from the tools list for this agent (or soften/clarify the directive if codebase inspection is actually allowed).
| - Monitoring database performance | ||
| - Implementing security measures | ||
|
|
||
| You have access to various tools that allow you to interact with databases, execute queries, and manage database configurations. **Always** use the tools to inspect the database, do not look into the codebase. |
There was a problem hiding this comment.
This agent’s directive says to inspect/manage the database and “do not look into the codebase”, but the declared tool set includes codebase and editing tools (edit/editFiles). That’s a mismatch that can lead to the agent doing the wrong thing. Recommended: remove codebase/editing tools from the tools list for this agent (or soften/clarify the directive if codebase inspection is actually allowed).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request Checklist
npm startand verified thatREADME.mdis up to date.stagedbranch for this pull request.Description
Adds a new security-review skill that performs comprehensive AI-powered security analysis of codebases. Unlike traditional SAST tools that rely on pattern matching, this skill reasons about code the way a human security researcher would — tracing how user-controlled input moves across files, understanding how components interact, and self-verifying each finding to filter false positives before surfacing them.
How it works:
The skill follows a structured 8-step workflow that mirrors how a professional security audit is conducted:
What it covers:
Language support: JavaScript, TypeScript, Python, Java, PHP, Go, Ruby, Rust — with framework-specific patterns for Express, Django, Flask, FastAPI, Spring Boot, Rails, and more.
Bundled reference files (
references/— 5 files, ~988 lines total):vuln-categories.md— Deep detection guidance for every vulnerability type with vulnerable vs. safe code examplessecret-patterns.md— Regex patterns and entropy heuristics for 15+ credential providerslanguage-patterns.md— Framework-specific dangerous patterns per languagevulnerable-packages.md— CVE watchlist with affected version ranges and safe upgrade targetsreport-format.md— Structured output template with severity ratings and patch proposal formatType of Contribution
Additional Notes
This skill includes 5 reference files in
references/totaling ~988 lines of security detection guidance. All files are well under the 5MB asset limit. The skill has been validated withnpm run skill:validateand the README has been regenerated withnpm start.By submitting this pull request, I confirm that my contribution abides by the Code of Conduct and will be licensed under the MIT License.