diff --git a/README.md b/README.md index f12897b..79f4ad4 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Cisco Workflows Automation -This repository contains automation workflows organized by target platform and functionality as described in the following repository structure: +This repository contains automation workflows organized by target platform and functionality, plus a public toolkit for reviewing and improving workflow exports before submission. ## Repository Structure @@ -11,15 +11,26 @@ CiscoWorkflowsAutomation/ ├── CONTRIBUTING.md # Contribution guidelines and procedures ├── README.md # This file - repository overview ├── AI/ # AI-powered automation workflows and integrations -├── Catalyst Center/ # Cisco Catalyst Center automation workflows -├── Cross Domain/ # Multi-platform and cross-domain integration workflows -├── Learning Series/ # Educational workflows and training materials +├── Catalyst_Center/ # Cisco Catalyst Center automation workflows +├── Cross_Domain/ # Multi-platform and cross-domain integration workflows +├── Learning_Series/ # Educational workflows and training materials ├── LICENSE # Repository license terms ├── Meraki/ # Cisco Meraki automation workflows and integrations -├── Restful WebService/ # RESTful API integration workflows and examples +├── Restful_WebService/ # RESTful API integration workflows and examples +├── Toolkit/ # Public review, remediation, CLI, primary MCP integration, and optional Cursor helpers └── SECURITY.md # Security policies and reporting procedures ``` +## Public Toolkit + +The `Toolkit/` area provides reusable helpers for the workflow review checklist in this repository. It is intended for: + +- VS Code and Cursor users who want a user-level MCP integration that works across workspaces +- solution engineers or contributors using another LLM +- external contributors who want a documented CLI and MCP-based path + +See `Toolkit/README.md` for the audience guide and `WorkflowReviewChecklist.md` for the canonical review contract. The recommended editor integration is the MCP path in `Toolkit/mcp/`, while the review and remediation guides live under `Toolkit/exchange-review/` and `Toolkit/exchange-remediation/`. + ## Contributing We welcome contributions to expand and improve the automation workflows and supporting utilities. Please review our contribution guidelines: diff --git a/Toolkit/README.md b/Toolkit/README.md new file mode 100644 index 0000000..15bb427 --- /dev/null +++ b/Toolkit/README.md @@ -0,0 +1,83 @@ +# Workflow Toolkit + +This toolkit turns the internal workflow review standard into reusable review and remediation paths for the team. + +`WorkflowReviewChecklist.md` at the repository root is the canonical review contract. The CLI ships a packaged fallback copy for installed environments, but everything in `Toolkit/` should reference the root checklist first so the source of truth stays in one place. + +## Who It Is For + +### VS Code and Cursor users + +Start with `Toolkit/mcp/` for the primary cross-workspace install path. The MCP server is the recommended integration for VS Code and Cursor because one user-level configuration can make it available in every workspace. + +Use `Toolkit/cursor/` only if you also want optional thin skill wrappers that call the shared CLI. + +### Other LLM users + +Use `Toolkit/exchange-review/` and `Toolkit/exchange-remediation/` as copyable playbooks, then run the CLI in `Toolkit/cli/` for deterministic enumeration and checklist resolution. + +### External contributors + +Start with: + +1. `WorkflowReviewChecklist.md` +2. `Toolkit/exchange-review/README.md` +3. `Toolkit/cli/README.md` + +That path does not require Cursor. + +## Layout + +```text +Toolkit/ +├── README.md +├── cli/ # Layer 1: shared Python CLI and core helpers +├── mcp/ # Layer 2: primary stdio MCP integration for editors +├── cursor/ # Layer 3: optional thin Cursor skill wrappers +├── examples/ # Example commands and sample output +├── exchange-remediation/ # Public remediation guidance and mode reference +└── exchange-review/ # Public review guidance and checklist companion +``` + +## Layering + +- Layer 1: the CLI is the shared core for enumeration, checklist resolution, review preparation with remediation suggestions, and remediation planning. +- Layer 2: the MCP server wraps the same core instead of re-implementing logic and is the recommended editor integration. +- Layer 3: Cursor skills stay thin, optional, and delegate deterministic work to the CLI. + +## Quick Start + +### Review a workflow export with the CLI + +```bash +cd Toolkit/cli +python3 -m workflow_review inspect-workflow-export "/path/to/workflow.json" +python3 -m workflow_review prepare-review "/path/to/workflow.json" --json +``` + +### Install the workflow review MCP + +```bash +bash Toolkit/mcp/install_mcp.sh +``` + +This is the recommended setup for VS Code and Cursor because it is configured at the user level rather than tied to a single workspace. + +### Install the optional Cursor wrappers + +```bash +bash Toolkit/cursor/install_cursor_skills.sh +``` + +### Run the stdio MCP scaffold + +```bash +cd Toolkit/cli +python3 -m workflow_review.mcp_server +``` + +For installed-client usage, see `Toolkit/mcp/README.md` for Cursor, VS Code, and generic stdio MCP examples that point at `workflow-review-mcp` directly and start with `review`; `inspect_export` is available as an advanced helper. Treat `Toolkit/cursor/` as an optional convenience layer rather than the primary install surface. + +## Scope + +This public toolkit intentionally focuses on shareable workflow review and remediation content. Internal Jira/reporting skills and local scratch exports should remain outside this repo unless they are scrubbed and clearly reusable. diff --git a/Toolkit/cli/README.md b/Toolkit/cli/README.md new file mode 100644 index 0000000..de30fe1 --- /dev/null +++ b/Toolkit/cli/README.md @@ -0,0 +1,89 @@ +# Workflow Review CLI + +This directory contains the Layer 1 core for the public toolkit. + +The initial CLI focuses on deterministic tasks that are useful across Cursor, other LLMs, and MCP-capable clients: + +- inspect a workflow export and list its parent and embedded workflows +- validate that an export can be parsed +- resolve the canonical checklist path +- prepare a structured review brief +- prepare a structured remediation plan + +## Install + +From this directory: + +```bash +python3 -m pip install -e . +``` + +If you do not want to install the package, you can run it in-place: + +```bash +PYTHONPATH=src python3 -m workflow_review inspect-workflow-export "/path/to/workflow.json" +``` + +## Commands + +### Enumerate workflows + +```bash +PYTHONPATH=src python3 -m workflow_review inspect-workflow-export "/path/to/workflow.json" +PYTHONPATH=src python3 -m workflow_review inspect-workflow-export "/path/to/workflow.json" --json +``` + +The legacy `enumerate` subcommand still works as an alias, but `inspect-workflow-export` is the clearer public-facing name. + +### Validate exports + +```bash +PYTHONPATH=src python3 -m workflow_review validate "/path/to/workflow.json" +``` + +### Resolve the checklist + +```bash +PYTHONPATH=src python3 -m workflow_review checklist +PYTHONPATH=src python3 -m workflow_review checklist --show +``` + +When run from this repository, the CLI resolves `WorkflowReviewChecklist.md` at the repo root first. If the repo-root file is not available, it falls back to the packaged checklist that ships with the installed package. + +### Prepare a review run + +```bash +PYTHONPATH=src python3 -m workflow_review prepare-review "/path/to/workflow.json" --json +``` + +### Prepare a remediation plan + +```bash +PYTHONPATH=src python3 -m workflow_review plan-remediation "/path/to/workflow.json" \ + --mode fix-high-only \ + --safety ask-before-major-change \ + --json +``` + +## Checklist Resolution + +The CLI resolves the checklist in this order: + +1. `--checklist /path/to/file.md` +2. `WORKFLOW_REVIEW_CHECKLIST=/path/to/file.md` +3. the repo-root `WorkflowReviewChecklist.md` +4. the packaged internal review standard + +That keeps the repository copy canonical while still allowing the team to point at another checklist during development or use the packaged fallback when installed elsewhere. + +## MCP + +The stdio MCP scaffold is implemented in the same package and exposed through: + +```bash +workflow-review-mcp +``` + +For the easiest MCP setup, run `bash Toolkit/mcp/install_mcp.sh` from the repo root. It installs the package and prints the exact command path to use. + +See `../mcp/README.md` for client configuration examples. For VS Code and Cursor, MCP is the recommended editor-facing install path; the Cursor skill wrappers are optional thin helpers on top of the same core. diff --git a/Toolkit/cli/pyproject.toml b/Toolkit/cli/pyproject.toml new file mode 100644 index 0000000..1a54cc2 --- /dev/null +++ b/Toolkit/cli/pyproject.toml @@ -0,0 +1,25 @@ +[build-system] +requires = ["hatchling>=1.25.0"] +build-backend = "hatchling.build" + +[project] +name = "cisco-workflow-review" +version = "0.1.0" +description = "Public CLI and MCP helpers for Cisco workflow review and remediation." +readme = "README.md" +requires-python = ">=3.9" +license = { text = "Cisco Sample Code License" } +authors = [ + { name = "Cisco Workflows Automation DevNet Team" } +] +dependencies = [] + +[project.scripts] +workflow-review = "workflow_review.cli:main" +workflow-review-mcp = "workflow_review.mcp_server:main" + +[tool.hatch.build.targets.wheel] +packages = ["src/workflow_review"] + +[tool.hatch.build.targets.wheel.force-include] +"../../WorkflowReviewChecklist.md" = "src/workflow_review/data/WorkflowReviewChecklist.md" diff --git a/Toolkit/cli/src/workflow_review/__init__.py b/Toolkit/cli/src/workflow_review/__init__.py new file mode 100644 index 0000000..e753d7c --- /dev/null +++ b/Toolkit/cli/src/workflow_review/__init__.py @@ -0,0 +1,5 @@ +"""Shared workflow review helpers for the public toolkit.""" + +__all__ = ["__version__"] + +__version__ = "0.1.0" diff --git a/Toolkit/cli/src/workflow_review/__main__.py b/Toolkit/cli/src/workflow_review/__main__.py new file mode 100644 index 0000000..df699c3 --- /dev/null +++ b/Toolkit/cli/src/workflow_review/__main__.py @@ -0,0 +1,5 @@ +from workflow_review.cli import main + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/Toolkit/cli/src/workflow_review/checklist.py b/Toolkit/cli/src/workflow_review/checklist.py new file mode 100644 index 0000000..9a47ef1 --- /dev/null +++ b/Toolkit/cli/src/workflow_review/checklist.py @@ -0,0 +1,64 @@ +from __future__ import annotations + +import os +from pathlib import Path +from tempfile import NamedTemporaryFile +from importlib import resources + + +CHECKLIST_ENV_VAR = "WORKFLOW_REVIEW_CHECKLIST" +CHECKLIST_FILENAME = "WorkflowReviewChecklist.md" +PACKAGE_CHECKLIST_RESOURCE = "data/WorkflowReviewChecklist.md" +_PACKAGED_CHECKLIST_PATH: Path | None = None + + +def _candidate_repo_roots() -> list[Path]: + current = Path(__file__).resolve() + return list(current.parents) + + +def resolve_checklist_path(explicit_path: str | None = None) -> Path: + candidates: list[Path] = [] + + if explicit_path: + candidates.append(Path(explicit_path).expanduser().resolve()) + + env_value = os.getenv(CHECKLIST_ENV_VAR) + if env_value: + candidates.append(Path(env_value).expanduser().resolve()) + + for parent in _candidate_repo_roots(): + candidates.append(parent / CHECKLIST_FILENAME) + + for candidate in candidates: + if candidate.exists() and candidate.is_file(): + return candidate + + package_resource = resources.files("workflow_review").joinpath(PACKAGE_CHECKLIST_RESOURCE) + if package_resource.is_file(): + return _materialize_packaged_checklist(package_resource) + + searched = ", ".join(str(path) for path in candidates[:5]) + raise FileNotFoundError( + f"Could not resolve {CHECKLIST_FILENAME}. Checked explicit path, " + f"{CHECKLIST_ENV_VAR}, and repo-relative candidates such as: {searched}" + ) + + +def read_checklist(explicit_path: str | None = None) -> tuple[Path, str]: + checklist_path = resolve_checklist_path(explicit_path=explicit_path) + return checklist_path, checklist_path.read_text(encoding="utf-8") + + +def _materialize_packaged_checklist(package_resource: resources.abc.Traversable) -> Path: + global _PACKAGED_CHECKLIST_PATH + if _PACKAGED_CHECKLIST_PATH and _PACKAGED_CHECKLIST_PATH.exists(): + return _PACKAGED_CHECKLIST_PATH + + with resources.as_file(package_resource) as checklist_path: + source_path = Path(checklist_path) + with NamedTemporaryFile("w", suffix=".md", delete=False, encoding="utf-8") as handle: + handle.write(source_path.read_text(encoding="utf-8")) + _PACKAGED_CHECKLIST_PATH = Path(handle.name) + + return _PACKAGED_CHECKLIST_PATH diff --git a/Toolkit/cli/src/workflow_review/cli.py b/Toolkit/cli/src/workflow_review/cli.py new file mode 100644 index 0000000..1f96122 --- /dev/null +++ b/Toolkit/cli/src/workflow_review/cli.py @@ -0,0 +1,136 @@ +from __future__ import annotations + +import argparse +import json +import sys +from typing import Any + +from workflow_review.checklist import read_checklist, resolve_checklist_path +from workflow_review.enumerate import enumerate_workflows, render_enumeration_text +from workflow_review.remediation import ( + list_remediation_modes, + list_safety_modes, + plan_remediation, +) +from workflow_review.review import prepare_review + + +def build_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser( + description="Workflow review and remediation helpers for Cisco workflow exports." + ) + subparsers = parser.add_subparsers(dest="command", required=True) + + inspect_parser = subparsers.add_parser( + "inspect-workflow-export", + help="Inspect a workflow export and list the parent and embedded workflows it contains.", + ) + inspect_parser.add_argument("workflow_json") + inspect_parser.add_argument("--json", action="store_true", dest="json_output") + + enumerate_parser = subparsers.add_parser( + "enumerate", + help="Alias for inspect-workflow-export.", + ) + enumerate_parser.add_argument("workflow_json") + enumerate_parser.add_argument("--json", action="store_true", dest="json_output") + + validate_parser = subparsers.add_parser( + "validate", + help="Validate that a workflow export can be parsed and enumerated.", + ) + validate_parser.add_argument("workflow_json") + + checklist_parser = subparsers.add_parser( + "checklist", + help="Resolve or show the canonical workflow review checklist.", + ) + checklist_parser.add_argument("--checklist") + checklist_parser.add_argument("--show", action="store_true") + + review_parser = subparsers.add_parser( + "prepare-review", + help="Prepare a structured review brief for a workflow export.", + ) + review_parser.add_argument("workflow_json") + review_parser.add_argument("--checklist") + review_parser.add_argument("--priority-focus") + review_parser.add_argument("--severity-threshold") + review_parser.add_argument("--json", action="store_true", dest="json_output") + + remediation_parser = subparsers.add_parser( + "plan-remediation", + help="Prepare a structured remediation plan without applying edits.", + ) + remediation_parser.add_argument("workflow_json") + remediation_parser.add_argument("--mode", required=True, choices=sorted(list_remediation_modes())) + remediation_parser.add_argument("--safety", required=True, choices=sorted(list_safety_modes())) + remediation_parser.add_argument("--findings") + remediation_parser.add_argument("--priority-focus") + remediation_parser.add_argument("--json", action="store_true", dest="json_output") + + return parser + + +def _print_payload(payload: Any, json_output: bool) -> int: + if json_output: + print(json.dumps(payload, indent=2)) + elif isinstance(payload, str): + print(payload) + else: + print(json.dumps(payload, indent=2)) + return 0 + + +def main(argv: list[str] | None = None) -> int: + parser = build_parser() + args = parser.parse_args(argv) + + try: + if args.command in {"inspect-workflow-export", "enumerate"}: + result = enumerate_workflows(args.workflow_json) + if args.json_output: + return _print_payload(result, json_output=True) + return _print_payload(render_enumeration_text(result), json_output=False) + + if args.command == "validate": + result = enumerate_workflows(args.workflow_json) + print( + f"Validated {result['workflow_count']} workflows in {result['file']}" + ) + return 0 + + if args.command == "checklist": + checklist_path = resolve_checklist_path(args.checklist) + if args.show: + _, checklist_text = read_checklist(args.checklist) + print(checklist_text) + else: + print(checklist_path) + return 0 + + if args.command == "prepare-review": + result = prepare_review( + workflow_path=args.workflow_json, + checklist_path=args.checklist, + priority_focus=args.priority_focus, + severity_threshold=args.severity_threshold, + ) + return _print_payload(result, json_output=args.json_output) + + if args.command == "plan-remediation": + result = plan_remediation( + workflow_path=args.workflow_json, + remediation_mode=args.mode, + safety_mode=args.safety, + findings_path=args.findings, + priority_focus=args.priority_focus, + ) + return _print_payload(result, json_output=args.json_output) + + except (FileNotFoundError, ValueError) as exc: + print(str(exc), file=sys.stderr) + return 1 + + parser.print_help() + return 1 diff --git a/Toolkit/cli/src/workflow_review/data/WorkflowReviewChecklist.md b/Toolkit/cli/src/workflow_review/data/WorkflowReviewChecklist.md new file mode 100644 index 0000000..c5c93e5 --- /dev/null +++ b/Toolkit/cli/src/workflow_review/data/WorkflowReviewChecklist.md @@ -0,0 +1,226 @@ +# Workflow Review Checklist + +This checklist is designed for LLM agents to systematically review JSON workflow files in the CiscoWorkflowsAutomation repository. Each item includes specific JSON elements to inspect and validation criteria for automated analysis. + +## 1. Workflow Inputs & Parameters + +**JSON Elements to Inspect:** `definition_workflow.properties.input_groups`, `definition_workflow.properties.inputs` + +- [ ] **Input Validation**: **If inputs exist**, check each input in `inputs` array has `name`, `type`, `description`, and appropriate `required` flag - **Note**: Workflows with no inputs are valid for automated/scheduled workflows +- [ ] **Secure String Review**: For inputs with `type: "SecureString"`, verify `name` is descriptive and `scope` is documented in description +- [ ] **Standard Outputs Present**: Verify outputs include `Result`, `Status Code`, `Status Message`, `Error Message` in `outputs` array +- [ ] **Default Values Safety**: Scan `default_value` fields for production data (real IPs, device serials, org IDs, passwords) - should be generic placeholders only +- [ ] **Input Redundancy**: Identify duplicate or similar inputs that could be consolidated +- [ ] **Variable Naming Convention**: + - Workflow variables (user-facing): Human-readable, capitalized (e.g., "Device ID", "Sort By Date") + - Code activity outputs: camelCase (e.g., "deviceID", "sortByDate") + - **Note**: System-generated `unique_name` fields use random alphanumeric patterns - this is expected XDR behavior +- [ ] **Prefix Check**: Ensure variable names don't use unnecessary "Input -" or "Output -" prefixes + +## 2. Targets & Target Groups + +**JSON Elements to Inspect:** `definition_workflow.properties.targets`, `definition_workflow.properties.target_groups` + +- [ ] **Hardcoded Target Analysis**: Check if `targets` array contains specific target references - validate if justified for scheduled/webhook workflows +- [ ] **Target Customization**: Verify `targets` can be overridden during installation (not hardcoded in workflow logic) - **Note**: Some workflows legitimately require specific target groups for proper operation +- [ ] **Target Group Genericity**: In `target_groups`, check `matching_conditions` don't contain environment-specific values (specific hostnames, IPs, user IDs) unless justified +- [ ] **Unused Target Cleanup**: Cross-reference targets/target_groups with actual usage in workflow activities + +## 3. Atomics & API Usage + +**JSON Elements to Inspect:** `definition_workflow.activities`, activity `type` fields, `api_requests` + +- [ ] **New Atomic Justification**: Identify activities with `type: "atomic_workflow"` - check if description explains why new atomic was needed +- [ ] **Generic API Usage**: Look for activities with `type: "web_service_request"` - verify if atomic alternative exists and document rationale +- [ ] **API Request Documentation**: For generic API calls, ensure future improvement notes are in activity description + +## 4. Groups & Categories + +**JSON Elements to Inspect:** `definition_workflow.properties.groups`, `definition_workflow.properties.categories` + +- [ ] **Group Functionality**: Review `groups` array for clear functional purpose vs. existing groups +- [ ] **Category Appropriateness**: Check that `categories` array contains relevant classifications for the workflow's purpose and domain +- [ ] **Unnecessary Category Creation**: Flag if new categories are being created when existing ones would be more appropriate (check against repository's existing category taxonomy) +- [ ] **Missing Categories**: Verify workflow isn't missing obvious category classifications that would help with organization and discoverability +- [ ] **Orphaned Elements**: Identify groups/categories defined but not referenced in workflow activities + +## 5. Logic & Flow + +**JSON Elements to Inspect:** `definition_workflow.activities`, activity relationships, conditional logic + +- [ ] **Group Activity Usage**: Verify major workflow sections use `type: "group"` activities with descriptive `name` and `description` +- [ ] **Validation Steps**: After API calls or sub-workflows, check for success/failure validation activities +- [ ] **Continue on Failure**: For activities that should continue on error, verify `continue_on_failure: true` is set appropriately +- [ ] **Idempotency Implementation**: **CRITICAL** - For CREATE/DELETE operations: + - CREATE: Look for existence checks before creation (conditional logic, error handling for "already exists") + - DELETE: Look for existence validation before deletion (handle "not found" gracefully) + - Verify error handling allows multiple executions without failure + - **Note**: Some operations may be inherently idempotent or designed for single execution - verify if multiple runs are expected +- [ ] **Loop Exit Conditions**: In loop activities, verify `break_conditions` or `max_iterations` prevent infinite execution +- [ ] **Modularity Assessment**: Identify repeated logic blocks that could be extracted to sub-workflows + +## 6. Error Handling + +**JSON Elements to Inspect:** Error handling activities, output variable assignments, `continue_on_failure` settings + +- [ ] **Explicit Success/Failure Paths**: Verify both success and failure branches update `Result`, `Status Code`, `Status Message`, `Error Message` outputs +- [ ] **Error Aggregation**: Check that errors from sub-workflows and API calls are captured and surfaced in outputs +- [ ] **Error Message Population**: Ensure failure paths set meaningful values in `Error Message` output +- [ ] **Timeout Implementation**: For long-running operations, verify `timeout` settings and retry logic exist + +## 7. Essential Hygiene & Security + +**JSON Elements to Inspect:** All string values, variable assignments, activity configurations + +- [ ] **Sensitive Data Scan**: Search for passwords, API keys, tokens in plain text - should use `SecureString` variables instead +- [ ] **Description Quality**: Check `definition_workflow.properties.description` includes purpose, prerequisites, limitations, dependencies +- [ ] **Placeholder Data**: Scan for sample/test data in variable defaults, API endpoints, or documentation +- [ ] **Naming Consistency**: Verify consistent naming patterns across inputs, outputs, groups, categories, activities +- [ ] **Cleanup Verification**: Ensure no unused variables, groups, categories, or debug artifacts remain +- [ ] **Grammar and Spelling**: Validate text fields for typos and grammatical errors + +## LLM Review Instructions + +### Prompt Template: + +``` +Context: You are a workflow validator, checking the quality of the Workflow before it can be approved. + +Objective: Validate the Workflow with the Workflow-Review-Checklist items so that there is consistency and quality in approved workflows. Provide overall assessment score. + +Workflow: [Workflow Name] + +MANDATORY MULTI-PHASE REVIEW PROCESS: + +PHASE 1 - ENUMERATION (Complete this FIRST): +- Identify and list ALL workflows in the JSON file +- For each workflow, document: name, type (parent/embedded), line range +- Present the enumeration list before proceeding +- Count: X workflows found (1 parent + Y embedded subworkflows) + +PHASE 2 - DETAILED REVIEW (One workflow at a time): +- Review each workflow individually against all 7 checklist categories +- Complete all categories for Workflow N before starting Workflow N+1 +- Report findings for each workflow separately + +PHASE 3 - AGGREGATE ASSESSMENT: +- Compile findings across all workflows +- Provide overall quality score and recommendations + +Audience: The feedback will be for workflow developers with technical expertise seeking approval of their workflows. Suggest improvements for the findings. + +Style: Provide a bulleted list of issues for each Checklist Category. + +Tone: Crisp, not too verbose. +``` + +**Optional Enhancements:** +- `File Path: [path/to/workflow.json]` - Helps LLM understand file context +- `Priority Focus: [Security/Performance/Maintainability]` - When specific concerns exist +- `Severity Threshold: [Critical/High/Medium/Low]` - To filter noise for mature workflows + +### Analysis Process: +1. **Parse Full JSON Structure**: Load the workflow JSON and identify the parent workflow plus all embedded subworkflows in the file. +2. **Enumerate Review Scope - MANDATORY FIRST STEP**: + - Build a complete list of ALL workflows to review BEFORE starting validation + - For each workflow, document: name, unique_name, line range, and type (parent/subworkflow) + - Present this enumeration to the reviewer/user for confirmation + - DO NOT proceed to validation until enumeration is complete and confirmed +3. **Systematic Validation - One Workflow at a Time**: + - Review workflows sequentially (parent first, then each subworkflow) + - For EACH workflow, apply ALL 7 checklist categories completely + - Mark each workflow as complete individually before moving to the next + - Report findings for each workflow separately, not in aggregate +4. **Pattern Recognition**: Look for anti-patterns such as hardcoded values, missing error handling, weak idempotency, and security vulnerabilities across parent and embedded workflows. +5. **Cross-Reference Analysis**: Verify consistency between definitions and actual usage within each workflow, and between parent/subworkflow interfaces. +6. **Report Findings**: For each failed check, provide: + - **Specific Location**: Clearly identify the workflow element (e.g., "Workflow Variable 'Variable Name'", "Activity 'Activity Title'", "Workflow's main 'Variables' section"). If a specific property within an element is relevant, include it (e.g., "Description of Activity 'Activity Title'"). + - Description of the problem + - Recommended fix or improvement + - Severity level (Critical, High, Medium, Low) + +### Quality Score Guidelines: + +- **9-10**: Excellent - Minor improvements only +- **7-8**: Good - Few moderate issues to address +- **5-6**: Acceptable - Several issues requiring attention +- **3-4**: Needs Work - Multiple critical/high issues +- **1-2**: Major Revision Required - Fundamental problems + +### Common JSON Patterns to Flag: + +- **Security Issues**: Plain text secrets, production URLs/IPs in defaults +- **Reliability Issues**: Missing error handling, no idempotency for CREATE/DELETE (when multiple executions expected) +- **Maintainability Issues**: Inconsistent naming, missing descriptions, redundant elements +- **Usability Issues**: Confusing variable names, missing standard outputs + +### Important Notes for Reviewers: + +- **Definition - Embedded Subworkflow**: Any workflow logic invoked from the parent workflow (for example, activities with `type: "atomic_workflow"` or equivalent nested workflow references in the same JSON context). +- **Default Review Scope**: Comprehensive reviews must include the parent workflow and all embedded subworkflows within the JSON file. +- **Coverage Expectation**: Do not mark a review complete until all embedded subworkflows have been validated against the checklist. +- **No Shortcuts**: Do not summarize or skip embedded subworkflow reviews. Each must be read and validated completely. +- **Sequential Completion**: Do not mark multiple review categories as "complete" simultaneously without actual validation. +- **Evidence Required**: For each finding, cite specific workflow name, activity name, or variable that demonstrates the issue. +- **System-Generated IDs**: Don't flag random alphanumeric `unique_name` fields - these are XDR-generated +- **Target Groups**: Some workflows legitimately require specific target configurations +- **Idempotency**: Not all workflows need to be idempotent - consider the use case +- **Context Matters**: Always consider the workflow's intended purpose and deployment scenario + +### Pre-Submission Verification: + +Before submitting your review, confirm: +- [ ] Enumerated all workflows in the file (parent + embedded subworkflows) +- [ ] Read the full definition of each enumerated workflow +- [ ] Applied all 7 checklist categories to each workflow +- [ ] Reported workflow-specific findings (not just parent workflow issues) +- [ ] Did not batch-complete validation steps without thorough review + +### Expected Output Format: + +``` +## Workflow Review Results + +### ✅ Passed Checks: [X/Y] +### ⚠️ Issues Found: [Count by severity] + +#### Critical Issues: +- [Issue description using user-friendly field names and business terminology with specific location and fix recommendation] + +#### High Priority Issues: +- [Issue description using user-friendly field names and business terminology with specific location and fix recommendation] + +#### Medium Priority Issues: +- [Issue description using user-friendly field names and business terminology with specific location and fix recommendation] + +#### Low Priority Issues: +- [Issue description using user-friendly field names and business terminology with specific location and fix recommendation] + +### Summary: +[Overall assessment and key recommendations using terminology end users understand] + +### Example Issue Descriptions: +- Instead of: "Variable 'hostIPAddress' in definition_workflow.properties.inputs lacks proper naming convention" +- Use: "Input field 'Server IP Address' should use a more user-friendly display name" + +- Instead of: "Missing continue_on_failure flag in activity uuid-1234" +- Use: "Step 'Device Registration' needs error handling to continue if the device already exists" + +- Instead of: "SecureString variable 'authToken' missing scope documentation" +- Use: "Password field 'API Authentication Token' needs usage description for end users" + +**- For Location Clarity:** +- Instead of: "Location: `definition_workflow.variables`" +- Use: "Location: Workflow's main `Variables` section." + +- Instead of: "Location: `variable_workflow_02O427U6E0UKK4pPkgaKcbm9H3Y1Oy4eUhF`" +- Use: "Location: Output variable `Output - Filtered JSON` in the Workflow's `Variables` section." + +- Instead of: "Location: `definition_activity_02O427U9YGQD32G3ACkUh3TgKs9D2TyfwVd`" +- Use: "Location: Activity `Check the HTTP Request for Errors`." + +- Instead of: "Location: `definition_activity_02O427U95N2YX0AbAR87yYHC6aDfWC3Ky8f.properties.description`" +- Use: "Location: Description of activity `JSON Placeholder Web Request`." +--- + +*This checklist enables systematic, automated review of workflow JSON files to ensure quality, security, and consistency across the CiscoWorkflowsAutomation repository.* diff --git a/Toolkit/cli/src/workflow_review/enumerate.py b/Toolkit/cli/src/workflow_review/enumerate.py new file mode 100644 index 0000000..f1aa03b --- /dev/null +++ b/Toolkit/cli/src/workflow_review/enumerate.py @@ -0,0 +1,188 @@ +from __future__ import annotations + +import json +import re +from pathlib import Path +from typing import Any + + +def load_json_file(path: Path) -> tuple[str, Any]: + if not path.exists(): + raise FileNotFoundError(f"Workflow export not found: {path}") + if not path.is_file(): + raise ValueError(f"Workflow export is not a file: {path}") + + text = path.read_text(encoding="utf-8") + try: + data = json.loads(text) + except json.JSONDecodeError as exc: + raise ValueError( + f"Invalid JSON in {path}: line {exc.lineno}, column {exc.colno}: {exc.msg}" + ) from exc + + return text, data + + +def iter_workflows(node: Any, path: list[Any] | None = None) -> list[dict[str, Any]]: + if path is None: + path = [] + + workflows: list[dict[str, Any]] = [] + + if isinstance(node, dict): + if node.get("object_type") == "definition_workflow": + workflows.append({"path": list(path), "workflow": node}) + + for key, value in node.items(): + workflows.extend(iter_workflows(value, [*path, key])) + elif isinstance(node, list): + for index, value in enumerate(node): + workflows.extend(iter_workflows(value, [*path, index])) + + return workflows + + +def is_simple_identifier(value: str) -> bool: + return bool(re.fullmatch(r"[A-Za-z_][A-Za-z0-9_]*", value)) + + +def format_path(path: list[Any]) -> str: + result = "$" + for part in path: + if isinstance(part, int): + result += f"[{part}]" + elif is_simple_identifier(part): + result += f".{part}" + else: + escaped = str(part).replace("\\", "\\\\").replace('"', '\\"') + result += f'["{escaped}"]' + return result + + +def find_start_line(text_lines: list[str], unique_name: str | None) -> int | None: + if not unique_name: + return None + + needle = f'"unique_name": "{unique_name}"' + for index, line in enumerate(text_lines, start=1): + if needle in line: + return index + return None + + +def classify_workflows(workflows: list[dict[str, Any]]) -> None: + parent_assigned = False + + for item in workflows: + path = item["path"] + if path == ["workflow"] and not parent_assigned: + item["workflow_type"] = "parent" + parent_assigned = True + else: + item["workflow_type"] = "embedded" + + if not parent_assigned and workflows: + workflows[0]["workflow_type"] = "parent" + + +def enrich_line_ranges(text: str, workflows: list[dict[str, Any]]) -> None: + text_lines = text.splitlines() + total_lines = len(text_lines) + + for item in workflows: + workflow = item["workflow"] + item["start_line"] = find_start_line(text_lines, workflow.get("unique_name")) + item["end_line"] = None + + indexed = sorted( + ( + (item["start_line"], index) + for index, item in enumerate(workflows) + if item["start_line"] is not None + ), + key=lambda pair: pair[0], + ) + + for position, (_, index) in enumerate(indexed): + next_start = indexed[position + 1][0] if position + 1 < len(indexed) else None + workflows[index]["end_line"] = (next_start - 1) if next_start else total_lines + + +def summarize_workflows(workflows: list[dict[str, Any]]) -> list[dict[str, Any]]: + summary: list[dict[str, Any]] = [] + + for item in workflows: + workflow = item["workflow"] + properties = workflow.get("properties", {}) + name = ( + workflow.get("title") + or properties.get("display_name") + or workflow.get("name") + or workflow.get("unique_name") + or "Unnamed Workflow" + ) + summary.append( + { + "name": name, + "unique_name": workflow.get("unique_name"), + "workflow_type": item["workflow_type"], + "path": format_path(item["path"]), + "start_line": item["start_line"], + "end_line": item["end_line"], + } + ) + + return summary + + +def enumerate_workflows(path: str | Path) -> dict[str, Any]: + workflow_path = Path(path).expanduser().resolve() + text, data = load_json_file(workflow_path) + workflows = iter_workflows(data) + if not workflows: + raise ValueError("No definition_workflow objects found in the supplied JSON export.") + + classify_workflows(workflows) + enrich_line_ranges(text, workflows) + summary = summarize_workflows(workflows) + parent_count = sum(1 for item in summary if item["workflow_type"] == "parent") + embedded_count = sum(1 for item in summary if item["workflow_type"] == "embedded") + + return { + "file": str(workflow_path), + "workflow_count": len(summary), + "parent_count": parent_count, + "embedded_count": embedded_count, + "workflows": summary, + } + + +def render_enumeration_text(result: dict[str, Any]) -> str: + lines = [ + "## Workflow Enumeration", + f"File: {result['file']}", + ( + f"Count: {result['workflow_count']} workflows found " + f"({result['parent_count']} parent + {result['embedded_count']} embedded)" + ), + "", + ] + + for index, item in enumerate(result["workflows"], start=1): + if item["start_line"] and item["end_line"]: + line_range = f"L{item['start_line']}-L{item['end_line']}" + else: + line_range = "unavailable" + + lines.extend( + [ + f"{index}. {item['name']}", + f" Type: {item['workflow_type']}", + f" Unique name: {item['unique_name'] or 'missing'}", + f" Path: {item['path']}", + f" Line range: {line_range}", + "", + ] + ) + + return "\n".join(lines).rstrip() diff --git a/Toolkit/cli/src/workflow_review/mcp_server.py b/Toolkit/cli/src/workflow_review/mcp_server.py new file mode 100644 index 0000000..c36f6fb --- /dev/null +++ b/Toolkit/cli/src/workflow_review/mcp_server.py @@ -0,0 +1,211 @@ +from __future__ import annotations + +import json +import sys +from typing import Any + +from workflow_review.checklist import resolve_checklist_path +from workflow_review.enumerate import enumerate_workflows +from workflow_review.remediation import list_remediation_modes, list_safety_modes, plan_remediation +from workflow_review.review import prepare_review + + +PROTOCOL_VERSION = "2024-11-05" + +CAPABILITIES = { + "tools": {"listChanged": False}, + "resources": {"subscribe": False, "listChanged": False}, + "prompts": {"listChanged": False}, +} + + +TOOLS = [ + { + "name": "inspect_export", + "description": "Advanced helper that lists the parent workflow and any embedded subworkflows in a JSON export.", + "inputSchema": { + "type": "object", + "required": ["workflow_path"], + "properties": { + "workflow_path": {"type": "string"}, + }, + }, + }, + { + "name": "load_checklist", + "description": "Resolve and load the internal review standard used by the workflow review toolkit.", + "inputSchema": { + "type": "object", + "properties": { + "checklist_path": {"type": "string"}, + }, + }, + }, + { + "name": "review", + "description": "Review a workflow export end to end: enumerate workflows first, then return the review brief that leads into findings, severity, and remediation suggestions.", + "inputSchema": { + "type": "object", + "required": ["workflow_path"], + "properties": { + "workflow_path": {"type": "string"}, + "checklist_path": {"type": "string"}, + "priority_focus": {"type": "string"}, + "severity_threshold": {"type": "string"}, + }, + }, + }, + { + "name": "plan_remediation", + "description": "Prepare a remediation plan without writing to disk.", + "inputSchema": { + "type": "object", + "required": ["workflow_path", "mode", "safety"], + "properties": { + "workflow_path": {"type": "string"}, + "mode": {"type": "string", "enum": sorted(list_remediation_modes())}, + "safety": {"type": "string", "enum": sorted(list_safety_modes())}, + "findings_path": {"type": "string"}, + "priority_focus": {"type": "string"}, + }, + }, + }, +] + + +def _write_message(message: dict[str, Any]) -> None: + payload = json.dumps(message).encode("utf-8") + sys.stdout.write(f"Content-Length: {len(payload)}\r\n\r\n") + sys.stdout.flush() + sys.stdout.buffer.write(payload) + sys.stdout.buffer.flush() + + +def _read_message() -> dict[str, Any] | None: + headers: dict[str, str] = {} + while True: + line = sys.stdin.buffer.readline() + if not line: + return None + if line == b"\r\n": + break + name, value = line.decode("utf-8").split(":", 1) + headers[name.strip().lower()] = value.strip() + + content_length = int(headers.get("content-length", "0")) + if content_length <= 0: + return None + + payload = sys.stdin.buffer.read(content_length) + return json.loads(payload.decode("utf-8")) + + +def _result_content(payload: Any) -> dict[str, Any]: + return { + "content": [{"type": "text", "text": json.dumps(payload, indent=2)}], + "structuredContent": payload, + } + + +def _handle_tool_call(name: str, arguments: dict[str, Any]) -> dict[str, Any]: + if name == "inspect_export": + return _result_content(enumerate_workflows(arguments["workflow_path"])) + if name == "load_checklist": + checklist_path = resolve_checklist_path(arguments.get("checklist_path")) + return _result_content( + { + "checklist_path": str(checklist_path), + "description": "Internal review standard used by the workflow review toolkit.", + } + ) + if name == "review": + return _result_content( + prepare_review( + workflow_path=arguments["workflow_path"], + checklist_path=arguments.get("checklist_path"), + priority_focus=arguments.get("priority_focus"), + severity_threshold=arguments.get("severity_threshold"), + ) + ) + if name == "plan_remediation": + return _result_content( + plan_remediation( + workflow_path=arguments["workflow_path"], + remediation_mode=arguments["mode"], + safety_mode=arguments["safety"], + findings_path=arguments.get("findings_path"), + priority_focus=arguments.get("priority_focus"), + ) + ) + raise ValueError(f"Unknown tool: {name}") + + +def _success(message_id: Any, result: dict[str, Any]) -> dict[str, Any]: + return {"jsonrpc": "2.0", "id": message_id, "result": result} + + +def _error(message_id: Any, code: int, text: str) -> dict[str, Any]: + return { + "jsonrpc": "2.0", + "id": message_id, + "error": {"code": code, "message": text}, + } + + +def handle_message(message: dict[str, Any]) -> dict[str, Any] | None: + method = message.get("method") + message_id = message.get("id") + params = message.get("params", {}) + + if method == "initialize": + return _success( + message_id, + { + "protocolVersion": PROTOCOL_VERSION, + "serverInfo": {"name": "cisco-workflow-review", "version": "0.1.0"}, + "capabilities": CAPABILITIES, + }, + ) + + if method == "notifications/initialized": + return None + + if method == "tools/list": + return _success(message_id, {"tools": TOOLS}) + + if method == "resources/list": + return _success(message_id, {"resources": []}) + + if method == "resources/templates/list": + return _success(message_id, {"resourceTemplates": []}) + + if method == "prompts/list": + return _success(message_id, {"prompts": []}) + + if method == "tools/call": + try: + result = _handle_tool_call(params["name"], params.get("arguments", {})) + return _success(message_id, result) + except Exception as exc: # pragma: no cover - best-effort server path + return _error(message_id, -32000, str(exc)) + + if method == "ping": + return _success(message_id, {}) + + if message_id is not None: + return _error(message_id, -32601, f"Unsupported method: {method}") + return None + + +def main() -> int: + while True: + message = _read_message() + if message is None: + return 0 + response = handle_message(message) + if response is not None: + _write_message(response) + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/Toolkit/cli/src/workflow_review/remediation.py b/Toolkit/cli/src/workflow_review/remediation.py new file mode 100644 index 0000000..60f3f2a --- /dev/null +++ b/Toolkit/cli/src/workflow_review/remediation.py @@ -0,0 +1,96 @@ +from __future__ import annotations + +from pathlib import Path +from typing import Any + +from workflow_review.enumerate import enumerate_workflows + + +REMEDIATION_MODES = { + "fix-all": "Apply all approved findings that are safe to fix now.", + "fix-high-and-medium": "Apply approved high and medium findings.", + "fix-high-only": "Apply only approved high-severity findings.", + "fix-low-only": "Apply low-risk cleanup and readability fixes only.", + "proposal-only": "Plan the work without editing files.", + "improve-activity-descriptions": "Improve activity titles and descriptions only.", + "improve-workflow-description": "Improve the main workflow description only.", +} + +SAFETY_MODES = { + "update-in-place": "Edit the current file directly.", + "propose-copy": "Create a sibling proposed file and keep the original unchanged.", + "ask-before-major-change": "Apply safe fixes, but stop before structural changes.", +} + +LOW_RISK_MODES = { + "fix-low-only", + "proposal-only", + "improve-activity-descriptions", + "improve-workflow-description", +} + + +def list_remediation_modes() -> dict[str, str]: + return dict(REMEDIATION_MODES) + + +def list_safety_modes() -> dict[str, str]: + return dict(SAFETY_MODES) + + +def major_change_risk(remediation_mode: str) -> bool: + return remediation_mode not in LOW_RISK_MODES + + +def plan_remediation( + workflow_path: str, + remediation_mode: str, + safety_mode: str, + findings_path: str | None = None, + priority_focus: str | None = None, +) -> dict[str, Any]: + if remediation_mode not in REMEDIATION_MODES: + raise ValueError(f"Unsupported remediation mode: {remediation_mode}") + if safety_mode not in SAFETY_MODES: + raise ValueError(f"Unsupported safety mode: {safety_mode}") + + workflow_path = str(Path(workflow_path).expanduser().resolve()) + findings_path = ( + str(Path(findings_path).expanduser().resolve()) if findings_path else None + ) + enumeration = enumerate_workflows(workflow_path) + requires_major_change_review = major_change_risk(remediation_mode) + + if remediation_mode in {"improve-activity-descriptions", "improve-workflow-description"}: + planned_fixes = [ + "Limit changes to user-facing descriptions and readability improvements.", + "Preserve workflow logic, outputs, categories, and targets.", + ] + elif remediation_mode == "proposal-only": + planned_fixes = ["Generate a remediation plan only. No file edits should be applied."] + else: + planned_fixes = [ + "Review approved findings against the selected severity scope.", + "Patch the current workflow incrementally rather than rewriting it.", + "Re-run review after changes and report fixed, remaining, and new issues.", + ] + + approval_required = safety_mode == "ask-before-major-change" and requires_major_change_review + + return { + "workflow_path": workflow_path, + "findings_path": findings_path, + "priority_focus": priority_focus, + "remediation_mode": remediation_mode, + "remediation_mode_description": REMEDIATION_MODES[remediation_mode], + "safety_mode": safety_mode, + "safety_mode_description": SAFETY_MODES[safety_mode], + "major_change_possible": requires_major_change_review, + "approval_required": approval_required, + "enumeration": enumeration, + "planned_fixes": planned_fixes, + "deferred": [ + "Do not apply unrelated cleanup outside the selected remediation scope.", + "Do not change workflow identity fields unless explicitly approved.", + ], + } diff --git a/Toolkit/cli/src/workflow_review/review.py b/Toolkit/cli/src/workflow_review/review.py new file mode 100644 index 0000000..d5b575c --- /dev/null +++ b/Toolkit/cli/src/workflow_review/review.py @@ -0,0 +1,51 @@ +from __future__ import annotations + +from pathlib import Path +from typing import Any + +from workflow_review.checklist import resolve_checklist_path +from workflow_review.enumerate import enumerate_workflows + + +def prepare_review( + workflow_path: str, + checklist_path: str | None = None, + priority_focus: str | None = None, + severity_threshold: str | None = None, +) -> dict[str, Any]: + workflow_path = str(Path(workflow_path).expanduser().resolve()) + enumeration = enumerate_workflows(workflow_path) + + resolved_checklist = None + checklist_error = None + try: + resolved_checklist = str(resolve_checklist_path(checklist_path)) + except FileNotFoundError as exc: + checklist_error = str(exc) + + return { + "workflow_path": workflow_path, + "checklist_path": resolved_checklist, + "checklist_error": checklist_error, + "priority_focus": priority_focus, + "severity_threshold": severity_threshold, + "enumeration": enumeration, + "review_contract": { + "sequence": [ + "Enumerate all workflows before review.", + "Review the parent workflow first.", + "Review each embedded workflow across all 7 checklist categories.", + "Lead with findings ordered by severity.", + "Finish with an overall assessment and next steps.", + ], + "categories": [ + "Inputs & Parameters", + "Targets & Target Groups", + "Atomics & API Usage", + "Groups & Categories", + "Logic & Flow", + "Error Handling", + "Essential Hygiene & Security", + ], + }, + } diff --git a/Toolkit/cli/tests/fixtures/sample_workflow.json b/Toolkit/cli/tests/fixtures/sample_workflow.json new file mode 100644 index 0000000..73e451d --- /dev/null +++ b/Toolkit/cli/tests/fixtures/sample_workflow.json @@ -0,0 +1,22 @@ +{ + "workflow": { + "object_type": "definition_workflow", + "unique_name": "definition_workflow_parent", + "name": "Parent Workflow", + "title": "Parent Workflow", + "properties": { + "display_name": "Parent Workflow" + }, + "embedded": [ + { + "object_type": "definition_workflow", + "unique_name": "definition_workflow_child", + "name": "Child Workflow", + "title": "Child Workflow", + "properties": { + "display_name": "Child Workflow" + } + } + ] + } +} diff --git a/Toolkit/cli/tests/test_enumerate.py b/Toolkit/cli/tests/test_enumerate.py new file mode 100644 index 0000000..21d1814 --- /dev/null +++ b/Toolkit/cli/tests/test_enumerate.py @@ -0,0 +1,28 @@ +from __future__ import annotations + +import json +import unittest +from pathlib import Path + +from workflow_review.enumerate import enumerate_workflows + + +FIXTURE_PATH = Path(__file__).parent / "fixtures" / "sample_workflow.json" + + +class EnumerateWorkflowTests(unittest.TestCase): + def test_enumerates_parent_and_embedded_workflows(self) -> None: + result = enumerate_workflows(FIXTURE_PATH) + self.assertEqual(result["workflow_count"], 2) + self.assertEqual(result["parent_count"], 1) + self.assertEqual(result["embedded_count"], 1) + self.assertEqual(result["workflows"][0]["workflow_type"], "parent") + self.assertEqual(result["workflows"][1]["workflow_type"], "embedded") + + def test_fixture_is_valid_json(self) -> None: + data = json.loads(FIXTURE_PATH.read_text(encoding="utf-8")) + self.assertIn("workflow", data) + + +if __name__ == "__main__": + unittest.main() diff --git a/Toolkit/cursor/README.md b/Toolkit/cursor/README.md new file mode 100644 index 0000000..b215089 --- /dev/null +++ b/Toolkit/cursor/README.md @@ -0,0 +1,36 @@ +# Cursor Wrappers + +This directory contains thin Cursor wrappers for the public toolkit. + +The wrappers intentionally delegate deterministic work to the shared CLI in `../cli/` and keep only the agent-facing orchestration in `SKILL.md`. + +## Position in the toolkit + +These wrappers are a secondary convenience layer. They are not the canonical install path and they are not the source of truth for the toolkit. For VS Code and Cursor, prefer the MCP setup in `../mcp/` first when you want the toolkit available in all workspaces. + +## Install + +From the repository root: + +```bash +bash Toolkit/cursor/install_cursor_skills.sh +``` + +This installs the public `exchange-review` and `exchange-remediation` skills into `~/.cursor/skills`. + +## Included wrappers + +- `exchange-review` +- `exchange-remediation` + +## When to use this + +Use these wrappers when you want skill-style prompting in Cursor after the MCP path is already available or when you specifically want a lightweight prompt wrapper around the shared CLI. + +## Design rule + +If a change can live in the CLI, it should live in the CLI. The Cursor wrappers should stay focused on: + +- collecting inputs +- choosing the right CLI calls +- presenting the result in a reviewer-friendly format diff --git a/Toolkit/cursor/exchange-remediation/SKILL.md b/Toolkit/cursor/exchange-remediation/SKILL.md new file mode 100644 index 0000000..77903cf --- /dev/null +++ b/Toolkit/cursor/exchange-remediation/SKILL.md @@ -0,0 +1,48 @@ +--- +name: exchange-remediation +description: Apply approved workflow review fixes safely by building a remediation plan, honoring remediation and safety modes, and preserving workflow identity unless the user explicitly approves structural changes. +disable-model-invocation: true +--- + +# Exchange Remediation + +## When to use + +- The user wants to apply workflow review feedback instead of only reading findings. +- The user asks to fix checklist issues by severity. +- The user wants description-only cleanup or a guarded remediation plan. + +## Inputs to collect + +- Workflow export JSON path +- Remediation mode +- Safety mode +- Optional findings source +- Optional priority focus + +## Required flow + +1. Reuse an approved findings set when available. Otherwise run the review wrapper first. +2. Resolve the real skill directory and repo root from this skill's file location. Do not assume the current working directory is the repo. +3. Enumerate the workflow scope before editing: + ```bash + PYTHONPATH="/Toolkit/cli/src" python3 -m workflow_review inspect-workflow-export "/path/to/workflow.json" --json + ``` +4. Build the remediation plan first: + ```bash + PYTHONPATH="/Toolkit/cli/src" python3 -m workflow_review plan-remediation "/path/to/workflow.json" --mode fix-high-only --safety ask-before-major-change --json + ``` +5. Apply the selected safety mode exactly. +6. Re-review after edits and report fixed, remaining, and new issues. + +## Guardrails + +- No rewrite-by-default. +- Preserve `name`, `title`, `unique_name`, and core intent unless the user explicitly approves otherwise. +- Keep changes inside the selected remediation mode. +- Stop if JSON validity or post-edit review fails. +- Description-only modes must remain non-structural. + +## Additional reference + +Use `reference.md` for mode definitions and the major-change rules. diff --git a/Toolkit/cursor/exchange-remediation/reference.md b/Toolkit/cursor/exchange-remediation/reference.md new file mode 100644 index 0000000..233f3c0 --- /dev/null +++ b/Toolkit/cursor/exchange-remediation/reference.md @@ -0,0 +1,18 @@ +# Exchange Remediation Wrapper Reference + +Use this wrapper with the shared CLI in `Toolkit/cli/`. + +## Minimum command set + +```bash +PYTHONPATH="/Toolkit/cli/src" python3 -m workflow_review inspect-workflow-export "/path/to/workflow.json" --json +PYTHONPATH="/Toolkit/cli/src" python3 -m workflow_review plan-remediation "/path/to/workflow.json" --mode fix-high-only --safety ask-before-major-change --json +``` + +## Key rules + +- Reuse approved findings when possible. +- Stay inside the chosen remediation scope. +- Treat branch, output, target, and category changes as potentially major. +- Re-review after edits. +- Keep `WorkflowReviewChecklist.md` at the repository root as the canonical source for the checklist, with the packaged copy as the installed fallback. diff --git a/Toolkit/cursor/exchange-review/SKILL.md b/Toolkit/cursor/exchange-review/SKILL.md new file mode 100644 index 0000000..86a706d --- /dev/null +++ b/Toolkit/cursor/exchange-review/SKILL.md @@ -0,0 +1,53 @@ +--- +name: exchange-review +description: Review exported Cisco workflow JSON against the canonical workflow review standard, enumerate the export first, and produce severity-ranked findings plus remediation suggestions and an overall readiness assessment. +--- + +# Exchange Review + +## When to use + +- The user provides a workflow export JSON file. +- The user wants Exchange review, standards validation, or approval-readiness feedback. +- The user wants findings aligned to the canonical workflow review standard. + +## Start here prompts + +- Review this workflow export. +- Review this file against the internal review standard. +- Review this export and include remediation suggestions. + +## Inputs to collect + +- Workflow export JSON path + +## Required flow + +1. Use `WorkflowReviewChecklist.md` from the repository root when this repo is checked out. Fall back to the packaged checklist only when the toolkit is installed outside the repo. +2. Resolve the real skill directory and the repo root from this skill's file location. Do not assume the current working directory is the repo. +3. Run the shared CLI first to review the export and establish scope: + - The review flow should enumerate first and return remediation suggestions in the first pass. + ```bash + PYTHONPATH="/Toolkit/cli/src" python3 -m workflow_review prepare-review "/path/to/workflow.json" --json + ``` +4. Present the enumeration list before deeper review. +5. Review one workflow at a time across all 7 checklist categories. +6. Lead with findings ordered by severity, then finish with the overall assessment, top improvements, and remediation suggestions. + +## Review rules + +- Do not skip embedded subworkflows. +- Prefer user-facing names over raw internal IDs. +- Use `reference.md` for the local review sequence and severity guidance. +- Keep the canonical review standard as the source of truth if there is any conflict. + +## Output sections + +- Enumeration +- Critical issues +- High priority issues +- Medium priority issues +- Low priority issues +- Workflow-by-workflow notes +- Overall assessment +- Remediation suggestions diff --git a/Toolkit/cursor/exchange-review/reference.md b/Toolkit/cursor/exchange-review/reference.md new file mode 100644 index 0000000..60735cd --- /dev/null +++ b/Toolkit/cursor/exchange-review/reference.md @@ -0,0 +1,19 @@ +# Exchange Review Wrapper Reference + +Use this wrapper with the shared CLI in `Toolkit/cli/`. + +## Minimum command set + +```bash +PYTHONPATH="/Toolkit/cli/src" python3 -m workflow_review inspect-workflow-export "/path/to/workflow.json" +PYTHONPATH="/Toolkit/cli/src" python3 -m workflow_review prepare-review "/path/to/workflow.json" --json +``` + +## Reminder + +- Enumerate first. +- Review parent workflow first. +- Cover all embedded workflows. +- Lead with findings. +- Include remediation suggestions in the first pass output. +- Keep `WorkflowReviewChecklist.md` at the repository root canonical, with the packaged checklist as the fallback for installed environments. diff --git a/Toolkit/cursor/install_cursor_skills.sh b/Toolkit/cursor/install_cursor_skills.sh new file mode 100755 index 0000000..1bc9296 --- /dev/null +++ b/Toolkit/cursor/install_cursor_skills.sh @@ -0,0 +1,33 @@ +#!/usr/bin/env bash +set -euo pipefail + +REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +SKILLS_SRC_DIR="$REPO_ROOT/Toolkit/cursor" +CURSOR_SKILLS_DIR="${HOME}/.cursor/skills" + +mkdir -p "$CURSOR_SKILLS_DIR" + +installed=0 + +for skill_dir in "$SKILLS_SRC_DIR"/*; do + [ -d "$skill_dir" ] || continue + [ -f "$skill_dir/SKILL.md" ] || continue + + skill_name="$(basename "$skill_dir")" + target_link="$CURSOR_SKILLS_DIR/$skill_name" + + if [ -L "$target_link" ] || [ -e "$target_link" ]; then + rm -rf "$target_link" + fi + + ln -s "$skill_dir" "$target_link" + printf 'Installed skill: %s -> %s\n' "$skill_name" "$target_link" + installed=$((installed + 1)) +done + +if [ "$installed" -eq 0 ]; then + printf 'No skill directories with SKILL.md were found under %s\n' "$SKILLS_SRC_DIR" >&2 + exit 1 +fi + +printf '\nInstalled %d skill(s) into %s\n' "$installed" "$CURSOR_SKILLS_DIR" diff --git a/Toolkit/examples/README.md b/Toolkit/examples/README.md new file mode 100644 index 0000000..2f83d29 --- /dev/null +++ b/Toolkit/examples/README.md @@ -0,0 +1,25 @@ +# Toolkit Examples + +This directory shows how the public toolkit is intended to be used without requiring a private repo or internal-only tooling. + +## Review example + +From `Toolkit/cli`: + +```bash +PYTHONPATH=src python3 -m workflow_review inspect-workflow-export "../../Meraki/CheckAvailableFirmwareForNetwork__definition_workflow_02M30GYJJSYJL0wQPPnkQgIcavBkG6796mF/definition_workflow_02M30GYJJSYJL0wQPPnkQgIcavBkG6796mF.json" +PYTHONPATH=src python3 -m workflow_review prepare-review "../../Meraki/CheckAvailableFirmwareForNetwork__definition_workflow_02M30GYJJSYJL0wQPPnkQgIcavBkG6796mF/definition_workflow_02M30GYJJSYJL0wQPPnkQgIcavBkG6796mF.json" --json +``` + +## Remediation planning example + +```bash +PYTHONPATH=src python3 -m workflow_review plan-remediation "../../Meraki/CheckAvailableFirmwareForNetwork__definition_workflow_02M30GYJJSYJL0wQPPnkQgIcavBkG6796mF/definition_workflow_02M30GYJJSYJL0wQPPnkQgIcavBkG6796mF.json" \ + --mode fix-low-only \ + --safety ask-before-major-change \ + --json +``` + +## Sample output + +See `review-output.md` for the intended structure of a reviewer-facing summary. The actual review contract lives in `WorkflowReviewChecklist.md` at the repository root. diff --git a/Toolkit/examples/review-output.md b/Toolkit/examples/review-output.md new file mode 100644 index 0000000..b629a5d --- /dev/null +++ b/Toolkit/examples/review-output.md @@ -0,0 +1,37 @@ +# Sample Review Output + +```markdown +## Exchange Review Results + +### Enumeration +- Workflow 1: Parent Workflow +- Workflow 2: Embedded Workflow + +### Critical issues +- None. + +### High priority issues +- Activity "Submit API Request" does not surface failure details in the workflow outputs. + +### Medium priority issues +- None. + +### Low priority issues +- The main workflow description is too thin for Exchange reviewers. + +### Workflow-by-workflow notes +#### Parent Workflow +- Logic & Flow: Add explicit success and failure output updates. + +#### Embedded Workflow +- Essential Hygiene & Security: Replace a real org ID default with a placeholder. + +### Overall assessment +- Score: 7/10 +- Approval readiness: approve with suggestions + +### Remediation suggestions +- Improve output handling. +- Remove production-like defaults. +- Strengthen reviewer-facing descriptions. +``` diff --git a/Toolkit/exchange-remediation/README.md b/Toolkit/exchange-remediation/README.md new file mode 100644 index 0000000..35fc4e7 --- /dev/null +++ b/Toolkit/exchange-remediation/README.md @@ -0,0 +1,44 @@ +# Exchange Remediation Guide + +Use this guide after a review has identified accepted findings to fix. + +The remediation path in this public toolkit is intentionally conservative: + +- review first +- scope fixes explicitly +- preserve workflow identity +- avoid rewrite-by-default +- stop before major structural edits unless the safety mode allows them + +## Planning first + +The initial public CLI prepares remediation plans without applying edits: + +```bash +cd Toolkit/cli +PYTHONPATH=src python3 -m workflow_review plan-remediation "/path/to/workflow.json" \ + --mode fix-high-only \ + --safety ask-before-major-change \ + --json +``` + +That plan can then drive: + +- a human review pass +- a Cursor skill invocation +- or a future automated remediation engine + +## Modes + +See `reference.md` for: + +- remediation modes such as `fix-all` or `improve-workflow-description` +- safety modes such as `update-in-place` or `ask-before-major-change` +- the public definition of a major change + +## Guardrails + +- Patch existing workflows instead of regenerating them. +- Keep changes inside the approved remediation scope. +- Preserve `name`, `title`, `unique_name`, and core workflow intent unless explicitly approved otherwise. +- Re-review after edits and report fixed, remaining, and new issues. diff --git a/Toolkit/exchange-remediation/reference.md b/Toolkit/exchange-remediation/reference.md new file mode 100644 index 0000000..a8ed654 --- /dev/null +++ b/Toolkit/exchange-remediation/reference.md @@ -0,0 +1,68 @@ +# Exchange Remediation Reference + +## Prerequisite + +Use exchange review first unless you already have an approved findings list. + +## Remediation modes + +### `fix-all` +Apply all approved findings that fit the selected safety mode. + +### `fix-high-and-medium` +Apply approved `High` and `Medium` findings. + +### `fix-high-only` +Apply approved `High` findings only. + +### `fix-low-only` +Apply low-risk cleanup and readability fixes only. + +### `proposal-only` +Produce a concrete plan without editing files. + +### `improve-activity-descriptions` +Improve activity titles and descriptions only. This must remain non-structural. + +### `improve-workflow-description` +Improve only the main workflow description. This must remain non-structural. + +## Safety modes + +### `update-in-place` +Edit the current workflow file directly. + +### `propose-copy` +Create a sibling proposed file and leave the original unchanged. + +### `ask-before-major-change` +Apply safe fixes directly, but stop before structural changes. + +## Major change definition + +Treat the remediation as a major change when it would: + +- add or remove branches +- add or remove outputs +- add or remove major logic blocks +- materially change target behavior +- materially change categories +- convert a non-atomic workflow into an atomic workflow +- redesign the workflow rather than patching it + +## Non-negotiable guardrails + +- No rewrite-by-default. +- Preserve workflow identity. +- Stay inside the selected remediation scope. +- Re-review after edits. + +## Useful command + +```bash +cd Toolkit/cli +PYTHONPATH=src python3 -m workflow_review plan-remediation "/path/to/workflow.json" \ + --mode fix-high-only \ + --safety ask-before-major-change \ + --json +``` diff --git a/Toolkit/exchange-review/README.md b/Toolkit/exchange-review/README.md new file mode 100644 index 0000000..e934e3b --- /dev/null +++ b/Toolkit/exchange-review/README.md @@ -0,0 +1,47 @@ +# Exchange Review Guide + +Use this guide when you want to review a workflow export against the canonical workflow review standard. + +## Source of truth + +`WorkflowReviewChecklist.md` at the repository root is the canonical review contract. The CLI also ships a packaged copy for installed environments, but this repo should reference the root checklist first. + +## Start here prompts + +- Review this workflow export. +- Review this file against the internal review standard. +- Review this export and include remediation suggestions. + +The recommended flow is: + +1. Enumerate the workflow scope first. +2. Review the parent workflow. +3. Review each embedded workflow. +4. Aggregate findings by severity. +5. End with an approval-readiness summary, next actions, and remediation suggestions. + +## CLI-first path + +```bash +cd Toolkit/cli +PYTHONPATH=src python3 -m workflow_review inspect-workflow-export "/path/to/workflow.json" +PYTHONPATH=src python3 -m workflow_review prepare-review "/path/to/workflow.json" --json +``` + +The `prepare-review` output is intentionally structured so it can be handed to another LLM or used by a thin Cursor or MCP wrapper. + +## Manual/LLM path + +If you are using another LLM directly: + +1. Use `WorkflowReviewChecklist.md` from the repository root. +2. Attach the workflow JSON export. +3. Use the reference in `reference.md` to keep the review sequence and output format consistent. + +## Expectations + +- Do not skip embedded subworkflows. +- Keep the canonical review standard as the source of truth. +- Lead with findings, sorted by severity. +- Include remediation suggestions in the first review output. +- Use user-facing names instead of raw internal IDs whenever possible. diff --git a/Toolkit/exchange-review/reference.md b/Toolkit/exchange-review/reference.md new file mode 100644 index 0000000..3b25ba8 --- /dev/null +++ b/Toolkit/exchange-review/reference.md @@ -0,0 +1,63 @@ +# Exchange Review Reference + +## Source of truth + +The canonical review standard is `WorkflowReviewChecklist.md` at the repository root. The CLI ships a packaged copy for installed environments, but checked-out work in this repo should reference the root file first. + +## Mandatory sequence + +1. Enumerate all workflows in the export first. +2. Present the parent workflow and every embedded workflow before deeper review. +3. Review one workflow at a time across all 7 checklist categories. +4. Lead with findings ordered by severity. +5. End with an overall assessment, next actions, and remediation suggestions. + +## Checklist categories + +1. Inputs & Parameters +2. Targets & Target Groups +3. Atomics & API Usage +4. Groups & Categories +5. Logic & Flow +6. Error Handling +7. Essential Hygiene & Security + +## Reporting rules + +- Use user-facing labels whenever possible. +- Avoid raw internal IDs in reviewer-facing comments. +- For each issue include: + - location + - problem + - recommended fix + - severity +- Reviews are incomplete until embedded workflows are covered too. + +## Severity guidance + +- `High`: correctness, hidden failure, security, or likely approval blocker +- `Medium`: meaningfully important but not a blocker +- `Low`: cleanup, readability, consistency, and maintainability + +## Review output shape + +Suggested sections: + +- Enumeration +- Critical issues +- High priority issues +- Medium priority issues +- Low priority issues +- Workflow-by-workflow notes +- Overall assessment +- Remediation suggestions + +## Useful commands + +```bash +cd Toolkit/cli +PYTHONPATH=src python3 -m workflow_review inspect-workflow-export "/path/to/workflow.json" +PYTHONPATH=src python3 -m workflow_review prepare-review "/path/to/workflow.json" --json +``` + +When installed, the CLI resolves `WorkflowReviewChecklist.md` from the repository root first. If you run the toolkit outside a checkout, the packaged checklist is used as the fallback. diff --git a/Toolkit/mcp/README.md b/Toolkit/mcp/README.md new file mode 100644 index 0000000..6d774b1 --- /dev/null +++ b/Toolkit/mcp/README.md @@ -0,0 +1,96 @@ +# Workflow Review MCP + +This directory documents the primary stdio MCP path for the public toolkit. + +The MCP server is intentionally thin. It wraps the same core package used by the CLI instead of re-implementing workflow review logic. + +## Recommended path + +For VS Code and Cursor, this is the primary integration surface. MCP is the recommended default because it can be configured once at the user level and then used across all workspaces. + +## Current tools + +- `review` +- `load_checklist` +- `plan_remediation` +- `inspect_export` (advanced helper, optional) + +These tools are read-only planning helpers in the first public slice. + +## Run the stdio server + +```bash +workflow-review-mcp +``` + +## One-time install + +For the easiest setup, run: + +```bash +bash Toolkit/mcp/install_mcp.sh +``` + +That installs the package and prints the exact `workflow-review-mcp` path you can use in Cursor, VS Code, or any stdio MCP client. + +## Start here + +Use one of these simple prompts to kick off the review: + +- "Review this workflow export." +- "Review this export and tell me what needs improvement." +- "Review this file against the internal review standard." +- "Review this export and include remediation suggestions." + +### Cursor + +Use `review` as the starting action. Cursor will enumerate the export first, then return findings and remediation suggestions. Prefer this MCP path over workspace-local skill wrappers when you want the toolkit available everywhere. + +See `cursor.example.json`. It includes both: + +- an installed-package configuration that uses `workflow-review-mcp` +- a dev-only fallback that still uses `python3 -m workflow_review.mcp_server` + +### VS Code + +If your MCP client supports stdio servers, use the installed command directly and start with `review`: + +```json +{ + "mcpServers": { + "cisco-workflow-review": { + "command": "/path/to/installed/workflow-review-mcp", + "args": [], + "env": {} + } + } +} +``` + +If your VS Code setup uses a different Python environment, point `command` at that environment’s installed `workflow-review-mcp` binary instead. The install helper prints the exact path for you. + +### Generic MCP client + +Any stdio-capable MCP client can use the same installed binary and start with `review`: + +```json +{ + "mcpServers": { + "cisco-workflow-review": { + "command": "workflow-review-mcp", + "args": [], + "env": {} + } + } +} +``` + +If the command is not on `PATH`, replace it with the full path to the installed script. + +## Design intent + +- User-level editor integration first +- Local stdio transport first +- Shared core with the CLI +- Narrow tools with explicit scope +- No file-writing remediation in the initial public scaffold diff --git a/Toolkit/mcp/cursor.example.json b/Toolkit/mcp/cursor.example.json new file mode 100644 index 0000000..bfffb7d --- /dev/null +++ b/Toolkit/mcp/cursor.example.json @@ -0,0 +1,21 @@ +{ + "mcpServers": { + "cisco-workflow-review": { + "command": "workflow-review-mcp", + "args": [], + "env": {} + }, + "cisco-workflow-review-dev": { + "command": "python3", + "args": [ + "-m", + "workflow_review.mcp_server" + ], + "cwd": "/path/to/CiscoWorkflowsAutomation/Toolkit/cli", + "env": { + "PYTHONPATH": "/path/to/CiscoWorkflowsAutomation/Toolkit/cli/src", + "WORKFLOW_REVIEW_CHECKLIST": "/path/to/CiscoWorkflowsAutomation/WorkflowReviewChecklist.md" + } + } + } +} diff --git a/Toolkit/mcp/install_mcp.sh b/Toolkit/mcp/install_mcp.sh new file mode 100755 index 0000000..b28d749 --- /dev/null +++ b/Toolkit/mcp/install_mcp.sh @@ -0,0 +1,30 @@ +#!/usr/bin/env bash +set -euo pipefail + +REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +CLI_DIR="$REPO_ROOT/Toolkit/cli" + +printf 'Installing workflow review MCP package...\n' +python3 -m pip install -e "$CLI_DIR" + +SCRIPT_PATH="$(python3 - <<'PY' +import site +from pathlib import Path + +user_base = Path(site.getuserbase()) +print(user_base / "bin" / "workflow-review-mcp") +PY +)" + +cat < -
-# Uploading Workflow for Submission -Workflows are submitted to the Exchange from the Workflow Designer under "More Actions"/"Share"/"Submit to Exchange". -This will bring the submitter to the submission screens where the information below must be provided. - -Use the information below to fill out the form requirements on submission. Submitters can monitor the status from the Automation/Exchange menu where the current status is provided. - -If a submission is rejected, which is common, the comments provided will explain the reasons. At this point the develloper of the workflow can return to their Workspace, unlock their workflow, and make the necessary edits. When complete, they can submit the workflow again. - -## Integration -- Indicate the relevant domains the Workflow operates across +This legacy guide has been superseded by the public toolkit. -## Display Name -- Name of your Workflow +## Use these canonical references instead -## Author -- Your Email Address - -## Contact & Support Information -- Your Email Address or possibly a Group Email Alias if more relevant. - -## Short Description +- `WorkflowReviewChecklist.md` for the review contract +- `Toolkit/README.md` for the toolkit overview and source-of-truth chain +- `Toolkit/mcp/README.md` for the recommended VS Code and Cursor install path +- `Toolkit/exchange-review/README.md` for review guidance +- `Toolkit/exchange-remediation/README.md` for remediation guidance +- `Toolkit/cli/README.md` for command-line usage +- `Toolkit/cursor/README.md` for optional Cursor skill wrappers -## Installation Instructions -The following is an example format of what can be provided in the Installations Instructions -This may vary depending on your submission. -Note the section Additional Reference Material can reference links added in the next section for "External Links" +## Why this file stays short -### Workflow Name -- The Name of your Workflow -> -### Key Features -- (Examples) -- Query JSON Placeholder Web Service for list of Users -- Locates specific User based on inputted Name -- Scan the returned data to find a matching email address for that User -- Return the Email if found, 400 if no User is located -> -### Installation Steps -1. (Examples) -2. Download the workflow from the Exchange -3. The Workflow will install an HTTP Target, "JSON Placeholder Web Service", if this Target does not already exist. -4. There is no specific configuration required for this HTTP endpoint. -> -### Prerequisites -- Note and requirements such as access rights, etc.. required to run this workflow. -> -### Additional Reference Material -- Please see the reference under External Links to the "Learning Series" YouTube content. - +The old submission walkthrough overlapped with the toolkit and used outdated wording. Keeping this file as a pointer avoids stale instructions and reduces the chance of contributors following the wrong path.