diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index c55006b..3a52146 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -2,16 +2,20 @@ ## 📊 Implementation Status -**Production-Tested Resources** (config-driven, zero code changes needed): -- ✅ **Aurora MySQL** - Production tested (config ready, awaiting endoflife.date data) -- ✅ **Aurora PostgreSQL** - Config ready, requires separate Wiz report ID -- ✅ **EKS** - Production tested (policy classification working) -- ✅ **ElastiCache (Redis/Valkey/Memcached)** - Production tested - -**Planned Resources** (add ~15 lines to `pkg/config/defaults/resources.yaml`): -- ✅ **OpenSearch** - Production tested (auto-detects legacy Elasticsearch versions) -- 📋 RDS MySQL/PostgreSQL -- 📋 Lambda runtimes +**Production-Tested Resources** (config-driven, zero code changes needed; all +ten ship in `pkg/config/defaults/resources.yaml` and run live at Block): + +- ✅ **Aurora MySQL** — production tested via the + [`deploy/endoflife-override`](./deploy/endoflife-override/) shim while + [endoflife.date#9534](https://github.com/endoflife-date/endoflife.date/pull/9534) + is still open upstream +- ✅ **Aurora PostgreSQL** +- ✅ **EKS** — policy classification working (standard / extended support split) +- ✅ **ElastiCache (Redis / Valkey / Memcached)** +- ✅ **OpenSearch** — auto-detects legacy Elasticsearch versions +- ✅ **RDS MySQL** +- ✅ **RDS PostgreSQL** +- ✅ **Lambda runtimes** — JSON-extracted from `graphEntity.properties.runtime` --- @@ -35,7 +39,7 @@ **Vision:** Version Guard is a **cloud-agnostic** version drift detection platform supporting multiple cloud providers. ### Phase 1 (Implemented): AWS -- **Resources**: ✅ Aurora MySQL (production tested), ✅ Aurora PostgreSQL (config ready), ✅ EKS (production tested), ✅ ElastiCache (production tested), ✅ OpenSearch (production tested), 📋 RDS, 📋 Lambda +- **Resources** (all production tested at Block): ✅ Aurora MySQL, ✅ Aurora PostgreSQL, ✅ EKS, ✅ ElastiCache (Redis/Valkey/Memcached), ✅ OpenSearch, ✅ RDS MySQL, ✅ RDS PostgreSQL, ✅ Lambda - **Inventory**: Wiz saved reports (primary) + Custom sources (extensible) - **EOL Data**: endoflife.date API (404 graceful degradation for products not yet listed) @@ -180,28 +184,29 @@ export WIZ_REPORT_IDS='{ ``` Version-Guard/ ├── cmd/ -│ ├── server/main.go # Server with Temporal worker + HTTP admin +│ ├── server/main.go # Server: Temporal worker + HTTP admin │ └── cli/main.go # CLI tool for operators │ -├── config/ -│ └── resources.yaml # Config-driven resource definitions -│ ├── pkg/ │ ├── types/ │ │ ├── resource.go # Core types: Resource, Finding -│ │ ├── status.go # Status enum (Red/Yellow/Green) +│ │ ├── snapshot.go # Snapshot + SnapshotSummary types +│ │ ├── status.go # Status enum (Red/Yellow/Green/Unknown) │ │ └── cloud.go # CloudProvider enum │ │ │ ├── config/ │ │ ├── types.go # Configuration schema -│ │ └── loader.go # Config loader and validator +│ │ ├── loader.go # Config loader and validator +│ │ ├── transforms.go # Transforms DSL (see TRANSFORMS.md) +│ │ └── defaults/resources.yaml # Embedded default resource catalog │ │ │ ├── inventory/ │ │ ├── inventory.go # InventorySource interface │ │ ├── wiz/ # Wiz implementation (multi-cloud) │ │ │ ├── generic.go # Generic config-driven inventory source -│ │ │ ├── client.go # Wiz HTTP client -│ │ │ └── helpers.go # CSV parsing, tag extraction +│ │ │ ├── client.go / http_client.go # Wiz API + HTTP layer +│ │ │ ├── helpers.go / tags.go # CSV parsing, tag extraction +│ │ │ └── transforms.go # Transforms applier (uses pkg/config DSL) │ │ └── mock/ # Mock for tests │ │ │ ├── eol/ @@ -209,8 +214,8 @@ Version-Guard/ │ │ ├── endoflife/ │ │ │ ├── client.go # endoflife.date HTTP client │ │ │ ├── provider.go # endoflife.date provider -│ │ │ ├── adapters.go # Schema adapters (standard, EKS) -│ │ │ └── ADAPTERS.md # Why EKS needs its own adapter (gotcha doc) +│ │ │ ├── adapters.go # Schema adapters (standard, declarative) +│ │ │ └── ADAPTERS.md # Lifecycle schema reference (standard vs declarative) │ │ └── mock/ # Mock for tests │ │ │ ├── policy/ @@ -219,12 +224,12 @@ Version-Guard/ │ │ │ ├── store/ │ │ ├── store.go # Store interface -│ │ └── memory/ -│ │ └── store.go # In-memory implementation +│ │ └── memory/store.go # In-memory implementation │ │ │ ├── snapshot/ │ │ ├── builder.go # Snapshot JSON builder -│ │ └── store.go # S3 storage operations +│ │ ├── store.go # S3 snapshot store +│ │ └── memory_store.go # In-process snapshot store (laptop / CI) │ │ │ ├── workflow/ │ │ ├── detection/ @@ -232,22 +237,23 @@ Version-Guard/ │ │ │ └── activities.go # Inventory, EOL, detection activities │ │ └── orchestrator/ │ │ ├── workflow.go # Main orchestrator (fan-out) -│ │ └── activities.go # Snapshot storage activity +│ │ ├── activities.go # Snapshot storage activity +│ │ └── notify.go # Optional out-of-process emitter webhook │ │ │ ├── scan/ -│ │ ├── scan.go # Scan trigger (HTTP + CLI) +│ │ ├── scan.go # Scan trigger (shared by HTTP + CLI) │ │ └── handler.go # HTTP handler for POST /scan │ │ +│ ├── schedule/schedule.go # Temporal Schedule create-or-update wiring +│ │ │ ├── emitters/ │ │ ├── emitters.go # Emitter interfaces (for custom implementations) -│ │ └── examples/ -│ │ └── logging_emitter.go # Example logging emitter +│ │ └── examples/logging_emitter.go # Reference logging emitter │ │ -│ └── registry/ -│ └── client.go # Service attribution interface +│ └── registry/client.go # Service attribution interface (optional) │ -└── docs/ - └── examples/ # Usage examples +├── charts/version-guard/ # Helm chart +└── deploy/ # Dockerfile + endoflife.date override shim ``` --- @@ -466,6 +472,14 @@ s3://your-bucket/snapshots/ ### Snapshot Schema +Snapshots are produced via Go's `encoding/json` defaults. Top-level fields on +`Snapshot` and `SnapshotSummary` carry explicit `snake_case` JSON tags +(see [pkg/types/snapshot.go](./pkg/types/snapshot.go)); per-`Finding` fields +have no JSON tags and therefore serialize as PascalCase. The `findings_by_type` +map is keyed by the resource **config ID** (e.g. `aurora-mysql`, `eks`, +`elasticache-redis`) — the uppercase `ResourceType` constants in +`pkg/types/resource.go` are test fixtures only. + ```json { "snapshot_id": "scan-2026-04-09-123456", @@ -473,15 +487,17 @@ s3://your-bucket/snapshots/ "generated_at": "2026-04-09T12:34:56Z", "scan_start_time": "2026-04-09T12:00:00Z", "scan_end_time": "2026-04-09T12:34:56Z", + "scan_duration_sec": 2096, "findings_by_type": { - "AURORA": [...], - "EKS": [...] + "aurora-mysql": [{"ResourceID": "...", "Status": "RED", "Message": "..."}], + "eks": [{"ResourceID": "...", "Status": "YELLOW", "Message": "..."}] }, "summary": { "total_resources": 150, "red_count": 12, "yellow_count": 35, "green_count": 103, + "unknown_count": 0, "compliance_percentage": 68.7, "by_service": {...}, "by_cloud_provider": {...} @@ -500,8 +516,10 @@ def handler(event, context): data = json.loads(snapshot['Body'].read()) # Send to your issue tracker, dashboard, etc. - for finding in data['findings_by_type']['AURORA']: - if finding['status'] == 'RED': + # findings_by_type is keyed by resource config ID; Finding fields are + # PascalCase (no JSON tags on pkg/types.Finding). + for finding in data['findings_by_type'].get('aurora-mysql', []): + if finding['Status'] == 'RED': create_jira_ticket(finding) ``` diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index f1c85b2..815206d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -106,23 +106,28 @@ make dev - Use table-driven tests where appropriate - Mock external dependencies -Example: +Example (table-driven, mirrors the style used across `pkg/policy`, +`pkg/config`, and `pkg/eol/endoflife`): + ```go -func TestDetector_Detect(t *testing.T) { +func TestPolicy_Classify(t *testing.T) { tests := []struct { - name string - input *types.Resource - want *types.Finding - wantErr bool + name string + resource *types.Resource + lifecycle *types.VersionLifecycle + want types.Status }{ { - name: "detects red status", + name: "past EOL classifies RED", // ... }, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { - // ... + got := policy.NewDefaultPolicy().Classify(tt.resource, tt.lifecycle) + if got != tt.want { + t.Fatalf("Classify() = %s, want %s", got, tt.want) + } }) } } @@ -130,9 +135,15 @@ func TestDetector_Detect(t *testing.T) { ### Integration Tests -- Tag integration tests with `// +build integration` -- Require actual external dependencies (Wiz, AWS, etc.) -- Document setup requirements +- Tag integration tests with the modern build constraint — + `//go:build integration` on the first line, optionally followed by the legacy + `// +build integration` for older toolchains. See + `pkg/eol/endoflife/integration_test.go` for the reference pattern. +- Require actual external dependencies (Wiz, endoflife.date, AWS, etc.) and + are excluded from `make test` by default; run them with + `make test-integration`. +- Document setup requirements (env vars, credentials) in the test file's + header comment. ## Using AI Skills to Add Resources @@ -203,31 +214,48 @@ changes**. ``` Version-Guard/ ├── cmd/ -│ ├── server/ # Main server binary +│ ├── server/ # Main server binary (worker + HTTP admin) │ └── cli/ # CLI tool ├── pkg/ -│ ├── types/ # Core data structures -│ ├── policy/ # Classification policies -│ ├── inventory/ # Inventory sources (Wiz, mock) +│ ├── types/ # Core data structures (Resource, Finding, Snapshot, Status) +│ ├── config/ # YAML loader, transforms DSL, embedded defaults +│ │ └── defaults/ # Canonical resources.yaml shipped with the binary +│ ├── policy/ # Classification policy (RED/YELLOW/GREEN/UNKNOWN) +│ ├── inventory/ # Inventory sources (Wiz generic source + mock) │ ├── eol/ # EOL data providers (endoflife.date + schema adapters) -│ ├── store/ # Finding storage -│ ├── snapshot/ # S3 snapshot management -│ ├── workflow/ # Temporal workflows -│ ├── scan/ # Scan trigger (HTTP + CLI) -│ └── emitters/ # Emitter interfaces + examples -├── docs/ # Documentation -└── .github/ # GitHub workflows +│ ├── registry/ # Optional registry client for service lookups +│ ├── store/ # Finding storage interface (in-memory implementation) +│ ├── snapshot/ # Snapshot builder + store (S3 / in-memory backends) +│ ├── workflow/ # Temporal workflows (orchestrator + detection) +│ ├── scan/ # Scan trigger (shared by HTTP /scan and CLI) +│ ├── schedule/ # Temporal Schedule create-or-update wiring +│ └── emitters/ # Emitter interfaces + reference logging emitter +├── charts/version-guard/ # Helm chart +├── deploy/ # Dockerfile + endoflife.date override shim +└── .github/ # GitHub Actions workflows ``` ## Release Process -Releases are managed by maintainers: - -1. Update version in relevant files -2. Update CHANGELOG.md -3. Create a git tag: `git tag -a v1.0.0 -m "Release v1.0.0"` -4. Push tag: `git push origin v1.0.0` -5. GitHub Actions will build and publish release artifacts +Releases are managed by maintainers. The container image and Helm chart are +cut from a single `vX.Y.Z` git tag by the +[`Docker & Helm` workflow](./.github/workflows/docker.yml): + +1. If the chart has changed, bump `version` (and `appVersion` if the image + changed) in + [charts/version-guard/Chart.yaml](./charts/version-guard/Chart.yaml). + Land the bump on `main` as a `chore(release)` PR. +2. Tag and push: `git tag -a vX.Y.Z -m "Release vX.Y.Z" && git push origin vX.Y.Z`. + The tag's version **must equal** `charts/version-guard/Chart.yaml`'s + `version` whenever the chart changed since the previous tag — the workflow + fails the release otherwise. +3. The workflow then: + - builds and pushes a multi-arch container image to + `ghcr.io/block/Version-Guard` (semver + `latest` tags), and + - if the chart changed since the previous `v*` tag, packages and pushes + the Helm chart to `oci://ghcr.io/block/charts`. + +There is no `CHANGELOG.md` — release notes live on the GitHub Releases page. ## Questions? diff --git a/README.md b/README.md index 731dec3..0f11b1d 100644 --- a/README.md +++ b/README.md @@ -100,12 +100,12 @@ Version Guard uses a **config-driven approach** - resources are defined in `pkg/ |----------|-----------|------------|--------| | **EKS** (Kubernetes) | Wiz | [amazon-eks](https://endoflife.date/amazon-eks) | ✅ Production tested | | **ElastiCache** (Redis/Valkey/Memcached) | Wiz | [amazon-elasticache-redis](https://endoflife.date/amazon-elasticache-redis), [valkey](https://endoflife.date/valkey) | ✅ Production tested | -| **Aurora MySQL** | Wiz | [amazon-aurora-mysql](https://endoflife.date/amazon-aurora-mysql) | ⚠️ Production tested, EOL data pending [endoflife.date#9534](https://github.com/endoflife-date/endoflife.date/pull/9534) | -| **Aurora PostgreSQL** | Wiz | [amazon-aurora-postgresql](https://endoflife.date/amazon-aurora-postgresql) | 🔜 Config ready, needs Wiz report ID | +| **Aurora MySQL** | Wiz | [amazon-aurora-mysql](https://endoflife.date/amazon-aurora-mysql) | ⚠️ Production tested via the [`deploy/endoflife-override`](./deploy/endoflife-override/) shim while [endoflife.date#9534](https://github.com/endoflife-date/endoflife.date/pull/9534) is still open upstream | +| **Aurora PostgreSQL** | Wiz | [amazon-aurora-postgresql](https://endoflife.date/amazon-aurora-postgresql) | ✅ Production tested | | **OpenSearch** | Wiz | [amazon-opensearch](https://endoflife.date/amazon-opensearch), [elasticsearch](https://endoflife.date/elasticsearch) | ✅ Production tested | -| **RDS MySQL** | — | [amazon-rds-mysql](https://endoflife.date/amazon-rds-mysql) | 📋 Planned (add to config) | -| **RDS PostgreSQL** | — | [amazon-rds-postgresql](https://endoflife.date/amazon-rds-postgresql) | 📋 Planned (add to config) | -| **Lambda** | — | [aws-lambda](https://endoflife.date/aws-lambda) | 📋 Planned (add to config) | +| **RDS MySQL** | Wiz | [amazon-rds-mysql](https://endoflife.date/amazon-rds-mysql) | ✅ Production tested | +| **RDS PostgreSQL** | Wiz | [amazon-rds-postgresql](https://endoflife.date/amazon-rds-postgresql) | ✅ Production tested | +| **Lambda** | Wiz | [aws-lambda](https://endoflife.date/aws-lambda) | ✅ Production tested | **Adding a new resource type requires:** 1. A Wiz saved report for the resource type @@ -152,7 +152,9 @@ export WIZ_REPORT_IDS='{ "eks":"your-eks-report-id", "elasticache-redis":"your-elasticache-report-id", "elasticache-valkey":"your-elasticache-report-id", - "elasticache-memcached":"your-elasticache-report-id" + "elasticache-memcached":"your-elasticache-report-id", + "opensearch":"your-opensearch-report-id", + "lambda":"your-lambda-report-id" }' docker compose up --build ``` @@ -208,7 +210,7 @@ make dev # Or with real Wiz inventory (requires credentials) export WIZ_CLIENT_ID_SECRET="your-client-id" export WIZ_CLIENT_SECRET_SECRET="your-client-secret" -export WIZ_REPORT_IDS='{"aurora-mysql":"report-id","eks":"report-id","elasticache-redis":"report-id","elasticache-valkey":"report-id","elasticache-memcached":"report-id"}' +export WIZ_REPORT_IDS='{"aurora-mysql":"report-id","eks":"report-id","elasticache-redis":"report-id","elasticache-valkey":"report-id","elasticache-memcached":"report-id","opensearch":"report-id","lambda":"report-id"}' make dev ``` @@ -318,6 +320,9 @@ Version Guard is configured via environment variables or CLI flags: |----------|-------------|---------| | `TEMPORAL_ENDPOINT` | Temporal server address | `localhost:7233` | | `TEMPORAL_NAMESPACE` | Temporal namespace | `version-guard-dev` | +| `TEMPORAL_TASK_QUEUE` | Temporal task queue used by the worker | `version-guard-detection` | +| `TEMPORAL_METRICS_ENABLED` | Enable the Temporal Go SDK Prometheus/OpenMetrics endpoint | `true` | +| `TEMPORAL_METRICS_LISTEN_ADDRESS` | Prometheus listen address for Temporal SDK metrics | `0.0.0.0:9090` | | `HTTP_PORT` | HTTP admin port (`POST /scan`) | `8081` | | `S3_BUCKET` | S3 bucket for snapshots | `version-guard-snapshots` | | `AWS_REGION` | AWS region (for S3 snapshots) | `us-west-2` | @@ -395,7 +400,9 @@ export WIZ_REPORT_IDS='{ "eks": "your-eks-report-id", "elasticache-redis": "your-elasticache-report-id", "elasticache-valkey": "your-elasticache-report-id", - "elasticache-memcached": "your-elasticache-report-id" + "elasticache-memcached": "your-elasticache-report-id", + "opensearch": "your-opensearch-report-id", + "lambda": "your-lambda-report-id" }' ``` @@ -495,17 +502,34 @@ s3://your-bucket/snapshots/latest.json ``` **Snapshot Schema:** + +Snapshots are produced via Go's `encoding/json` defaults. Top-level fields on +`Snapshot` and `SnapshotSummary` carry explicit `snake_case` tags (see +[pkg/types/snapshot.go](./pkg/types/snapshot.go)); per-`Finding` fields have no +JSON tags and therefore serialize as PascalCase. The `findings_by_type` map is +keyed by the resource config ID (e.g. `aurora-mysql`, `eks`), not by the +`ResourceType` constants used in tests. + ```json { "snapshot_id": "scan-2026-04-09-123456", "version": "v3", "generated_at": "2026-04-09T12:34:56Z", + "scan_start_time": "2026-04-09T12:00:00Z", + "scan_end_time": "2026-04-09T12:34:56Z", + "scan_duration_sec": 2096, "findings_by_type": { - "aurora": [ + "aurora-mysql": [ { - "resource_id": "db-cluster-1", - "status": "red", - "message": "Running deprecated version 13.3 (EOL: 2025-03-01)" + "ResourceID": "db-cluster-1", + "ResourceType": "aurora-mysql", + "CloudProvider": "aws", + "CurrentVersion": "5.6.10a", + "Engine": "aurora-mysql", + "Status": "RED", + "Message": "Running deprecated version 5.6.10a (EOL: 2024-02-29)", + "DetectedAt": "2026-04-09T12:34:56Z", + "UpdatedAt": "2026-04-09T12:34:56Z" } ] }, @@ -514,6 +538,7 @@ s3://your-bucket/snapshots/latest.json "red_count": 12, "yellow_count": 35, "green_count": 103, + "unknown_count": 0, "compliance_percentage": 68.7 } } diff --git a/SKILLS.md b/SKILLS.md index 15b1ea6..9db55bb 100644 --- a/SKILLS.md +++ b/SKILLS.md @@ -6,7 +6,7 @@ AI agent skills automate complex workflows in Version Guard, enabling any AI age ### What are AI Skills? -AI skills are structured instruction sets that teach AI agents how to perform specific tasks autonomously. They follow the [Agent Skills standard](https://github.com/agent-skills/specification) with: +AI skills are structured instruction sets that teach AI agents how to perform specific tasks autonomously. They follow the conventions established by [Anthropic's Agent Skills](https://www.anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills) and adopted by Amp / Claude Code / Goose: - **YAML Frontmatter**: Metadata (name, description, version, status) - **Tool Allowlist**: Explicit list of permitted operations @@ -113,7 +113,8 @@ goose "Add OpenSearch to Version Guard" ### For Amp -Amp auto-discovers skills from the project's directory structure. +Amp auto-discovers skills from the project's `skills/` directory and surfaces +them in the system prompt for each thread opened in this repo. **Setup:** ```bash @@ -121,11 +122,12 @@ Amp auto-discovers skills from the project's directory structure. git clone https://github.com/block/Version-Guard.git cd Version-Guard -# Verify skill exists +# Verify the skill exists ls skills/add-version-guard-resource/SKILL.md -# Scan for skills (optional) -amp scan skills +# (Optional) inspect what Amp sees / install additional skills +amp skill list +amp skill info add-version-guard-resource ``` **Verification:** @@ -161,136 +163,59 @@ cp -r skills/add-version-guard-resource ~/.config/your-agent/skills/ ## Usage Examples -### Example 1: Adding Amazon OpenSearch +The skill produces YAML that conforms to the schema in +[pkg/config/types.go](pkg/config/types.go) — the real fields are +`id` / `type` / `cloud_provider` / `inventory.{source, native_type_pattern, +required_mappings, field_mappings}` / `transforms` / `eol.{provider, product, +schema, lifecycle}`. Wiz **report IDs are not in the YAML**; they live in the +`WIZ_REPORT_IDS` JSON env var, keyed by the resource `id`. Reshaping +inventory columns (extracting from JSON, stripping prefixes, normalizing +engine names) is declared in the `transforms` block — see +[TRANSFORMS.md](TRANSFORMS.md). -**Request:** -``` -Use the add-version-guard-resource skill to add OpenSearch support -``` +Reference outputs the skill produces for each canonical shape live alongside +it under +[skills/add-version-guard-resource/examples/](skills/add-version-guard-resource/examples/). -**What the AI agent does:** +### Example 1: OpenSearch — `strip_prefixes` + `from_version_major` -1. **Validates EOL data** - Queries endoflife.date API: - ``` - curl https://endoflife.date/api/opensearch.json - ``` +**Request:** `Use the add-version-guard-resource skill to add OpenSearch support` -2. **Gathers inputs** - Asks for: - - Resource ID: `opensearch` - - Display name: `OpenSearch` - - Wiz report ID: `wiz#report#abc123` +The skill validates EOL coverage on `amazon-opensearch` and `elasticsearch`, +auto-detects the Wiz `versionDetails.version` column shape (`OpenSearch_2.13`, +`Elasticsearch_7.10`), and generates a resource block that strips the prefix +and derives the engine from the major version. The final YAML matches what +ships in [pkg/config/defaults/resources.yaml](pkg/config/defaults/resources.yaml) +under `id: opensearch`. -3. **Detects schema** - Examines existing Wiz fixtures: - ``` - pkg/inventory/wiz/testdata/elasticache-redis.csv - ``` +**Time saved**: ~30–45 minutes → ~2–3 minutes. -4. **Checks adapters** - Verifies if EOL data needs custom parsing - -5. **Generates config** - Adds to `pkg/config/defaults/resources.yaml`: - ```yaml - - id: opensearch - name: OpenSearch - wiz_report_id: "wiz#report#abc123" - eol_product: opensearch - version_field: EngineVersion - name_field: ClusterName - schema_type: standard - ``` +### Example 2: Aurora PostgreSQL — `substring_lookup` (engine canonicalization) -6. **Runs tests** - Executes: - ```bash - go test ./pkg/config/... - go test ./pkg/inventory/wiz/... - go test ./pkg/workflow/detection/... - ``` +**Request:** `Add Aurora PostgreSQL to Version Guard` -7. **Creates commit**: - ``` - Add OpenSearch resource support +Aurora's `typeFields.kind` column says `AuroraPostgreSQL`, but +endoflife.date keys cycles by `aurora-postgresql`. The skill emits a +`substring_lookup` transform to canonicalize the engine — **no Go-side EOL +adapter required**, because the standard endoflife.date schema already covers +the lifecycle shape. See +[skills/add-version-guard-resource/examples/aurora-pg.yaml](skills/add-version-guard-resource/examples/aurora-pg.yaml). - - Added OpenSearch to pkg/config/defaults/resources.yaml - - EOL product: opensearch - - Wiz report: wiz#report#abc123 - - Uses standard schema (EngineVersion, ClusterName) - ``` +**Time saved**: ~30–60 minutes → ~2–3 minutes. -**Time saved**: ~30-45 minutes of manual work reduced to 2-3 minutes +### Example 3: EKS — `default_if_empty` + declarative lifecycle ---- - -### Example 2: Adding Amazon Aurora PostgreSQL - -**Request:** -``` -Add Aurora PostgreSQL to Version Guard -``` +**Request:** `Enable EKS detection in Version Guard` -**What's different:** +EKS reports have no engine column (the engine is implicitly `eks`) and use a +non-standard endoflife.date lifecycle that splits standard vs. extended +support. The skill produces a `default_if_empty` engine transform plus an +`eol.schema: declarative` block with a YAML `lifecycle` mapping the +upstream `eol` and `extendedSupport` fields onto Version Guard's +deprecation / extended-support windows. See +[skills/add-version-guard-resource/examples/eks.yaml](skills/add-version-guard-resource/examples/eks.yaml). -Aurora uses a **non-standard schema** with separate major/minor version fields: -- `EngineVersion` = `16.3` -- `EngineMajorVersion` = `16` - -The skill detects this by examining existing Aurora MySQL fixtures and prompts: -``` -Aurora uses non-standard schema with EngineMajorVersion. -Should we use the standard version_field or custom extraction? - -Options: -1. Use EngineVersion (standard) - recommended -2. Use custom extraction with EngineMajorVersion -``` - -If you choose option 1, the generated config uses: -```yaml -version_field: EngineVersion # Uses 16.3 format -``` - -The skill also detects that Aurora PostgreSQL needs a custom EOL adapter: -``` -endoflife.date uses product ID "amazon-rds-postgresql" -but Wiz data contains "aurora-postgresql". - -Created adapter in pkg/eol/endoflife/adapters.go: -- Handles "aurora-postgresql" → "amazon-rds-postgresql" mapping -``` - -**Time saved**: ~1-2 hours (including schema analysis and adapter creation) - ---- - -### Example 3: Adding EKS (Kubernetes) - -**Request:** -``` -Enable EKS detection in Version Guard -``` - -**What's different:** - -EKS uses a completely different Wiz schema (not AWS RDS): -- Version field: `K8sVersion` (not `EngineVersion`) -- Name field: `Name` (not `ClusterName`) - -The skill auto-detects this by examining `pkg/inventory/wiz/testdata/eks.csv`: -```csv -Name,K8sVersion,Region -my-eks-cluster,1.28,us-east-1 -``` - -Generated configuration: -```yaml -- id: eks - name: Amazon EKS - wiz_report_id: "wiz#report#eks123" - eol_product: amazon-eks - version_field: K8sVersion - name_field: Name - schema_type: non_standard -``` - -**Time saved**: ~45-60 minutes (including schema investigation) +**Time saved**: ~45–60 minutes → ~2–3 minutes. --- @@ -307,19 +232,25 @@ mkdir -p skills/your-skill-name/references ### 2. Create SKILL.md -**Required structure:** +**Required structure** (matches the +[`add-version-guard-resource` skill](skills/add-version-guard-resource/SKILL.md)): + ```markdown --- name: your-skill-name description: Brief description of what the skill does -version: 1.0.0 -status: beta -tool_allowlist: +roles: [] +metadata: + version: "1.0.0" + status: beta +user-invocable: true +disable-model-invocation: false +allowed-tools: - Read - Write - Edit - Bash(command_pattern:*) - - WebFetch(domain:example.com) + - WebFetch --- # Your Skill Name @@ -448,9 +379,11 @@ test -f pkg/workflow/detection/activities.go && echo "✅ Detection activities e ``` **Solutions:** -- Implement Version Guard Phase 1 (generic infrastructure) first -- Skills cannot work without the foundation in place -- See [config-driven approach documentation](scratch/config-driven-approach/) +- Make sure you're on a clean checkout of `main` so the generic + infrastructure (config loader, generic Wiz inventory source, detection + workflow) is present. +- Skills cannot work without that foundation — see + [ARCHITECTURE.md](ARCHITECTURE.md) for the component layout. --- @@ -493,19 +426,26 @@ go test ./pkg/workflow/detection/ -v **Common Issues:** -1. **Wiz report ID incorrect** - - Verify report ID in `pkg/config/defaults/resources.yaml` - - Check that Wiz report exists and is accessible +1. **Wiz report ID incorrect or missing** + - Wiz report IDs live in the `WIZ_REPORT_IDS` JSON env var (or + `.env` for docker-compose), keyed by the resource `id`. They are + **not** in `resources.yaml`. + - Confirm the report exists in Wiz and your credentials can access it. 2. **EOL product mismatch** - - Confirm product exists on endoflife.date - - Check for custom adapter requirements + - Confirm `eol.product` exists on endoflife.date. + - If the product has non-standard lifecycle semantics, switch + `eol.schema` to `declarative` and supply an `eol.lifecycle` block + (see [pkg/eol/endoflife/ADAPTERS.md](pkg/eol/endoflife/ADAPTERS.md)). -3. **Schema detection wrong** - - Manually verify Wiz CSV column names - - Update `version_field` or `name_field` if needed +3. **Wrong inventory column mapping** + - Cross-check against an existing fixture under + `pkg/inventory/wiz/testdata/` to confirm Wiz column names. + - Adjust `inventory.required_mappings` / `inventory.field_mappings`, + or add a `transforms` block (see [TRANSFORMS.md](TRANSFORMS.md)) if + the raw column needs reshaping. -**Solution**: Review generated configuration and adjust as needed. +**Solution**: Review the generated configuration and adjust as needed. --- @@ -602,7 +542,7 @@ We welcome contributions of new skills and improvements to existing ones! - [Troubleshooting Guide](skills/add-version-guard-resource/references/troubleshooting.md) **External Resources:** -- [Agent Skills Standard](https://github.com/agent-skills/specification) +- [Anthropic — Equipping agents for the real world with Agent Skills](https://www.anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills) - [endoflife.date API](https://endoflife.date/docs/api/) - [Version Guard Issues](https://github.com/block/Version-Guard/issues) diff --git a/USAGE.md b/USAGE.md index 816420a..1adf277 100644 --- a/USAGE.md +++ b/USAGE.md @@ -153,16 +153,20 @@ export S3_PREFIX=snapshots/ #### Run with Docker +The canonical container build is `deploy/Dockerfile`, which `docker-compose` +invokes automatically. To build the image standalone: + ```bash -# Build Docker image -make docker-build +# Build the server image +docker build -f deploy/Dockerfile -t version-guard:dev . -# Run container -docker run -p 8080:8080 \ +# Run the container (HTTP admin defaults to :8081, Temporal SDK metrics to :9090) +docker run --rm \ + -p 8081:8081 -p 9090:9090 \ -e TEMPORAL_ENDPOINT=host.docker.internal:7233 \ -e TEMPORAL_NAMESPACE=version-guard-dev \ -e AWS_REGION=us-west-2 \ - version-guard:latest + version-guard:dev ``` #### Run Full Stack with docker-compose (Recommended for Testing) @@ -204,10 +208,9 @@ docker compose ps docker compose logs version-guard # Look for these lines: -# ✓ Configuration loaded: 4 resource(s) defined +# ✓ Configuration loaded: N resource(s) defined (N = number of entries in resources.yaml; default is 10) # ✓ Wiz credentials configured — using live inventory -# ✓ Total detectors initialized: 3 -# ✓ Temporal worker starting on queue: version-guard-detection +# ✓ Connected to Temporal at localhost:7233 (namespace: version-guard-dev) # Version Guard is ready! ```