SREP-3620: Add feature-testing evidence collection commands

- osdctl aws cloudtrail errors: surface AWS permission errors
- osdctl cluster snapshot: capture cluster state
- osdctl cluster diff: compare snapshots
- osdctl evidence collect: all-in-one collection
- Support ROSA Classic and HCP clusters
- Add README documentation

Author: MitaliBhalla

README.md

@@ -635,3 +635,84 @@ Note : Here node-id refers to the node that has the unhealthy etcd member. This
 ```
 osdctl swarm secondary
 ```
+
+### Feature Testing Evidence Collection
+
+These commands help SRE teams collect evidence during feature validation testing (IAM policies, operators, etc.).
+
+#### AWS CloudTrail Errors
+
+Surface permission errors and other AWS API errors from CloudTrail. Useful when validating new IAM policies or features that interact with AWS APIs.
+
+```bash
+# Get permission errors from the last hour
+osdctl aws cloudtrail errors -C <cluster-id> --since 1h
+
+# Get errors from the last 30 minutes with JSON output
+osdctl aws cloudtrail errors -C <cluster-id> --since 30m --json
+
+# Get errors with AWS console links
+osdctl aws cloudtrail errors -C <cluster-id> --since 2h --link
+
+# Filter for specific error types
+osdctl aws cloudtrail errors -C <cluster-id> --since 1h --error-types AccessDenied,UnauthorizedOperation
+```
+
+**Note:** For ROSA HCP clusters, CloudTrail events only show customer account activity. Control plane activity is in Red Hat's account and not visible.
+
+#### Cluster Snapshot
+
+Capture a point-in-time snapshot of cluster state for evidence collection. The snapshot includes nodes, ClusterOperators, and namespaces.
+
+```bash
+# Capture cluster snapshot to a file
+osdctl cluster snapshot -C <cluster-id> -o before.yaml
+
+# Capture snapshot with specific namespaces
+osdctl cluster snapshot -C <cluster-id> -o snapshot.yaml --namespaces openshift-monitoring,openshift-operators
+
+# Capture additional resource types
+osdctl cluster snapshot -C <cluster-id> -o snapshot.yaml --resources pods,deployments,services
+```
+
+#### Cluster Diff
+
+Compare two cluster snapshots to identify changes. Useful for understanding what changed during feature testing.
+
+```bash
+# Compare two snapshots
+osdctl cluster diff before.yaml after.yaml
+
+# Compare snapshots with JSON output
+osdctl cluster diff before.yaml after.yaml --json
+```
+
+Changes are categorized as:
+- `+` added: Resource exists in after but not in before
+- `-` removed: Resource exists in before but not in after
+- `~` modified: Resource exists in both but with different values
+
+#### Evidence Collection (All-in-One)
+
+Collect comprehensive evidence from a cluster and AWS for feature testing. This all-in-one command gathers cluster state, CloudTrail events, and optionally Kubernetes events and must-gather output.
+
+```bash
+# Collect all evidence to a directory
+osdctl evidence collect -C <cluster-id> --output ./evidence/
+
+# Collect evidence from the last 2 hours
+osdctl evidence collect -C <cluster-id> --output ./evidence/ --since 2h
+
+# Collect evidence without CloudTrail (for non-AWS or limited access)
+osdctl evidence collect -C <cluster-id> --output ./evidence/ --skip-cloudtrail
+
+# Include Kubernetes events in collection
+osdctl evidence collect -C <cluster-id> --output ./evidence/ --include-events
+
+# Include must-gather output
+osdctl evidence collect -C <cluster-id> --output ./evidence/ --include-must-gather
+```
+
+The collected evidence includes:
+- `evidence.yaml` - Main evidence file with cluster state and CloudTrail data
+- `summary.txt` - Human-readable summary of findings

cmd/aws/cloudtrail/cmd.go

@@ -0,0 +1,24 @@
+package cloudtrail
+
+import (
+	"github.com/spf13/cobra"
+)
+
+// NewCmdCloudtrail returns the cloudtrail command group under aws
+func NewCmdCloudtrail() *cobra.Command {
+	cloudtrailCmd := &cobra.Command{
+		Use:   "cloudtrail",
+		Short: "AWS CloudTrail utilities for feature testing",
+		Long: `AWS CloudTrail utilities for feature testing and evidence collection.
+
+Use these commands to surface CloudTrail events, particularly permission
+errors that occur during feature validation testing.`,
+		Run: func(cmd *cobra.Command, args []string) {
+			_ = cmd.Help()
+		},
+	}
+
+	cloudtrailCmd.AddCommand(newCmdErrors())
+
+	return cloudtrailCmd
+}