feat: integrate Pulumi event stream with OpenTelemetry for provisioning audit

[adrianriobo]

Summary

Currently, the otelcol-contrib filelog collector runs on the provisioned machine and ships runtime logs (syslog, auth, audit, gitlab-runner) to an OTLP endpoint. However, Pulumi provisioning and decommissioning events are not captured anywhere — meaning there is no audit trail for who created what, when, and whether it succeeded.

This issue tracks integrating the Pulumi Automation API event stream with OpenTelemetry so that stack-up and stack-destroy operations are also exported as OTel log records.

Motivation

For IBM Power and IBM Z targets running in an internal network, auditability of all actions is a hard requirement — not just what runs on the machine, but also the provisioning lifecycle itself. GitLab job logs are already being shipped via otelcol, but the provisioning events that precede and follow any job are currently dark.

Proposed approach

1. Hook into the Pulumi Automation API event stream

auto.Stack.Up() and auto.Stack.Destroy() accept auto.EventStreams(ch), which delivers typed events in real time:

• engine.ResourcePreEvent — before a resource operation starts
• engine.ResourceOutputsEvent — after a resource is created/updated
• engine.DiagnosticEvent — log messages from providers/programs
• engine.SummaryEvent — final outcome (succeeded/failed, duration, resource change counts)
• engine.CancelEvent — cancellation

2. Convert events to OTel log records

Consume the channel in a goroutine inside pkg/manager (where UpStack/DestroyStack live). Convert each relevant event to an OTel log record using the Go OTel SDK and export directly to the configured --otel-endpoint.

Relevant resource attributes (matching what the filelog collector already attaches):

• appcode — from --otel-app-code
• com.redhat.otel.auth_token — from --otel-auth-token
• project-name — from --project-name
• operation — up or destroy
• stack — Pulumi stack name

3. Dependencies

Add to the mapt binary:

• go.opentelemetry.io/otel
• go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp
• go.opentelemetry.io/otel/sdk/log

The --otel-* flags are already first-class CLI params on all targets that support otelcol, so no new flags are needed.

4. Integration point

The event consumer would be enabled only when OtelAppCode and OtelAuthToken are set in the mapt context (pkg/manager/context). The manager's UpStack/DestroyStack wrappers are the right place to attach it, keeping the concern isolated from individual provider implementations.

Tradeoffs

	This approach	Current otelcol on machine
Captures provisioning events	✅	❌ (machine doesn't exist yet)
Captures runtime logs	❌	✅
Requires otelcol on target	No	Yes
OTel SDK in mapt binary	Yes	No
Needs new flags	No	No

The two approaches are complementary and can coexist.

Related

• PR adding GitLab runner + otelcol support for IBM Power and IBM Z: feat(ibmcloud): add GitLab runner support for IBM Power (ppc64le) and IBM Z (s390x) #812