feat: add options.WithHTTPTransportWrapper to inject HTTP middleware in provider clients#3090
Conversation
…in provider clients Adds a new Opt that allows callers embedding docker-agent programmatically to wrap the HTTP transport used by all provider clients. The wrapper function receives the already-built transport (post-OTel, post-SSE-filter, post-Desktop-proxy) and must return a new RoundTripper that delegates to it. Changes: - options.ModelOptions: new transportWrapper field, WithHTTPTransportWrapper Opt, TransportWrapper() accessor, and FromModelOptions round-trip support - Anthropic provider: wrapper applied in direct and gateway paths - OpenAI provider: wrapper applied in direct and gateway paths - Gemini provider: wrapper applied in direct and gateway paths (GeminiAPI backend); Vertex AI path logs a warning (AWS SDK manages its own client) - Bedrock provider: logs a warning (AWS SDK manages its own client) - All four active call sites guard against a nil return from the wrapper (original transport preserved, slog.Warn emitted) - Tests: unit tests in options package; direct and gateway integration tests in anthropic, openai, and gemini provider packages Fixes docker#3089
| } else { | ||
| slog.WarnContext(ctx, "HTTP transport wrapper returned nil; using original transport") | ||
| } | ||
| } |
There was a problem hiding this comment.
[MEDIUM] Transport wrapper silently not applied to WebSocket pool
The wrapper is correctly applied to the REST httpClient here, but when transport: websocket is configured (checked later at getTransport(cfg) == "websocket"), the wsPool is created via newWSPool(httpToWSURL(baseURL), client.buildWSHeaderFn()) — using only auth headers, not the transport wrapper. Callers relying on the wrapper for observability, tracing, or auth injection will silently get no coverage over WebSocket streaming calls.
Both the Bedrock and Gemini Vertex AI paths emit a slog.Warn when the wrapper cannot be applied; this path should do the same. Consider adding near the wsPool creation:
if w := globalOptions.TransportWrapper(); w != nil {
slog.WarnContext(ctx, "HTTP transport wrapper is set but not applied: WebSocket transport uses an SDK-managed connection")
}There was a problem hiding this comment.
why can't we apply it to the websocket ?
There was a problem hiding this comment.
Good question! Technically we could apply it, but it would require more work than just emitting the warning.
The WebSocket pool uses gorilla/websocket.Dialer (line 92 in ws_stream.go), which only accepts headers for the handshake — it doesn't expose an http.RoundTripper hook where the transport wrapper could be injected. To apply the wrapper, you'd need to either:
- Use a custom dialer that wraps the underlying TCP connection (messy), or
- Switch to a WebSocket library that accepts an
http.Clientwith a pluggable transport
Since WebSocket connections bypass the normal HTTP request/response cycle after the initial handshake, most observability wrappers won't capture the streaming data anyway — they'd only see the upgrade request. The warning (already present at line 175) lets users know their wrapper won't cover WebSocket traffic, which matches what Bedrock and Gemini do when SDK-managed clients can't be wrapped.
If you'd prefer to make the wrapper work, I'd suggest looking into whether nhooyr.io/websocket (which accepts *http.Client) would fit better. Otherwise the current approach (warn + document) seems reasonable.
- Replace assert.Greater(t, x, int64(0)) with assert.Positive(t, x) in all three wrap_transport_test files to satisfy testifylint - Emit slog.WarnContext when transport wrapper is set but OpenAI WebSocket transport is in use (consistent with Bedrock and Gemini Vertex AI)
|
Fixed in 59057b5:
|
| } | ||
|
|
||
| if globalOptions.TransportWrapper() != nil { | ||
| slog.WarnContext(ctx, "HTTP transport wrapper is set but not applied: Bedrock provider uses the AWS SDK HTTP client") |
There was a problem hiding this comment.
There really is no way to give bedrock an http?
There was a problem hiding this comment.
I do not understand: I see the transport wrapper applied on line 99. Why do we have this warning ?
The AWS SDK v2 supports a custom HTTP client via bedrockruntime.Options.HTTPClient (which accepts any aws.HTTPClient, and *http.Client satisfies that interface). Replace the warn-and-skip approach with the same pattern used for Anthropic, OpenAI, and Gemini: - Build the docker-agent HTTP client (OTel, SSE decompression, user-agent) - If a bearer token is configured, chain bearerTokenTransport on top of that base (previously it wrapped http.DefaultTransport, bypassing OTel/SSE entirely) - Apply the caller's transport wrapper over the full chain - Inject the result into bedrockruntime.Options.HTTPClient Also remove the Bedrock limitation bullet from the WithHTTPTransportWrapper docstring since it is now fully supported.
|
Re: rumpl's comment on Yes, the AWS SDK v2 absolutely supports a custom HTTP client: The fix:
The Bedrock limitation bullet has been removed from the |
The warning 'HTTP transport wrapper is set but not applied' was a leftover from before the wrapper was wired in (d7c5411). The wrapper is now applied at line 99 via bedrockruntime.Options.HTTPClient, so the upfront warn block was wrong and is removed.
|
Re: simonferquel's question about the WebSocket path — the warning is accurate and must stay. The transport wrapper is For WebSocket, gorilla/websocket's It dials a raw TCP connection via The gorilla API only exposes Auth for the WebSocket handshake is already handled separately via |
|
OpenAI is supported trough cagent-proxy, including with injecting the docker-desktop bearer token. How does it work in that case ? |
|
Good question — this is actually a non-issue because WebSocket and gateway are mutually exclusive in the existing code. Two guards enforce this: At client construction ( if getTransport(cfg) == "websocket" && globalOptions.Gateway() == "" {
client.wsPool = newWSPool(...) // only created in direct mode
}At stream selection ( if transport == "websocket" && c.ModelOptions.Gateway() == "" {
// use WebSocket
} else if transport == "websocket" {
// Gateway is set → log debug + fall through to SSE
}So when OpenAI is used through cagent-proxy (gateway configured), the code unconditionally falls back to SSE regardless of the The |
|
Ok, do the same when a transport wrapper is specified then (fallback to sse automatically). Vertex AI is not supported by the cagent proxy in some similar fashion ? |
…bsocket gorilla/websocket dials raw TCP/TLS and never calls http.RoundTripper, so an HTTP transport wrapper cannot intercept WebSocket connections. Rather than warning and silently dropping the wrapper, add TransportWrapper() == nil to the WebSocket guards in both NewClient and CreateResponseStream so that callers who register a wrapper automatically get SSE (where the wrapper is fully applied) instead of WebSocket. Changes: - NewClient: wsPool is not created when TransportWrapper() != nil - CreateResponseStream: split the fallback log into two debug messages -- one for the gateway case, one for the transport-wrapper case - Remove the now-unnecessary slog.Warn about WebSocket bypassing the wrapper - Add TestNewClient_WebSocketFallsBackToSSEWhenTransportWrapperSet
|
Both addressed in 5bf5128. Request 1 — WebSocket auto-fallback to SSE when transport wrapper is set Done. Added
The old Request 2 — Vertex AI via cagent-proxy Same mutual-exclusivity pattern as WebSocket. Looking at if gateway := globalOptions.Gateway(); gateway == "" {
// Direct path: may use Vertex AI if project/location are set
// (httpClient = nil → slog.Warn when wrapper is set)
} else {
// Gateway path: always uses BackendGeminiAPI with httpclient.NewHTTPClient
// Vertex AI is never reached here
}When a gateway is configured (cagent-proxy / Docker Desktop), the code always takes the |
|
Ok, use the same logic as with openAI: if a custom wrapper is specified, use genai.Client with BackendGeminiAPI |
…pper is set Lint fix: - Rewrite the if-else chain in openai/client.go (CreateResponseStream) as a switch to satisfy the gocritic ifElseChain linter rule Vertex AI fallback (mirrors WebSocket → SSE): - In gemini/client.go, compute useVertexAI = wantVertexAI && TransportWrapper() == nil - When a transport wrapper is registered, override the Vertex AI backend selection and use BackendGeminiAPI + httpclient.NewHTTPClient so the wrapper can intercept all requests - Remove the slog.Warn that previously fired when wrapper + Vertex AI coexisted - Update wrap_transport_test.go: replace TestNewClient_TransportWrapperVertexAIWarns with TestNewClient_TransportWrapperVertexAIFallsBackToGeminiAPI which asserts the wrapper IS called on a real stream when project/location are set
|
Both fixed in 05e6902. Lint — replaced the Vertex AI fallback — mirrors the WebSocket → SSE pattern exactly: _, useVertexAIEnv := env.Get(ctx, "GOOGLE_GENAI_USE_VERTEXAI")
wantVertexAI := cfg.ProviderOpts["project"] != nil || cfg.ProviderOpts["location"] != nil || useVertexAIEnv
useVertexAI := wantVertexAI && globalOptions.TransportWrapper() == nilWhen a transport wrapper is set and Vertex AI would normally be used,
|
Closes #3089
Adds
options.WithHTTPTransportWrapper— a newOptthat lets callers embedding docker-agent inject HTTP middleware into the transport chain of all provider clients.What changed
options.ModelOptions: newtransportWrapperfield,WithHTTPTransportWrapperOpt,TransportWrapper()accessor,FromModelOptionsround-trip supportslog.Warnis emitted and limitations are documentedTests
optionspackage (field set/get,FromModelOptionsround-trip)anthropic,openai, andgeminipackages (httptest server + counting transport)Deliberate decisions
if wrapped := w(...); wrapped != nil) appears at each call site rather than in a shared helper. This avoids adding a new exported symbol and keeps the pattern visible to reviewers; the six occurrences are mechanical.