Skip to content

feat: add options.WithHTTPTransportWrapper to inject HTTP middleware in provider clients#3090

Open
simonferquel-clanker wants to merge 7 commits into
docker:mainfrom
simonferquel-clanker:feat/http-transport-wrapper
Open

feat: add options.WithHTTPTransportWrapper to inject HTTP middleware in provider clients#3090
simonferquel-clanker wants to merge 7 commits into
docker:mainfrom
simonferquel-clanker:feat/http-transport-wrapper

Conversation

@simonferquel-clanker

Copy link
Copy Markdown
Contributor

Closes #3089

Adds options.WithHTTPTransportWrapper — a new Opt that lets callers embedding docker-agent inject HTTP middleware into the transport chain of all provider clients.

What changed

  • options.ModelOptions: new transportWrapper field, WithHTTPTransportWrapper Opt, TransportWrapper() accessor, FromModelOptions round-trip support
  • Anthropic, OpenAI, Gemini (GeminiAPI backend): wrapper applied in both direct and gateway paths
  • Gemini Vertex AI / Bedrock: wrapper cannot be applied (SDK-managed HTTP clients); a slog.Warn is emitted and limitations are documented
  • All call sites guard nil returns from the wrapper (original transport preserved, warning logged)

Tests

  • Unit tests in options package (field set/get, FromModelOptions round-trip)
  • Direct-path and gateway-path integration tests in anthropic, openai, and gemini packages (httptest server + counting transport)
  • Vertex AI no-op path verified in Gemini tests

Deliberate decisions

  • The nil-return guard (if wrapped := w(...); wrapped != nil) appears at each call site rather than in a shared helper. This avoids adding a new exported symbol and keeps the pattern visible to reviewers; the six occurrences are mechanical.
  • Bedrock and Gemini Vertex AI are documented as non-supported rather than wired, because those SDKs own their HTTP transport stack.

…in provider clients

Adds a new Opt that allows callers embedding docker-agent programmatically
to wrap the HTTP transport used by all provider clients.

The wrapper function receives the already-built transport (post-OTel,
post-SSE-filter, post-Desktop-proxy) and must return a new RoundTripper
that delegates to it.

Changes:
- options.ModelOptions: new transportWrapper field, WithHTTPTransportWrapper
  Opt, TransportWrapper() accessor, and FromModelOptions round-trip support
- Anthropic provider: wrapper applied in direct and gateway paths
- OpenAI provider: wrapper applied in direct and gateway paths
- Gemini provider: wrapper applied in direct and gateway paths (GeminiAPI
  backend); Vertex AI path logs a warning (AWS SDK manages its own client)
- Bedrock provider: logs a warning (AWS SDK manages its own client)
- All four active call sites guard against a nil return from the wrapper
  (original transport preserved, slog.Warn emitted)
- Tests: unit tests in options package; direct and gateway integration tests
  in anthropic, openai, and gemini provider packages

Fixes docker#3089
@simonferquel-clanker simonferquel-clanker requested a review from a team as a code owner June 12, 2026 13:06

@docker-agent docker-agent left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assessment: 🟡 NEEDS ATTENTION

} else {
slog.WarnContext(ctx, "HTTP transport wrapper returned nil; using original transport")
}
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MEDIUM] Transport wrapper silently not applied to WebSocket pool

The wrapper is correctly applied to the REST httpClient here, but when transport: websocket is configured (checked later at getTransport(cfg) == "websocket"), the wsPool is created via newWSPool(httpToWSURL(baseURL), client.buildWSHeaderFn()) — using only auth headers, not the transport wrapper. Callers relying on the wrapper for observability, tracing, or auth injection will silently get no coverage over WebSocket streaming calls.

Both the Bedrock and Gemini Vertex AI paths emit a slog.Warn when the wrapper cannot be applied; this path should do the same. Consider adding near the wsPool creation:

if w := globalOptions.TransportWrapper(); w != nil {
    slog.WarnContext(ctx, "HTTP transport wrapper is set but not applied: WebSocket transport uses an SDK-managed connection")
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why can't we apply it to the websocket ?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question! Technically we could apply it, but it would require more work than just emitting the warning.

The WebSocket pool uses gorilla/websocket.Dialer (line 92 in ws_stream.go), which only accepts headers for the handshake — it doesn't expose an http.RoundTripper hook where the transport wrapper could be injected. To apply the wrapper, you'd need to either:

  1. Use a custom dialer that wraps the underlying TCP connection (messy), or
  2. Switch to a WebSocket library that accepts an http.Client with a pluggable transport

Since WebSocket connections bypass the normal HTTP request/response cycle after the initial handshake, most observability wrappers won't capture the streaming data anyway — they'd only see the upgrade request. The warning (already present at line 175) lets users know their wrapper won't cover WebSocket traffic, which matches what Bedrock and Gemini do when SDK-managed clients can't be wrapped.

If you'd prefer to make the wrapper work, I'd suggest looking into whether nhooyr.io/websocket (which accepts *http.Client) would fit better. Otherwise the current approach (warn + document) seems reasonable.

- Replace assert.Greater(t, x, int64(0)) with assert.Positive(t, x) in
  all three wrap_transport_test files to satisfy testifylint
- Emit slog.WarnContext when transport wrapper is set but OpenAI WebSocket
  transport is in use (consistent with Bedrock and Gemini Vertex AI)
@simonferquel-clanker

Copy link
Copy Markdown
Contributor Author

Fixed in 59057b5:

  • Lint: replaced assert.Greater(t, x, int64(0)) with assert.Positive(t, x) in all three wrap_transport_test files (testifylint negative-positive rule).
  • WebSocket warning: added slog.WarnContext adjacent to the wsPool creation when globalOptions.TransportWrapper() != nil — consistent with the Bedrock and Gemini Vertex AI paths.

Comment thread pkg/model/provider/bedrock/client.go Outdated
}

if globalOptions.TransportWrapper() != nil {
slog.WarnContext(ctx, "HTTP transport wrapper is set but not applied: Bedrock provider uses the AWS SDK HTTP client")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There really is no way to give bedrock an http?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not understand: I see the transport wrapper applied on line 99. Why do we have this warning ?

The AWS SDK v2 supports a custom HTTP client via bedrockruntime.Options.HTTPClient
(which accepts any aws.HTTPClient, and *http.Client satisfies that interface).

Replace the warn-and-skip approach with the same pattern used for Anthropic,
OpenAI, and Gemini:
- Build the docker-agent HTTP client (OTel, SSE decompression, user-agent)
- If a bearer token is configured, chain bearerTokenTransport on top of that
  base (previously it wrapped http.DefaultTransport, bypassing OTel/SSE entirely)
- Apply the caller's transport wrapper over the full chain
- Inject the result into bedrockruntime.Options.HTTPClient

Also remove the Bedrock limitation bullet from the WithHTTPTransportWrapper
docstring since it is now fully supported.
@simonferquel-clanker

Copy link
Copy Markdown
Contributor Author

Re: rumpl's comment on bedrock/client.go:64 — fixed in d7c5411.

Yes, the AWS SDK v2 absolutely supports a custom HTTP client: bedrockruntime.Options.HTTPClient accepts anything that implements Do(*http.Request) (*http.Response, error), and *http.Client satisfies that interface.

The fix:

  • Builds the docker-agent HTTP client (httpclient.NewHTTPClient) so Bedrock also gets OTel, SSE decompression, and user-agent transport.
  • Chains bearerTokenTransport on top of that base when a bearer token is set (previously it wrapped http.DefaultTransport, bypassing OTel/SSE entirely — a bonus improvement).
  • Applies the caller's transport wrapper over the full chain with the same nil-guard pattern used elsewhere.
  • Injects the result via o.HTTPClient = httpClient in the bedrockruntime.Options closure.

The Bedrock limitation bullet has been removed from the WithHTTPTransportWrapper docstring.

@aheritier aheritier added area/providers For features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.) area/providers/openai For features/issues/fixes related to the usage of OpenAI models area/providers/anthropic For features/issues/fixes related to the usage of Anthropic models area/providers/bedrock AWS Bedrock provider support area/providers/gemini Google Gemini provider support kind/feat PR adds a new feature (maps to feat: commit prefix) labels Jun 12, 2026
The warning 'HTTP transport wrapper is set but not applied' was a leftover
from before the wrapper was wired in (d7c5411). The wrapper is now applied
at line 99 via bedrockruntime.Options.HTTPClient, so the upfront warn block
was wrong and is removed.
@simonferquel-clanker

Copy link
Copy Markdown
Contributor Author

Good catch — that slog.Warn at line 65 was a leftover from before the wrapper was wired in (d7c5411). Since the wrapper is now correctly applied further down via bedrockruntime.Options.HTTPClient, the upfront warning was wrong. Removed in b387a68.

@simonferquel-clanker

Copy link
Copy Markdown
Contributor Author

Re: simonferquel's question about the WebSocket path — the warning is accurate and must stay.

The transport wrapper is func(http.RoundTripper) http.RoundTripper. For REST calls the wrapper intercepts http.RoundTripper.RoundTrip, which is the right place for auth injection, tracing, etc.

For WebSocket, gorilla/websocket's Dialer never calls RoundTrip at all. Looking at its implementation:

// gorilla/websocket: Dialer.DialContext (client.go)
type Dialer struct {
    NetDialContext func(ctx context.Context, network, addr string) (net.Conn, error)
    NetDialTLSContext ...
    Proxy func(*http.Request) (*url.URL, error)
    TLSClientConfig *tls.Config
    // ...no http.RoundTripper field
}

It dials a raw TCP connection via net.Dialer, performs TLS itself (tls.Client), then writes the HTTP upgrade request as raw bytes over that socket. http.RoundTripper is completely bypassed — there is no intercept point for the wrapper in that code path.

The gorilla API only exposes NetDialContext (a net.Conn factory) as a hook, which is a different abstraction. We can't feed a func(http.RoundTripper) http.RoundTripper into it in any meaningful way.

Auth for the WebSocket handshake is already handled separately via buildWSHeaderFn() which attaches auth headers before the dial. The slog.Warn is kept so callers know upfront that their wrapper is not covering WebSocket streaming calls.

@simonferquel

Copy link
Copy Markdown
Contributor

OpenAI is supported trough cagent-proxy, including with injecting the docker-desktop bearer token. How does it work in that case ?

@simonferquel-clanker

Copy link
Copy Markdown
Contributor Author

Good question — this is actually a non-issue because WebSocket and gateway are mutually exclusive in the existing code.

Two guards enforce this:

At client construction (NewClient, line 195):

if getTransport(cfg) == "websocket" && globalOptions.Gateway() == "" {
    client.wsPool = newWSPool(...)   // only created in direct mode
}

At stream selection (CreateChatCompletionStream, line 536):

if transport == "websocket" && c.ModelOptions.Gateway() == "" {
    // use WebSocket
} else if transport == "websocket" {
    // Gateway is set → log debug + fall through to SSE
}

So when OpenAI is used through cagent-proxy (gateway configured), the code unconditionally falls back to SSE regardless of the transport: websocket setting. The Docker Desktop bearer token injection happens in the SSE gateway clientFn closure — that path uses our httpclient.NewHTTPClient transport chain, and the transport wrapper IS applied there.

The slog.Warn for the WebSocket case is therefore only ever emitted in direct mode (no gateway, no Docker Desktop JWT involved). There is no gap in the proxy/bearer-token scenario.

@simonferquel

Copy link
Copy Markdown
Contributor

Ok, do the same when a transport wrapper is specified then (fallback to sse automatically).

Vertex AI is not supported by the cagent proxy in some similar fashion ?

…bsocket

gorilla/websocket dials raw TCP/TLS and never calls http.RoundTripper, so
an HTTP transport wrapper cannot intercept WebSocket connections. Rather
than warning and silently dropping the wrapper, add TransportWrapper() == nil
to the WebSocket guards in both NewClient and CreateResponseStream so that
callers who register a wrapper automatically get SSE (where the wrapper is
fully applied) instead of WebSocket.

Changes:
- NewClient: wsPool is not created when TransportWrapper() != nil
- CreateResponseStream: split the fallback log into two debug messages --
  one for the gateway case, one for the transport-wrapper case
- Remove the now-unnecessary slog.Warn about WebSocket bypassing the wrapper
- Add TestNewClient_WebSocketFallsBackToSSEWhenTransportWrapperSet
@simonferquel-clanker

Copy link
Copy Markdown
Contributor Author

Both addressed in 5bf5128.

Request 1 — WebSocket auto-fallback to SSE when transport wrapper is set

Done. Added && globalOptions.TransportWrapper() == nil to both the wsPool creation guard in NewClient and the if transport == "websocket" branch in CreateResponseStream. When a wrapper is registered:

  • wsPool is never created (no WebSocket connections will be opened)
  • CreateResponseStream logs a debug message and falls straight through to the SSE path, where the wrapper is fully applied

The old slog.Warn is gone. A new test TestNewClient_WebSocketFallsBackToSSEWhenTransportWrapperSet asserts that wsPool is nil when a wrapper is set alongside transport: websocket.


Request 2 — Vertex AI via cagent-proxy

Same mutual-exclusivity pattern as WebSocket. Looking at gemini/client.go:

if gateway := globalOptions.Gateway(); gateway == "" {
    // Direct path: may use Vertex AI if project/location are set
    // (httpClient = nil → slog.Warn when wrapper is set)
} else {
    // Gateway path: always uses BackendGeminiAPI with httpclient.NewHTTPClient
    // Vertex AI is never reached here
}

When a gateway is configured (cagent-proxy / Docker Desktop), the code always takes the else branch and builds a genai.Client with BackendGeminiAPI and our httpclient.NewHTTPClient transport. The Vertex AI path (httpClient = nil) is only reachable in direct mode (no gateway). So the existing slog.Warn for Vertex AI is accurate and only ever fires in direct mode where cagent-proxy is not involved — no gap.

@simonferquel

Copy link
Copy Markdown
Contributor

Ok, use the same logic as with openAI: if a custom wrapper is specified, use genai.Client with BackendGeminiAPI

…pper is set

Lint fix:
- Rewrite the if-else chain in openai/client.go (CreateResponseStream) as a
  switch to satisfy the gocritic ifElseChain linter rule

Vertex AI fallback (mirrors WebSocket → SSE):
- In gemini/client.go, compute useVertexAI = wantVertexAI && TransportWrapper() == nil
- When a transport wrapper is registered, override the Vertex AI backend
  selection and use BackendGeminiAPI + httpclient.NewHTTPClient so the
  wrapper can intercept all requests
- Remove the slog.Warn that previously fired when wrapper + Vertex AI coexisted
- Update wrap_transport_test.go: replace TestNewClient_TransportWrapperVertexAIWarns
  with TestNewClient_TransportWrapperVertexAIFallsBackToGeminiAPI which asserts
  the wrapper IS called on a real stream when project/location are set
@simonferquel-clanker

Copy link
Copy Markdown
Contributor Author

Both fixed in 05e6902.

Lint — replaced the if/else if/else if chain in CreateResponseStream with a switch statement to satisfy the gocritic ifElseChain rule.

Vertex AI fallback — mirrors the WebSocket → SSE pattern exactly:

_, useVertexAIEnv := env.Get(ctx, "GOOGLE_GENAI_USE_VERTEXAI")
wantVertexAI := cfg.ProviderOpts["project"] != nil || cfg.ProviderOpts["location"] != nil || useVertexAIEnv
useVertexAI := wantVertexAI && globalOptions.TransportWrapper() == nil

When a transport wrapper is set and Vertex AI would normally be used, useVertexAI is false and the code falls through to the GeminiAPI branch (with httpclient.NewHTTPClient + wrapper applied). A debug log records the fallback. The old slog.Warn is removed.

TestNewClient_TransportWrapperVertexAIWarns replaced by TestNewClient_TransportWrapperVertexAIFallsBackToGeminiAPI which spins up an httptest server, sets project/location + GOOGLE_API_KEY, and asserts the wrapper's RoundTrip is called on a real stream.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/providers/anthropic For features/issues/fixes related to the usage of Anthropic models area/providers/bedrock AWS Bedrock provider support area/providers/gemini Google Gemini provider support area/providers/openai For features/issues/fixes related to the usage of OpenAI models area/providers For features/issues/fixes related to LLM providers (Bedrock, LiteLLM, Qwen, custom, etc.) kind/feat PR adds a new feature (maps to feat: commit prefix)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

options.WithHTTPTransportWrapper: allow callers to inject HTTP middleware around all provider clients

5 participants