Add async iterable support to `ChatClient.completeStreamingChat()` and `AudioClient.transcribeStreaming()` by Copilot · Pull Request #521 · microsoft/Foundry-Local

Copilot · 2026-03-12T16:36:40Z

completeStreamingChat() and transcribeStreaming() only accepted callbacks, causing confusing runtime errors when developers used the OpenAI SDK's familiar for await pattern. This adds async iterable support to both clients while keeping the callback overloads.

API changes

`ChatClient.completeStreamingChat()`

Two new overloads return AsyncIterable<any> when no callback is passed:

// New: async iterable (consistent with OpenAI SDK)
for await (const chunk of chatClient.completeStreamingChat(messages)) {
  process.stdout.write(chunk.choices?.[0]?.delta?.content ?? '');
}

// New: with tools
for await (const chunk of chatClient.completeStreamingChat(messages, tools)) { ... }

// Callback pattern — also supported
await chatClient.completeStreamingChat(messages, (chunk) => { ... });
await chatClient.completeStreamingChat(messages, tools, (chunk) => { ... });

`AudioClient.transcribeStreaming()`

Same pattern applied:

// New: async iterable
for await (const chunk of audioClient.transcribeStreaming(audioFilePath)) {
  process.stdout.write(chunk.text ?? '');
}

// Callback pattern — also supported
await audioClient.transcribeStreaming(audioFilePath, (chunk) => { ... });

Implementation

_streamChat() — private async generator that bridges executeCommandStreaming's native callback to an async iterable via a chunk queue + wakeConsumer notify pattern. await streamPromise in the finally block ensures native resources are cleaned up even on early iterator exit.
_completeStreamingChatWithCallback() — extracted from the original completeStreamingChat() implementation; behavior unchanged.
_transcribeStream() — same async generator pattern for audio streaming.
_transcribeStreamingWithCallback() — extracted from the original transcribeStreaming() implementation; behavior unchanged.
Public methods dispatch to the appropriate implementation based on whether a callback argument is present.

Other

Tests: added async iterable integration tests for both clients; updated validation tests to cover both patterns.
Docs (ChatClient.md, AudioClient.md) and sample app updated to reflect all overloads.
C# SDK already returns IAsyncEnumerable<T> from its streaming methods — no changes needed there.

Original prompt

This section details on the original issue you should resolve

<issue_title>JavaScript ChatClient Streaming Uses Callbacks, Not Async Iterators</issue_title>
<issue_description>

JavaScript ChatClient Streaming Uses Callbacks, Not Async Iterators

Severity: Documentation gap
Component: foundry-local-sdk JavaScript v0.9.x — ChatClient.completeStreamingChat()

The ChatClient returned by model.createChatClient() provides a completeStreamingChat() method, but it uses a callback pattern rather than returning an async iterable:
// ❌ This does NOT work — throws "stream is not async iterable"
for await (const chunk of chatClient.completeStreamingChat(messages)) { ... }

// ✅ Correct pattern — pass a callback
await chatClient.completeStreamingChat(messages, (chunk) => {
  process.stdout.write(chunk.choices?.[0]?.delta?.content ?? "");
});
Impact: Developers familiar with the OpenAI SDK's async iteration pattern (for await) will encounter confusing errors. The callback must be a valid function or the SDK throws "Callback must be a valid function."

Expected: Document the callback pattern in the SDK reference. Alternatively, support the async iterable pattern for consistency with the OpenAI SDK.

---</issue_description>

<agent_instructions>Can you update the JS SDK to support openai sdk's async iteration pattern in the js sdk instead of the current way of using callbacks?</agent_instructions>

Comments on the Issue (you are @copilot in this section)

Fixes JavaScript ChatClient Streaming Uses Callbacks, Not Async Iterators #507

💬 Send tasks to Copilot coding agent from Slack and Teams to turn conversations into code. Copilot posts an update in your thread when it's finished.

vercel · 2026-03-12T16:36:47Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
foundry-local	Ready	Preview, Comment	Mar 16, 2026 11:16pm

Co-authored-by: baijumeswani <12852605+baijumeswani@users.noreply.github.com>

prathikr

does audio client also need this change?

prathikr · 2026-03-16T22:08:49Z

sdk_v2/js/src/openai/chatClient.ts

+     * }
+     * ```
+     *
+     * Or with the callback pattern (for backward compatibility):


we don't need to call out "backward compatibility" since it appears we are going to support both methods moving forward

@copilot please address this feedback.

Done in b47f975 — removed all "backward-compatible" / "backward compatibility" language from the JSDoc overload descriptions in chatClient.ts and ChatClient.md.

prathikr · 2026-03-16T22:09:14Z

sdk_v2/js/src/openai/chatClient.ts

+     * @returns An async iterable that yields each chunk of the streaming response.
+     * @throws Error - If messages or tools are invalid, or streaming fails.
+     */
+    public completeStreamingChat(messages: any[]): AsyncIterable<any>;


does c# need this change as well? will also need to note this for impending rust/python SDKs

The C# SDK already returns IAsyncEnumerable<T> from CompleteChatStreamingAsync() (and CompleteAudioStreamingAsync()), which is C#'s equivalent of the async iterable pattern — so no changes needed there. Agreed that the Rust and Python SDKs should follow the same convention when they add streaming APIs.

…anguage Co-authored-by: baijumeswani <12852605+baijumeswani@users.noreply.github.com>

Copilot · 2026-03-16T23:15:36Z

does audio client also need this change?

Yes — applied the same async iterable pattern to AudioClient.transcribeStreaming() in b47f975. It now supports both:

// async iterable
for await (const chunk of audioClient.transcribeStreaming(filePath)) { ... }
// callback
await audioClient.transcribeStreaming(filePath, (chunk) => { ... });

Copilot

Pull request overview

Adds async-iterable support to the JS SDK streaming APIs so callers can use the familiar for await ... of pattern (matching the OpenAI SDK) while preserving the existing callback overloads.

Changes:

Added async-iterable overloads for ChatClient.completeStreamingChat() and AudioClient.transcribeStreaming(), backed by internal async generators bridging the native callback stream.
Refactored the prior callback implementations into _completeStreamingChatWithCallback() / _transcribeStreamingWithCallback().
Added/updated integration + validation tests, docs, and updated the JS sample app to demonstrate both patterns.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
sdk_v2/js/src/openai/chatClient.ts	Adds async-iterable overloads + `_streamChat()` generator and dispatch logic.
sdk_v2/js/src/openai/audioClient.ts	Adds async-iterable overload + `_transcribeStream()` generator and dispatch logic.
sdk_v2/js/test/openai/chatClient.test.ts	Adds async-iterable integration test; expands validation coverage.
sdk_v2/js/test/openai/audioClient.test.ts	Adds async-iterable integration test; expands validation coverage.
sdk_v2/js/docs/classes/ChatClient.md	Documents new overloads and usage examples.
sdk_v2/js/docs/classes/AudioClient.md	Documents new overload and usage examples.
samples/js/native-chat-completions/app.js	Updates sample to show async-iterable + callback usage.

Comments suppressed due to low confidence (1)

samples/js/native-chat-completions/app.js:62

The callback streaming example is extracting chunk.choices?.[0]?.message?.content, but the SDK’s streaming chunks (and the async-iterable example above, plus tests) use choices[0].delta.content. Using message.content here will likely print nothing. Update the callback sample to read from delta.content for consistency.

await chatClient.completeStreamingChat(
    [{ role: 'user', content: 'Write a short poem about nature.' }],
    (chunk) => {
        const content = chunk.choices?.[0]?.message?.content;
        if (content) {
            process.stdout.write(content);
        }

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

sdk_v2/js/test/openai/audioClient.test.ts

+        // Test with async iterable pattern
+        try {
+            const iter = audioClient.transcribeStreaming('');
+            await iter[Symbol.asyncIterator]().next();


sdk_v2/js/src/openai/chatClient.ts

        const tools = Array.isArray(toolsOrCallback) ? toolsOrCallback : undefined;
-        const callback = (Array.isArray(toolsOrCallback) ? maybeCallback : toolsOrCallback) as ((chunk: any) => void) | undefined;
+        const callback = typeof toolsOrCallback === 'function' ? toolsOrCallback : maybeCallback;

        this.validateMessages(messages);
        this.validateTools(tools);

-        if (!callback || typeof callback !== 'function') {
-            throw new Error('Callback must be a valid function.');
+        if (callback !== undefined) {
+            if (typeof callback !== 'function') {
+                throw new Error('Callback must be a valid function.');
+            }
+            return this._completeStreamingChatWithCallback(messages, tools, callback);
        }

+        return this._streamChat(messages, tools);


sdk_v2/js/src/openai/chatClient.ts

+        const chunks: any[] = [];
+        let streamDone = false;
+        let streamError: Error | null = null;
+        let notify: (() => void) | null = null;
+
+        const wakeConsumer = () => {
+            if (notify) { const n = notify; notify = null; n(); }
+        };
+
+        const streamPromise = this.coreInterop.executeCommandStreaming(
+            'chat_completions',
+            { Params: { OpenAICreateRequest: JSON.stringify(request) } },
+            (chunkStr: string) => {
+                // Skip processing if we already encountered an error
+                if (streamError) return;
+
+                if (chunkStr) {
+                    let chunk: any;
+                    try {
+                        chunk = JSON.parse(chunkStr);
+                    } catch (e) {
+                        // Don't throw from callback - store first error and stop processing
+                        streamError = new Error(
+                            `Failed to parse streaming chunk: ${e instanceof Error ? e.message : String(e)}`,
+                            { cause: e }
+                        );
+                        wakeConsumer();
+                        return;
+                    }
+                    chunks.push(chunk);
+                    wakeConsumer();
+                }
+            }
+        ).then(() => {
+            streamDone = true;
+            wakeConsumer();
+        }).catch((err: unknown) => {
+            streamError = err instanceof Error ? err : new Error(String(err));
+            streamDone = true;
+            wakeConsumer();
+        });
+
+        try {
+            while (!streamDone && !streamError) {
+                while (chunks.length > 0) {
+                    yield chunks.shift()!;
+                }
+                if (!streamDone && !streamError) {
+                    await new Promise<void>(resolve => { notify = resolve; });
+                }
+            }
+            // Drain any remaining chunks that arrived before streamDone was observed
+            while (chunks.length > 0) {
+                yield chunks.shift()!;
+            }


sdk_v2/js/src/openai/audioClient.ts

+        const chunks: any[] = [];
+        let streamDone = false;
+        let streamError: Error | null = null;
+        let notify: (() => void) | null = null;
+
+        const wakeConsumer = () => {
+            if (notify) { const n = notify; notify = null; n(); }
+        };
+
+        const streamPromise = this.coreInterop.executeCommandStreaming(
+            "audio_transcribe",
+            { Params: { OpenAICreateRequest: JSON.stringify(request) } },
+            (chunkStr: string) => {
+                // Skip processing if we already encountered an error
+                if (streamError) return;
+
+                if (chunkStr) {
+                    let chunk: any;
+                    try {
+                        chunk = JSON.parse(chunkStr);
+                    } catch (e) {
+                        // Don't throw from callback - store first error and stop processing
+                        streamError = new Error(
+                            `Failed to parse streaming chunk: ${e instanceof Error ? e.message : String(e)}`,
+                            { cause: e }
+                        );
+                        wakeConsumer();
+                        return;
+                    }
+                    chunks.push(chunk);
+                    wakeConsumer();
+                }
+            }
+        ).then(() => {
+            streamDone = true;
+            wakeConsumer();
+        }).catch((err: unknown) => {
+            streamError = err instanceof Error ? err : new Error(String(err));
+            streamDone = true;
+            wakeConsumer();
+        });
+
+        try {
+            while (!streamDone && !streamError) {
+                while (chunks.length > 0) {
+                    yield chunks.shift()!;
+                }
+                if (!streamDone && !streamError) {
+                    await new Promise<void>(resolve => { notify = resolve; });
+                }
+            }
+            // Drain any remaining chunks that arrived before streamDone was observed
+            while (chunks.length > 0) {
+                yield chunks.shift()!;
+            }


sdk_v2/js/test/openai/chatClient.test.ts

+                // Trigger iteration to materialize the error
+                const iter = client.completeStreamingChat(invalidMessage);
+                await iter[Symbol.asyncIterator]().next();


Initial plan

bf7f19a

Copilot AI assigned Copilot and baijumeswani Mar 12, 2026

Copilot started work on behalf of baijumeswani March 12, 2026 16:36 View session

vercel bot deployed to Preview March 12, 2026 16:37 View deployment

Add async iterable support to ChatClient.completeStreamingChat()

f00ef1d

Co-authored-by: baijumeswani <12852605+baijumeswani@users.noreply.github.com>

vercel bot deployed to Preview March 12, 2026 16:46 View deployment

Copilot AI changed the title ~~[WIP] Fix ChatClient streaming to use async iterators~~ Add async iterable support to ChatClient.completeStreamingChat() Mar 12, 2026

Copilot AI requested a review from baijumeswani March 12, 2026 16:47

Copilot finished work on behalf of baijumeswani March 12, 2026 16:47

baijumeswani closed this Mar 16, 2026

baijumeswani reopened this Mar 16, 2026

baijumeswani marked this pull request as ready for review March 16, 2026 22:04

prathikr reviewed Mar 16, 2026

View reviewed changes

Copilot started work on behalf of baijumeswani March 16, 2026 23:10 View session

Apply async iterable pattern to AudioClient; remove backward-compat l…

b47f975

…anguage Co-authored-by: baijumeswani <12852605+baijumeswani@users.noreply.github.com>

Copilot AI changed the title ~~Add async iterable support to ChatClient.completeStreamingChat()~~ Add async iterable support to ChatClient.completeStreamingChat() and AudioClient.transcribeStreaming() Mar 16, 2026

vercel bot deployed to Preview March 16, 2026 23:16 View deployment

Copilot finished work on behalf of baijumeswani March 16, 2026 23:16

baijumeswani requested a review from Copilot March 16, 2026 23:18

Copilot started reviewing on behalf of baijumeswani March 16, 2026 23:19 View session

Copilot AI reviewed Mar 16, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add async iterable support to `ChatClient.completeStreamingChat()` and `AudioClient.transcribeStreaming()`#521

Add async iterable support to `ChatClient.completeStreamingChat()` and `AudioClient.transcribeStreaming()`#521
Copilot wants to merge 3 commits intomainfrom
copilot/fix-chatclient-streaming-callbacks

Copilot AI commented Mar 12, 2026 •

edited

Loading

Uh oh!

vercel bot commented Mar 12, 2026 •

edited

Loading

Uh oh!

prathikr left a comment

Uh oh!

prathikr Mar 16, 2026

Uh oh!

baijumeswani Mar 16, 2026

Uh oh!

Copilot AI Mar 16, 2026

Uh oh!

prathikr Mar 16, 2026

Uh oh!

Copilot AI Mar 16, 2026

Uh oh!

Copilot AI commented Mar 16, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

Copilot AI commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

API changes

ChatClient.completeStreamingChat()

AudioClient.transcribeStreaming()

Implementation

Other

JavaScript ChatClient Streaming Uses Callbacks, Not Async Iterators

Comments on the Issue (you are @copilot in this section)

Uh oh!

vercel bot commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

prathikr left a comment

Choose a reason for hiding this comment

Uh oh!

prathikr Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

baijumeswani Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

prathikr Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI commented Mar 16, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Copilot AI commented Mar 12, 2026 •

edited

Loading

`ChatClient.completeStreamingChat()`

`AudioClient.transcribeStreaming()`

vercel bot commented Mar 12, 2026 •

edited

Loading