Support Partial Tool Results / AsyncGenerator for Tool Calls in streamText

[BrianHung]

Feature Description

For tool calls which take a couple of seconds to finish, it would be useful if partial tool results were supported so that the UI can reflect incremental progress.

For server side tools, this could be implemented by allowing execute to be an AsyncGenerator. This is similar to how the generate function in streamUI allows for intermediate updates to the UI:

ai/packages/ai/README.md

Lines 136 to 141 in 8653fe5

	generate: async function* ({ repositoryName }) {
	yield <div>Cloning repository {repositoryName}...</div>;
	await new Promise(resolve => setTimeout(resolve, 3000));
	yield <div>Building repository {repositoryName}...</div>;
	await new Promise(resolve => setTimeout(resolve, 2000));
	return <div>{repositoryName} deployed!</div>;

For client side tools, it could be done with an addPartialToolResult.

import { stream } from "fetch-event-stream"

onToolCall: async ({ toolCall }) => {

  if (toolCall.name == "searchWeb") {
	const events = await stream('http://websearch.tool?q=' + encodeURIComponent(toolCall.args.query), {
		method: 'POST',
		body: JSON.stringify({
			stream: true,
		}),
	});
	for await (const event of events) {
		addPartialToolResult({data: JSON.parse(event.data) })
	}
  }

}

In both cases, we could add a new stream part toolResultDeltaStreamPart that handles this. Similar to how partial tool calls toolCallDeltaStreamPart are supported when arguments are being generated:

ai/packages/ui-utils/src/stream-parts.ts

Line 334 in 8653fe5

const toolCallDeltaStreamPart: StreamPart<

Use Case

Take for example a web search tool. The flow would be something like

{"message":"searching web"}
{"message":"links found!"}
{"message":"crawling links..."}
{"message":"results"}

Huggingface has a good reference implementation here of that behavior, where it emits intermediate updates to its UI.

Additional context

Tools implemented with edge functions typically time out if no response is given within a fixed limit like 10s or 30s.

A mix of returning an event-stream with waitUntil gets around that limitation.

      return new Response(readable, {
        headers: {
          "Content-Type": "text/event-stream",
          "Transfer-Encoding": "chunked",
          "content-encoding": "identity",
        },
      });