Streaming with tools

Stream a response while the model calls tools mid-stream, updating the UI as data arrives.

This recipe shows how to stream model output to the client and let the model call tools during the stream — for example, a search box that streams a draft answer and updates partial results as the model fetches additional context.

Primitives used

prompt() with tools
@crux/ai stream() returning a toUIMessageStreamResponse()
AI SDK useChat on the client
Optional: context() to share tools across multiple prompts

When to reach for this pattern

You're building a chat or assistant UI and need responsive streaming
The model needs to call tools mid-stream (search, fetch, compute) and the UI should reflect tool results
You want type-safe tool inputs and outputs — Zod schemas all the way through

Full code

`lib/ai/tools.ts`

import { tool } from '@crux/ai'
import { z } from 'zod'

export const searchDocs = tool({
  description: 'Search the documentation index for a query.',
  inputSchema: z.object({
    query: z.string().describe('The search query — what the user is asking about'),
    limit: z.number().int().min(1).max(10).default(5),
  }),
  execute: async ({ query, limit }) => {
    const results = await searchIndex(query, limit)
    return {
      results: results.map((r) => ({
        title: r.title,
        url: r.url,
        excerpt: r.excerpt,
      })),
    }
  },
})

export const fetchPage = tool({
  description: 'Fetch the full text of a documentation page by URL.',
  inputSchema: z.object({ url: z.string().url() }),
  execute: async ({ url }) => {
    const text = await fetchPageContent(url)
    return { text }
  },
})

`lib/ai/prompts.ts`

import { prompt, context } from '@crux/core'
import { z } from 'zod'

import { searchDocs, fetchPage } from './tools'

const docsTools = context({
  id: 'docs-tools',
  system:
    `You can search the docs and fetch full pages. Always start with searchDocs to find relevant pages.
     Use fetchPage when you need the full text of a single page to answer accurately.`,
  tools: () => ({ searchDocs, fetchPage }),
})

export const docsAssistant = prompt({
  id: 'docs-assistant',
  use: [docsTools],
  input: z.object({ message: z.string() }),
  system:
    `You answer questions about our product documentation.
     Be concise. Cite the page title in your answer when you used a search result.`,
  prompt: ({ input }) => input.message,
})

`app/api/chat/route.ts` (Next.js)

import { stream } from '@crux/ai'
import { openai } from '@ai-sdk/openai'

import { docsAssistant } from '@/lib/ai/prompts'

export async function POST(req: Request) {
  const { messages } = await req.json()

  const result = await stream(docsAssistant, {
    model: openai('gpt-4o'),
    input: { message: messages.at(-1)?.content ?? '' },
    messages,
    stopWhen: { stepCountIs: 5 },  // bound the tool-loop
  })

  return result.toUIMessageStreamResponse()
}

Client

'use client'

import { useChat } from 'ai/react'

export function DocsChat() {
  const { messages, input, handleInputChange, handleSubmit } = useChat({
    api: '/api/chat',
  })

  return (
    <div>
      {messages.map((m) => (
        <div key={m.id}>
          <b>{m.role}:</b>
          <div>{m.content}</div>

          {/* Show tool calls as they happen */}
          {m.toolInvocations?.map((t) => (
            <div key={t.toolCallId} className="text-sm text-gray-500">
              {t.toolName}({JSON.stringify(t.args)})
              {t.state === 'result' && (
                <pre>{JSON.stringify(t.result, null, 2)}</pre>
              )}
            </div>
          ))}
        </div>
      ))}

      <form onSubmit={handleSubmit}>
        <input value={input} onChange={handleInputChange} />
        <button type="submit">Send</button>
      </form>
    </div>
  )
}

How it works

Tools come from a context, not the prompt. Putting searchDocs and fetchPage on a context() means any prompt that uses docsTools inherits them. If you later add a docsSummarizer prompt that also needs these tools, you reuse the context — no copy-paste.
stream() returns a streaming result, not a Promise. The handler immediately returns the Response from toUIMessageStreamResponse(). Tool calls happen mid-stream and the AI SDK pipes their input/output through the response as UI message parts the client renders.
stopWhen: { stepCountIs: 5 } bounds the tool-loop. Without a stop condition, the model could enter an infinite tool-call loop. stepCountIs is the simplest bound. For more nuanced control, use hasToolCall(name) to stop after a specific tool call resolves.
The client renders tool invocations from message.toolInvocations. Each invocation has a state (partial-call, call, result) so you can show "calling search..." while the tool runs and the result when it lands.

Variations

Constrain which tools can run together

If fetchPage is expensive and you only want it called after a search succeeded, write a guardrail or surface that constraint in the system prompt. Tools themselves don't enforce ordering — that's the model's job, guided by your system text.

Stream a structured object instead

If you want a typed progressive result (not free-form chat), use streamObject from the AI SDK with a resolved prompt. See the structured extraction recipe for the pattern.

Persist conversation across reloads

Drop the chat-with-memory recipe alongside this one — episodic memory + sliding window keep the conversation alive across server restarts and reloads.

Where to next

Tools guide

Defining tools, lifecycle hooks, error handling.

Chat with memory

Add persistent memory to this chat loop.

Compositions

Combine multiple agents instead of a single tool-using agent.

Streaming with tools

Tools guide

Chat with memory

Compositions

On this page