@use-crux/ai

AI SDK ModelMessage and useChat media remain model-owned. The adapter preserves image/audio/video/file input and ordered mixed assistant output, and exports native SDK generateImage(), transcribe(), and generateSpeech() operations. Convex re-exports those functions by exact identity. Provider options stay in typed extra, and no operation persists media implicitly. Completed image and speech calls accept guardrails/safety; transcription also accepts one-shot output-text constraints. Canonical decisions appear in result.safety, while AI SDK-native raw results remain unguarded. See the content-primitive matrix.

import {
  generate,
  stream,
  createUIMessageStreamResponse,
  pipeUIMessageStreamToResponse,
  embedding,
  retrievalModel,
  reranker,
  aiSdk,
  aiSdkProviderRuntime,
} from "@use-crux/ai";
import {
  generateObjectFn,
  generateTextFn,
  createCruxAi,
  CruxAIError,
} from "@use-crux/ai";
import { resolve } from "@use-crux/ai/agent";

Request capacity and counting

The adapter reports a ModelCapacityProfile before each planned language request. Use adapter.capacity(model) for the bound model object, or call the exported aiSdkModelCapacity({ provider, modelId }) when you already have its normalized identity.

import { aiSdkModelCapacity } from "@use-crux/ai";

const profile = aiSdkModelCapacity({
  provider: "openai",
  modelId: "gpt-4o-mini",
});

Known profiles use model-id prefix matching. They cover OpenAI GPT 3.5 through 5 and reasoning families o1, o3, and o4; Anthropic generations 2 through 4; and Google Gemini 1.5 through 3. Versioned model aliases inherit their family profile.

Unknown models use Core's conservative fallback: an 8,192-token context window with a 2,048-token default output reserve. Known profiles report countingConfidence: "estimated"; the fallback reports "conservative".

This adapter does not make a provider token-counting request. Crux measures the complete request with its provider-aware estimator and applies the confidence margin shown in receipt inspection. Set inputBudget.max below the derived limit when your product needs a smaller strict boundary. See Input budgets for selection behavior.

Architecture

@use-crux/ai exports aiSdkProviderRuntime, a normal Crux provider runtime with ownership: 'loop-owned' for the Vercel AI SDK. Core owns all policy (prompt resolution, router()/split()/retry()/fallback()/cascade() routing, validation retry, the Safety session for constraints/guardrails, the tool-approval protocol, instrumentation, timeouts); the adapter delegates mechanics to AI SDK natives (stopWhen, prepareStep, experimental_repairText, SDK-owned tool approval hooks, abortSignal, experimental_transform for streaming guardrails). The bound runtime also exposes AI SDK embedding through the same SdkGateway seam.

The only module that calls AI SDK functions is the SdkGateway, inject a scripted one via createCruxAi to test without module mocks.

Internally, @use-crux/ai keeps AI SDK request planning and result projection in a private call-plan codec. The loop runtime asks the codec for a generateText, generateObject, streamText, streamObject, or replay plan, invokes the selected SdkGateway method, then decodes or attaches the raw SDK result. This is not a public extension API; it exists to keep the gateway seam stable and the adapter behavior testable.

For Anthropic AI SDK models, Crux converts resolved systemBlocks into AI SDK system messages and places providerOptions.anthropic.cacheControl on the single cacheBoundary block at the end of the stable cached prefix.

aiSdk(native)

Bind a native AI SDK language model or same-adapter route tree to a frozen GenerationModel for Agents, durable Agent Sessions, Runtime programs, and Eval identity:

import { aiSdk } from "@use-crux/ai";
import { agent } from "@use-crux/core/agent";

const supportModel = aiSdk(nativeModel("nebula-text-v2"));
const support = agent({
  id: "support",
  model: supportModel,
  prompt: supportPrompt,
});

One argument only. The adapter owns capability derivation from native identity and catalog evidence. The return value is a frozen AdapterBoundGenerationModel with secret-free definition.id / definition.fingerprint, complete language capability evidence, and an opaque runtime port that constructs an AgentExecutor through aiSdkProviderRuntime. Application code does not pass capabilities or install a global executor.

Same-adapter routers may be bound once. Do not nest already-bound GenerationModel leaves inside aiSdk(); compose bound leaves with Core routers when needed. Declare every Session-selected model on createRuntimeProgram({ generationModels }) or the generated program.

Bare native model objects remain valid for ordinary generation, but durable Agent Sessions require a bound GenerationModel. Eval task identity projects bound models from their portable definition; unbound objects remain model_identity_unattested because constructor names and ambient provider metadata are not secret-free durable identity.

Multimodal content

@use-crux/ai accepts canonical ContentPart[] message content and encodes it to AI SDK text, image, and file parts. It also accepts native AI SDK ModelMessage[] from convertToModelMessages() and passes native file/image parts plus provider options through to the AI SDK loop without a Crux media conversion step. SDK control parts such as tool calls and approval responses move into Message.metadata only for the canonical result history.

import { convertToModelMessages } from "ai";
import { createUIMessageStreamResponse, stream } from "@use-crux/ai";

export async function POST(req: Request) {
  const { messages } = await req.json();
  const result = await stream(chatPrompt, {
    model,
    input: { tenantId: "acme" },
    messages: await convertToModelMessages(messages),
  });

  return createUIMessageStreamResponse(result);
}

Cross-adapter parity: switching a prompt between @use-crux/ai and a native adapter (@use-crux/openai, @use-crux/anthropic, @use-crux/google) changes nothing observable except model behavior itself, same default tool-loop budget (maxSteps: 10), same corrective-retry messages, same approval suspend/resume protocol, same canonical tool.call / tool.args / raw and model-facing tool.result telemetry, same routing and audit metadata. This is enforced by a cross-dialect parity suite in @use-crux/core. Note this differs from the raw AI SDK's own generateText default (stopWhen: stepCountIs(1)): Crux adapters loop tools by default; pass maxSteps: 1 for single-step behavior.

Hallucinated tool calls are part of that parity: where the raw AI SDK throws NoSuchToolError/InvalidToolInputError and aborts the loop, @use-crux/ai uses the SDK's native experimental_repairToolCall to feed the model the same error tool result core-driven adapters produce ({"error":"Tool \"x\" not found"}), so the model self-corrects and the loop continues. The SDK's tool-error taxonomy never reaches your code.

Portable and Node transcription

The @use-crux/ai root is portable and performs no hidden audio download. transcribe() accepts bytes, ArrayBuffer, Blob, data URLs, and data assets. The AI SDK adapter currently rejects provider-file assets before gateway I/O; hydrate them to bytes first. If an HTTPS source requires Crux to materialize it, the portable operation throws UnsupportedCapabilityError before gateway I/O. Provide bytes, or use the explicit Node entrypoint:

import {
  transcribe,
  createAiSdkTranscribe,
} from "@use-crux/ai/transcription/node";

const result = await transcribe({ model, audio: remoteAudioUrl });
const custom = createAiSdkTranscribe(scriptedGateway);

The Node entrypoint uses the bounded, DNS-pinned Crux downloader. It exports only those two runtime values plus the existing transcription types. The portable root keeps custom gateway binding on createCruxAi({ gateway }) and does not export createAiSdkTranscribe.

@use-crux/ai/agent

Use @use-crux/ai/agent for AI SDK-compatible agent frameworks that own their own model loop, such as Convex Agent or Mastra. It composes instructions through the normal core prompt pipeline, then wraps the model with AI SDK middleware when Crux execution hooks are installed.

import { resolve } from "@use-crux/ai/agent";

const { instructions, model } = await resolve(chatPrompt, {
  model: languageModel,
  input: { mode: "support" },
  tools: Object.keys(tools),
});

The returned model reports generate/stream traces, stream progress, tool timing estimates, provider metadata cost, and parent-child links back to the prompt-resolution trace.

generate(prompt, options)

Execute a prompt using the Vercel AI SDK.

Returns: GenerateResult<TRaw, TOutput>, accumulated .text, optional complete .usage, optional .cost, .steps, .finalStep, provider-neutral .messages, retained ._meta, and typed .raw for the AI SDK result. With an output schema, .object is typed from the prompt schema.

stream(prompt, options)

Stream a prompt execution. Same options as generate(), except cascade() is not supported (cascade needs full results for tier evaluation). Streaming guardrails execute automatically on text streams through the Safety stream transform, so sentence-gated holds, rewrites, and blocks reach textStream consumers. An assert constraint gates release transactionally: the stream withholds output while it is unresolved, and a failure discards the attempt and re-streams with corrective feedback. suggest constraints run report-only and their audits land on the completion meta.

Returns: StreamResult<TOutput, TPartial>, one managed logical stream with .textStream, .fullStream, .partialOutputStream, .completion, and .cancel(). There is no .raw: the AI SDK result resolves before terminal Safety and describes only one attempt, so exposing it would bypass the gates this contract enforces.

const result = await stream(editDraft, {
  model: openai("gpt-4o"),
  input: { instruction: "Fix the intro" },
});
// Canonical partial `z.input`: manifest-decoded, never provider wire JSON,
// and never published from an attempt Safety went on to discard.
for await (const partial of result.partialOutputStream) {
  console.log(partial);
}

Stream results additionally carry a typed completion promise:

const result = await stream(chatPrompt, { model, input: { message } });
for await (const delta of result.textStream) process.stdout.write(delta);
const completion = await result.completion;
// { text, usage?, cost?, steps, finalStep, messages }

For AI SDK UI routes, pass the canonical result to the helper:

const result = await stream(chatPrompt, { model, input: { message } });
return createUIMessageStreamResponse(result);

Use pipeUIMessageStreamToResponse(result, options) for Node ServerResponse targets.

toParams(resolved, options) / fromResponse(response)

toParams() converts a ResolvedPrompt plus an AI SDK { model, settings?, extra? } object into generateText() args. fromResponse() normalizes an AI SDK generate result into Crux AdapterResponse facts.

These codecs are translation-only. Use managed generate()/stream() when Crux should run tools, approvals, validation retry, memory capture, safety, and observability.

prepare(prompt, options)

Prepare a headless AI SDK call. Crux resolves the prompt and returns the planned SdkGateway params without invoking the AI SDK. Feed the raw SDK result back with finish(response).

generate(prompt, { ...options, transport }) is the BYO-wire mode where Crux keeps owning the loop and invokes your callback for each SdkGateway request. AI SDK structured validation retry still happens when the transport throws the same validation/parse errors the SDK would throw. stream() with transport is intentionally unsupported and rejects with CruxTransportStreamUnsupportedError.

createCruxAi(options?)

Build a @use-crux/ai instance bound to a specific SdkGateway. The package-level exports are an instance created with the live gateway; reach for this when you need the test seam.

Returns: CruxAi, { generate, stream, prepare, generateTextFn, generateObjectFn, embedding, retrievalModel, reranker }

import { createCruxAi } from "@use-crux/ai";

const ai = createCruxAi({ gateway: myScriptedGateway });
const result = await ai.generate(myPrompt, { model, input });

Built-in lowering is inferred only from direct OpenAI, Anthropic, Google, and Vertex provider identities. Aggregators such as OpenRouter and unknown models receive the canonical schema unchanged by default; Crux still validates the returned value with the authored schema. The same policy applies to generate(), stream(), schema-bearing tools, and this instance's generateObjectFn.

createAiSdkLoopRuntime(gateway)

The low-level adapter from a SdkGateway to core's LoopRuntimePort, the gateway-closed runtime that aiSdkProviderRuntime binds and createCruxAi drives. Reach for it only when you build a bespoke runtime compiler or run the loopRuntimePortConformance() suite directly; most callers use createCruxAi or aiSdkProviderRuntime.

Returns: AiSdkLoopRuntime, a LoopRuntimePort<LanguageModel> with runTextLoop/runStructuredAttempt/runStream/replayStream.

CruxAIError

Coded error wrapper. generate()/stream() propagate underlying errors unchanged (existing ValidationExhaustedError/AggregateError handling keeps working); use CruxAIError.classify(error) at your boundary for a stable, machine-readable code.

generateObjectFn

Pre-bound GenerateObjectFn for @use-crux/core APIs (judges, extraction). Pass model per-call.

Signature: GenerateObjectFn, (opts: { model, system?, prompt, schema }) => Promise<{ object: T }>

generateObjectFn shares the same AI SDK structured-attempt mechanics used by generate() for prompts with an output schema: provider-specific schema sanitation, core-backed experimental_repairText, and router/cascade model resolution all happen before the helper returns { object }.

The returned object always passes the authored schema. An authored-schema failure throws ValidationExhaustedError without exposing the rejected candidate. Because that error is classified as invalid_response, fallback([...], { on: ["invalid_response"] }) can try the next configured model.

import { judge } from "@use-crux/core/scoring";

const evaluator = judge({
  id: "helpfulness",
  criteria: "Does the answer resolve the request?",
  scale: { min: 1, max: 5 },
  generate: generateObjectFn,
  model,
});

generateTextFn

Pre-bound GenerateTextFn for provider-neutral @use-crux/core APIs.

Signature: GenerateTextFn, (opts: { model, system?, prompt }) => Promise<{ text: string }>

embedding(config)

Create a dense Crux embedding backed by AI SDK embedMany().

const docsEmbedding = embedding({
  name: "docs-embedding",
  model: openai.textEmbeddingModel("text-embedding-3-small"),
  dimensions: 1536,
  maxInputTokens: 8192,
});

Use it with memory, indexers, or retrievers when you want the AI SDK model registry and provider abstraction, but still want a first-class Crux embedding object with batching and telemetry.

The installed AI SDK Embedding Model V3 contract accepts Array<string>, so this helper declares modalities: ['text']. It does not caption or otherwise emulate native media embeddings.

For object models, the helper identity uses { provider, modelId }; string models use the raw string id. The user version is appended rather than replacing that identity. Headers and providerOptions are deliberately excluded because they may be sensitive or non-canonical. Bump version when an option changes vector semantics.

retrievalModel(config)

Create a RetrievalModel for model-backed recipe steps such as rerank() and compressToBudget().

import { retrievalModel } from "@use-crux/ai";

const model = retrievalModel({ model: openai("gpt-4o-mini") });

reranker(config)

Create a core Reranker backed by the AI SDK native rerank() endpoint.

import { cohere } from "@ai-sdk/cohere";
import { reranker } from "@use-crux/ai";

const engine = reranker({
  model: cohere.reranking("rerank-v3.5"),
  topN: 12,
});

Use this with rerank({ engine }) in retrieval recipes. Native reranking models are dedicated ranking calls; they do not spend generation tokens.

Tool approvals

When toolApproval requires approval for a tool call, generation suspends: the result has _meta.finishReason === 'tool_approval_required', pendingApprovals carries the minted requests (with anti-forgery tokens), and messages ends with the approval-request message. Persist the messages, collect a decision, append a tool-approval-response, and call generate() again.

import { appendToolApprovalResponse } from "@use-crux/core/tool-middleware";

const first = await generate(assistant, {
  model,
  input,
  tools,
  toolApproval: { deletePost: "always" },
});

if (first.pendingApprovals?.length) {
  const approval = first.pendingApprovals[0];
  const resumed = await generate(assistant, {
    model,
    input,
    tools,
    toolApproval: { deletePost: "always" },
    messages: appendToolApprovalResponse(first.messages, {
      approvalId: approval.approvalId,
      approved: true,
      approvalToken: approval.approvalToken,
    }),
  });
}

The approved tool executes during resume, its result is replayed into the loop as a normal tool round, and generation continues. Denied tools produce an execution-denied output the model can reason about.

What is not exported

AI SDK helpers and AI SDK types are no longer re-exported, import provider-native helpers from 'ai' directly when you need them under extra. Portable loop helpers (maxSteps, hasToolCall) come from @use-crux/core. Agent compositions come from @use-crux/core/agent or from loopRuntimeAdapter(). The legacy toMessages/fromMessages/createAIExecutor exports were removed (RFC use-crux/crux#28).

Types

import type {
  AIGenerateOptions,
  AIEmbeddingConfig,
  AIRerankerConfig,
  AiSdkLoopRuntime,
  CruxAi,
  CruxAiOptions,
  CruxAIErrorCode,
  SdkGateway,
  AIStreamCallbacks,
  GenerateReturn,
  StreamReturn,
} from "@use-crux/ai";

Related

• Guide: Execution
• Reference: Retrieval
• Reference: Prompts
• Cookbook: Streaming with tools