← All posts
EngineeringJun 9, 2026 · 9 min read

Adding memory to the Vercel AI SDK

The AI SDK deliberately leaves memory open. How to add typed, budgeted, observable memory to an AI SDK app: recent messages, working state, and long-term facts, without switching frameworks.

HSHenri van 't Santmemoryai-sdktutorial

The Vercel AI SDK is very good at what it owns: calling models, streaming, tool loops, structured output. Memory is deliberately not on that list. The SDK orchestrates the conversation; what persists across conversations is left to you.

That is the right call, because "memory" is not one feature. The last twelve messages, the current task state, and what you learned about the user last month are different kinds of state with different lifetimes and write policies. Flatten them into one vector store and you get an assistant that remembers a user's dog's name but forgets what it was doing.

This post adds each kind to an AI SDK app using Crux memory blocks. Crux composes over the AI SDK. The AI SDK still makes every model call.

Setup

pnpm add @use-crux/core @use-crux/ai ai @ai-sdk/openai zod

Crux's generate() wraps AI SDK generation with prompt resolution around it. An existing generateText call site maps over mechanically: the model stays, the messages become a typed prompt.

Remembering the conversation

The minimum viable memory is short-term continuity. One composed memory, one block:

import { memory, recentMessages } from '@use-crux/core/memory'
import { prompt } from '@use-crux/core'
import { generate } from '@use-crux/ai'
import { openai } from '@ai-sdk/openai'
import { z } from 'zod'

const chat = memory({
  id: 'assistant',
  records,                                          // your record store
  namespace: ({ input }) => `user:${input.userId}`, // isolation per user
  blocks: [recentMessages({ id: 'recent', maxMessages: 12 })],
})

const assistant = prompt({
  id: 'assistant',
  use: [chat],
  input: z.object({ userId: z.string(), message: z.string() }),
  system: 'Use memory when relevant. Do not invent preferences.',
  prompt: ({ input }) => input.message,
})

const result = await generate(assistant, {
  model: openai('gpt-4o'),
  input: { userId, message },
})

Each call renders the block into prompt context before generation. After the model finishes, the adapter submits one completed turn to memory. Streaming uses the same lifecycle after tokens have already streamed. The default deferred mode uses the configured host binding to retain capture; without one, Crux waits inline so memory is never silently lost. Explicit inline capture can delay completion, but it never blocks token delivery.

Note the namespace function. Cross-user memory leakage is the classic memory bug. Making isolation a declared property of the composition, rather than a WHERE clause you remember to add, removes that failure mode structurally.

Remembering the task

recentMessages degrades the way transcripts degrade: important details scroll out of the window. For "what are we doing right now," use one typed value the app overwrites as the task progresses:

import { workingState } from '@use-crux/core/memory'

workingState({
  id: 'state',
  priority: 90,
  schema: z.object({
    goal: z.string().optional(),
    openQuestions: z.array(z.string()).default([]),
  }),
})

The schema is enforced with Zod. A malformed write fails at the write, not as silent prompt garbage three turns later.

Remembering the user

Cross-session knowledge ("prefers concise answers", "team ships Tuesdays") is extracted facts, not transcript. The facts block extracts candidates from each turn, embeds them, and recalls them semantically against the current message:

import { facts } from '@use-crux/core/memory'

facts({
  id: 'about-user',
  embed: dense,                    // your embedding function
  extract: extractFactsFromTurn,
  write: { mode: 'propose' },
  render: {
    strategy: 'semantic',
    query: ({ input }) => String(input.message),
    limit: 5,
  },
})

Two settings here are worth copying whatever library you use.

write: { mode: 'propose' } means extracted facts do not become active memory until approved. LLM-extracted facts are wrong often enough that write-through extraction slowly poisons the well. With a propose step, a misheard fact is a rejected proposal instead of a permanent belief.

limit: 5 bounds recall. Unbounded memory recall is how prompts bloat until something else important gets dropped.

Events and habits

Two more block types round out the model, and knowing when to reach for them saves you from misusing facts for everything.

episodes() is an append-only event log of things that happened: conversations, tool results, decisions, user actions. With an embedding it recalls related events semantically. Use retention to declare the real eviction window instead of letting the log grow forever:

import { episodes, procedures } from '@use-crux/core/memory'

episodes({
  id: 'history',
  embed: dense,
  retention: '90d',
  render: { strategy: 'semantic', query: ({ input }) => String(input.message), limit: 4 },
})

procedures() is operating memory: learned instructions that should shape future behavior, like "prefer direct answers before examples". It renders as guidance, not as recalled events, and pairs naturally with write: { mode: 'propose' } so the assistant cannot teach itself a bad habit without review.

The rule of thumb: facts are conclusions, episodes are events, procedures are behavior. When a memory could be two of these, pick the one whose lifecycle matches. Conclusions get corrected, events expire, behavior gets reviewed.

What about token budgets?

Memory competes with everything else in the prompt for finite space. Most memory tutorials skip this. It is where the production bugs live.

Crux makes the arbitration explicit. Blocks have priorities, and the composed memory has a token budget:

const chat = memory({
  id: 'assistant',
  records,
  namespace: ({ input }) => `user:${input.userId}`,
  capture: { mode: 'deferred' },
  budget: { maxTokens: 1200 },
  blocks: [
    workingState({ id: 'state', priority: 90, schema }),
    recentMessages({ id: 'recent', maxMessages: 12, priority: 80 }),
    facts({ id: 'about-user', embed, write: { mode: 'propose' }, render, priority: 60 }),
  ],
})

Under pressure, working state survives, then recent messages, then facts. A declared policy, not an accident of concatenation order.

capture.mode defaults to deferred. Configure retention once through config({ host }); when the active environment cannot retain work, Crux captures inline and emits one development warning. Use inline when you need read-your-writes semantics. After retained generation returns, flush() waits for previously accepted capture and reports its first unobserved failure.

How do I debug it?

Because memory is composed blocks rather than string glue, it is inspectable:

assistant.inspect({ input })  // what each block rendered, kept, or trimmed

At runtime, every memory read and write shows up in the devtools trace per turn: which entries were recalled, what was extracted, what was proposed versus written. "Why does it think my name is Dave" becomes a lookup instead of an afternoon of console.log.

Common pitfalls

A few failure modes we see repeatedly, all avoidable up front:

  • One giant block. If everything lives in facts, extraction quality becomes your single point of failure. Split by lifecycle first; it also makes budgets meaningful.
  • Write-through extraction. Skipping propose feels faster until the first hallucinated "fact" persists. Turn it off only for state the app itself writes.
  • Unbounded recall. Put a limit on every semantic render. Without one, memory wins the budget war against your retrieval and instructions.
  • Namespace by conversation when you meant user. thread: namespaces forget everything on a new chat; user: namespaces remember across chats. Pick deliberately. Mixing them up is invisible until a user notices.
  • No eviction story. retention on episodes plus a sweep job. An append-only log with semantic recall degrades slowly and mysteriously as junk accumulates.

Where this fits

There are several good memory options around the AI SDK: Mem0, MongoDB's memory provider, Redis-backed history. Most give you a hosted or database-specific memory service, and if all you want is transcript persistence, a Redis recipe is simpler.

Crux memory is a different shape. It is part of a typed harness layer, so the same composition mechanism also carries retrieval, guardrails, and routing, and one trace explains all of them. Reach for it when memory has to coexist with other context under one budget, because that is when the "which part of my prompt ate the other part" bugs start.

Crux is open source and alpha. The full memory guide (episodes, procedures, custom blocks, stores, eviction) is at cruxjs.dev/docs/guides/memory.

KEEP READING

safety ·guardrails ·security ·
fn safety()
EngineeringJul 13, 2026

Guardrails that compose: practical LLM safety boundaries in TypeScript

Prompt injection can't be solved with a wrapper function and a regex. Model safety as typed boundaries (input, output, tool calls, memory writes) with declared actions, streaming semantics, and evidence for every decision.

HS9 min
routing ·cost ·quality ·
fn routing()
EngineeringJul 8, 2026

Cut your AI bill with a routing policy you can prove

Vendors promise 40–85% savings from model routing. The claims are plausible, and unverifiable without two things most routing setups lack: a declared policy and a receipt per request.

HS9 min
retrieval ·rag ·freshness ·
fn retrieval()
EngineeringJul 3, 2026

The RAG freshness problem is a harness problem

Vector similarity has no concept of time: a stale chunk at 0.92 outranks the current one at 0.87, every time. Why freshness can't be fixed inside the index, and what it looks like as a first-class, testable property of the turn.

HS8 min