Middleware & Hooks

Middleware

Middleware wraps every generate() and stream() call across all prompts. Use it for logging, timing, error handling, or custom retry logic.

config({
  generation: {
    middleware: async (args, next) => {
      const start = Date.now()
      try {
        const result = await next(args)
        console.log(`${args.promptId}: ${Date.now() - start}ms`)
        return result
      } catch (error) {
        console.error(`${args.promptId} failed after ${Date.now() - start}ms`)
        throw error
      }
    },
  },
})

Middleware receives { promptId, preparedArgs } and a next function. It can inspect or modify args, measure timing, transform results, or handle errors.

Standalone setup (when not using config()):

import { updateRuntime } from '@crux/core'
updateRuntime({ middleware: async (args, next) => { ... } })

Lifecycle hooks

Per-prompt hooks give you fine-grained observability on individual prompts:

prompt({
  hooks: {
    onPrepare: (args) => {
      console.log('System tokens:', args.systemTokens)
      if (args.droppedContexts.length > 0) {
        console.warn(
          'Dropped:',
          args.droppedContexts.map((c) => c.source),
        )
      }
    },
    onGenerate: (args) => {
      console.log(`${args.promptId}: ${args.durationMs}ms`)
    },
    onError: (args) => {
      reportError(args.promptId, args.error)
    },
  },
})

Hook argument types

Each lifecycle hook receives a typed argument object. Here are the full shapes:

PrepareHookArgs

Passed to onPrepare after the prompt's system message and contexts are composed.

interface PrepareHookArgs {
  promptId: string // ID of the prompt being prepared
  systemTokens: number // total tokens in composed system message
  droppedContexts: DroppedContext[] // contexts dropped due to token budget
}

droppedContexts includes the source name and tokens count for each context that was trimmed. Use this to detect when important context is being lost.

GenerateHookArgs

Passed to onGenerate after a successful generation completes.

interface GenerateHookArgs {
  promptId: string // ID of the prompt that generated
  durationMs: number // generation wall-clock time in milliseconds
  result: unknown // the generation result (adapter-specific)
}

The result type depends on your adapter — for Vercel AI SDK it will be the generateText or generateObject result, for OpenAI it will be the chat completion response.

ErrorHookArgs

Passed to onError when an error occurs during preparation or generation.

interface ErrorHookArgs {
  promptId: string // ID of the prompt that errored
  error: unknown // the error that occurred
  phase: 'prepare' | 'generate' // which phase the error happened in
}

The phase field tells you whether the error occurred during context composition (prepare) or during the LLM call (generate), so you can handle each case differently.

Instrumentation hooks

For deeper observability into memory, scoring, and agent operations, Crux provides InstrumentationHooks. These are wired up when you explicitly enable a local devtools server/tunnel via config({ devtools: { serverUrl } }) — you don't need to configure them manually.

The instrumentation hooks cover three areas:

Memory hooks

• onMemoryRead — fired when a memory store is queried (includes the query and number of results)
• onMemoryWrite — fired when new entries are written to memory
• onCompactStart / onCompactEnd — fired around memory compaction operations, useful for tracking compaction duration and token savings

Budget and context hooks

• onBudgetCheck — fired during token budget evaluation, reports which contexts fit and which are candidates for dropping
• onBlackboardUpdate — fired when a shared blackboard (cross-prompt state) is updated

Agent hooks

• onHandoffPrepare — fired when an agent prepares to hand off to another agent or sub-prompt
• onDelegateStart / onDelegateComplete — fired around agent delegation calls, tracking which sub-agent was invoked and its result
• onJudgeResult — fired when an LLM judge returns a score, includes the metric ID, score, and reasoning

Production logging

Middleware gives you structured data about every generate() call. Use it to forward traces to your observability stack:

config({
  generation: {
    middleware: async (args, next) => {
      const start = Date.now()
      try {
        const result = await next(args)
        myLogger.send({
          type: 'llm.generate',
          promptId: args.promptId,
          model: result._meta?.model,
          durationMs: Date.now() - start,
          tokens: result._meta?.usage,
          cost: result._meta?.cost,
          status: 'success',
        })
        return result
      } catch (error) {
        myLogger.send({
          type: 'llm.generate',
          promptId: args.promptId,
          durationMs: Date.now() - start,
          status: 'error',
          error: error.message,
        })
        throw error
      }
    },
  },
})

This works with any logging service — Datadog, Sentry, Posthog, or a simple console.log. The _meta object on results contains normalized model, usage, and cost data regardless of which adapter you used.

Next steps