Routing & Fallback

Routing and fallback are model-level policies. They do not change what a prompt says; they decide which model reference reaches the adapter and what should happen when that model is not the right one.

npm install @crux/core
@crux/ai

The Mental Model

A Crux call has three separate concerns:

Keep those concerns separate. Prompt authors should not have to scatter provider if statements through every call site, and runtime code should not have to know every quality tier in your product.

await generate(supportPrompt, {
  model: supportModelPolicy,
  input: {
    accountTier: 'enterprise',
    question: 'Can you summarize this invoice dispute?',
  },
})

Which Primitive Should I Use?

The most common production setup combines them:

const supportModelPolicy = router({
  id: 'support-model-policy',
  classify: classifySupportRequest,
  routes: {
    fast: fallback(gpt4oMini, claudeHaiku),
    careful: cascade({
      id: 'careful-answer-cascade',
      tiers: [
        { model: fallback(gpt4oMini, claudeHaiku), evaluate: quickQualityCheck },
        { model: fallback(claudeSonnet, gpt4o), evaluate: strongQualityCheck },
        { model: claudeOpus },
      ],
    }),
    default: fallback(claudeSonnet, gpt4o),
  },
})

How They Compose

All three primitives wrap models:

• A router route can point at a raw model, fallback, cascade, or another routed policy.
• A cascade tier can use a raw model or fallback.
• A fallback option can be a raw model or another model wrapper.

Keep composition boring and named. If a routing policy is product behavior, give it a stable id so devtools, traces, and the project index can connect runtime behavior back to source.

Observability

Routing decisions are written into result metadata and Crux observability records.

const result = await generate(supportPrompt, {
  model: supportModelPolicy,
  input,
})

console.log(result._meta.router)
console.log(result._meta.cascade)
console.log(result._meta.fallback)

The project index records authored routing definitions:

Index lints warn when a policy has no stable id, a router omits default, a target cannot be resolved, or a cascade tier accepts before later tiers can run.

Pick A Topic