GuidesEvals
Errors and remedies
Exact preflight and task-contract diagnostics for Evals.
Eval errors name the violated contract and the next action. You never need to know an evidence key, storage implementation, or Runtime execution degree to recover.
| Message | Remedy |
|---|---|
Managed Eval task descriptor is incompatible. This usually means @use-crux/core and @use-crux/ai are on mixed fixed versions; align both packages to the same compatible release. | Install compatible fixed Core and AI package versions together. |
file-backed Cases require a managed task with an input Standard Schema | Use inline Cases for an opaque function, or bind the production task with generate.task() / stream.task(). |
Cannot enforce --max-cost for Eval '<id>' because a conservative maximum is unavailable. Missing pricing keys: <keys>. | Add experimental.eval.pricing maxUsdPerCall ceilings for every listed model, or add default. |
Managed generate task returned no structured output; the adapter must return a validated object or throw its validation failure. | Keep the prompt output schema and let the adapter surface validation failure. |
model_identity_unattested | Wrap the AI SDK model with stableModel(model). For custom configuration, pass a secret-free versioned key. |
task_binding_untracked | Move generate.task() or stream.task() into a production module and import that task into the Eval with literal ESM. |
unresolved_source_dependency | Import the production task with literal ESM. Route ambient state through input, call options, or Variants, or run fresh. |
nondeterministic_renderer | The same Case rendered a different prompt. Move environment, time, randomness, filesystem, or network state into input, call options, or a Variant; otherwise run with --fresh. |
Set CRUX_EVAL_HOST_URL. | Select the remote Runtime URL, unless the active adapter can infer it. |
Set CRUX_EVAL_HOST_DEPLOYMENT_ID. | Set the expected deployment id, unless the active adapter can infer it. |
Set CRUX_EVAL_HOST_TOKEN. | Add the dedicated bearer to the environment or secret store. |
Only a complete Eval run can be set as a Baseline | Rerun every selected Case and required score without errors. |
A filtered Eval run cannot be set as a Baseline | Run the full Eval population before accepting it. |
feedback() requires a configured durable feedback destination. Configure the Crux observability destination before submitting feedback. | Configure the existing observability destination with durable feedback support. |
feedback() requires a message with valid metadata.crux.runId from a Crux stream result. | Pass the AI message produced by the Crux stream integration. |
EVAL_RESULT_REDACTION_REQUIRED | Return a safe task projection or remove sensitive fields before the remote result persistence boundary. |
Project privacy policy is not ready for strict offline Evals. Run `crux runtime generate`, then retry. | Regenerate the project Runtime artifacts before using --offline. |
The selected Runtime was generated with a different observability.redactPaths policy. | Run crux runtime generate, redeploy the selected Runtime, then retry. |
Provider, abort, timeout, adapter, and validation-exhaustion errors pass through unchanged. Fix them at their originating provider or adapter layer.