Crux
GuidesObservability

Runs and delivery health

What incomplete, suspended, and degraded mean in the Runs list, and what to do about each.

You opened Runs and a row says incomplete. Or a run has been suspended for an hour. Or the delivery chip is stuck on unknown. None of these are UI bugs: they are truthful statements about what telemetry the local server actually observed. Here is what each status means and what to do.

Run status

Status describes the run's lifecycle. A logical run can span multiple physical segments (process, isolate, or invocation) when it suspends and resumes.

StatusKindMeaning
runningLiveThe only live state. Work is in progress in at least one open segment.
suspendedDurable pauseNon-terminal. Waiting on a signal, event, or timer; may resume later in a new segment.
okTerminalCompleted successfully.
errorTerminalCompleted with a thrown / failed outcome.
cancelledTerminalExplicitly cancelled.
incompleteTerminal (reconciled)Telemetry ended without a clean run:end; most often the host froze or exited before flushing.
conflictedTerminal (reconciled)Stored terminal evidence or trace identity conflicts; identity could not be resolved.

What to do:

  • incomplete: the run may have succeeded but its process exited before reporting a terminal status. Bind the host lifecycle wrappers from Runtime setup so run:end actually lands before the worker freezes.
  • suspended: nothing is wrong; the run is durably waiting. It resolves when the resuming invocation calls observe.resumeRun() and eventually ends the run.
  • conflicted: two different terminal records (or two logical runs behind one trace alias) were observed. Fix the identity conflict at the source; the server never overwrites the first terminal record.

Fix the underlying flush, host lifecycle, or identity conflict when you need a cleaner terminal status; do not try to "force green" client-side. More diagnosis paths: Troubleshooting.

Delivery health

Delivery health is an independent axis: it describes whether telemetry for the run was observed cleanly, not whether the run succeeded.

StatusChipMeaning
unknownmutedDefault whenever the server cannot prove clean delivery. Never treated as healthy.
healthyokClean terminal run with causal ordering and no gaps, conflicts, or rejected delivery.
degradedwarnSome telemetry records were rejected or delivery could not complete cleanly.

A still-running run stays unknown; a process that never connected stays unknown. The server never invents healthy. The exact promotion conditions, segment/ordering signals, and the plain-language UI copy for every state live in the Observability reference.

Filtering Runs by definition

Catalog View N runs opens Runs pre-filtered to one definition id. See Catalog runtime evidence.

Bounded media streams

For the producing operation, consumption, Safety, and asset-ownership model, start with Streaming generated media and Storage and delivery.

An executed streamImage() or streamSpeech() appears as one logical bounded operation with a separate Physical attempts timeline. The logical summary owns route and commitment state, attempt count, preview/delta/final counts, byte totals, validated media types, time to first public event, total duration, and terminal ok, error, cancelled, or timeout state. A pre-commit fallback therefore shows several physical attempts beneath one logical success; a post-commit provider failure remains one failed logical operation. Each physical attempt row owns its preview/delta/final counts, byte total, and validated media types, so fallback history is not confused with the logical aggregate.

Safety rows identify preview/final provenance and whether enforcing policy held provisional deltas before releasing or discarding them. The read model never contains the held bytes. There are no thumbnails, playback controls, prompts, base64, URLs, filenames, hashes, refs, or provider-native events in this presentation.

Current project health

When a run contains DefinitionRef identities, Run Detail correlates them at read time with lint findings in the currently materialized Project Index. The Current project health card separates active and suppressed findings and links matched definitions back to Catalog only when they resolve in the current Catalog with a supported kind. Unresolved or unsupported definitions remain plain, non-navigable evidence. Suppressed rows are explicitly tagged and include directive location, scope, and an authored reason when one exists; finding source and directive source remain distinct.

This card describes current authored state, not historical run evidence. It does not prove that a finding or suppression existed when the run executed, does not manufacture an observability span, and does not affect run status or lint/check gates. With no current Project Index the context is simply absent. See Catalog runtime evidence and Index Lint.

Host lifecycle and incomplete runs

Ephemeral hosts that return before flushing the delivery queue commonly produce incomplete or long-lived running presentation until reconciliation. Use the host wrappers documented under Runtime setup so run:end / span:end actually land.

Evidence delivery

evidence.record() returns after local synchronous acceptance. Destination idempotency conflicts, privacy tombstones, and staging outcomes arrive later as per-record delivery dispositions. A permanent evidence rejection increments observe.flush().rejected; flush().status can still be drained once every record has a final outcome, and evidence delivery never replaces the application result.

Run Detail's Evidence view is independent of the run lifecycle status. A claim may arrive after an explicit terminal record, outlive routine run retention, or remain queryable with its payload expired. See Execution evidence.

On this page