Overview

Harness errors preserve enough typed information to distinguish invalid configuration, a live runtime failure, and a lifecycle or restore failure. Recovery starts by inspecting the chain, then deciding whether the durable journal contains progress that should be restored rather than recreated.

Classify errors by boundary

Use the package that owns the failed boundary. The public packages expose stable type and kind fields; internal runtime packages add ownership and cleanup context without putting provider payloads in error text.

Inspect the error chain

Harness uses both single-error and multi-error unwrapping. Public wrappers such as session.SessionError, session.RestoreError, rig.LifecycleError, and journal.AppendError implement Unwrap() error. Owned runtime failures such as hustleruntime.RunError, QueueFailureError, and CloseError implement Unwrap() []error so errors.As and errors.Is can see the primary cause and cleanup failures.

var restoreErr *session.RestoreError
if errors.As(err, &restoreErr) {
	log.Printf("restore stage=%s", restoreErr.Kind)
}

var mismatch *session.RestoreRuntimeMismatchError
if errors.As(err, &mismatch) {
	// Choose a configured runtime or report the category to the operator.
	log.Printf("restore runtime category=%s", mismatch.Kind)
}

if errors.Is(err, context.DeadlineExceeded) {
	// A wrapped deadline is still machine-detectable.
}

Do not branch on Error() strings. Error text intentionally omits credentials, model responses, raw tool arguments, and other provider-controlled values.

Durable progress changes recovery

A returned error does not imply that no state was written. Session construction and restore use leases and journal lifecycle events. A restore that has opened the journal and appended RestoreStarted records RestoreErrored on later failure, releases its lease, and leaves the original stream available for a later retry. A failed setup before RestoreStarted has no restore error event, but it still releases the acquired lease.

Live runtime failures can also be durable. A session may enter the SessionFaulted state after a persistence or workspace-integrity failure, and shutdown still owns cleanup and lease release. Treat the session ID and journal as the source of truth before deciding to create a replacement session.

%%{init: {"theme":"dark"}}%%
flowchart TD
    A[Returned error] --> B{errors.As typed wrapper}
    B -- configuration --> C[Fix definition or binding]
    B -- runtime --> D{Session still owns durable state?}
    B -- restore --> E[Inspect RestoreErrored or drift category]
    B -- shutdown cleanup --> F[Wait for owner result and inspect children]
    D -- yes --> G[Read status or journal; restore same ID]
    D -- no --> H[Inspect lease and storage before retry]
    E --> I[Correct drift/runtime and retry RestoreSession with same ID]

Choose a recovery action

Use this order when handling a failure:

1. Extract the outer typed kind with errors.As.
2. Follow Unwrap to distinguish the primary cause from cleanup or caller cancellation.
3. Read the session status or journal when a session ID exists.
4. Retry only after correcting a transient condition such as capacity, availability, or a released lease.
5. Preserve the original ID for restore. Create a new session only when the application intentionally discards the durable history.

The focused pages cover the details: configuration, runtime, restore, and shutdown.

Source and runnable proof

The public error types are defined in pkg/session/errors.go, pkg/rig/errors.go, pkg/loop/errors.go, pkg/hustle/definition_errors.go, pkg/gate/validate.go, pkg/hook/errors.go, pkg/journal/errors.go, and pkg/sessionstore/replay.go. Owned-run and cleanup chains are in internal/hustleruntime/errors.go. Representative chain assertions are in pkg/session/errors_test.go, internal/hustleruntime/cleanup_test.go, and internal/sessionruntime/session_hub_test.go.