Skip to documentation
Documentation navigation

Documentation navigation

Documentation / guides

Context Thresholds

Describe thresholds and rearming for automatic compaction.

developer

Automatic compaction uses occupancy of the complete counted request, not a message-length guess. The count comes from the configured Inference context counter. Occupancy is compared with explicit policy thresholds and the context tracker remembers the automatic basis that already triggered.

Occupancy and pressure

The counter returns InputTokens, InputLimit, model identity, quality, basis, and request fingerprint. Harness computes occupancy in basis points. With automatic mode enabled:

OccupancyPressure
Below CompactAtPressureNormal.
At or above CompactAt, below full scalePressureCompact.
At or above event.FullScaleBasisPointsPressureHardLimit.

At PressureCompact or PressureHardLimit, a new automatic attempt is coordinated unless the same context basis already exhausted automatic compaction. A hard limit with no new eligible attempt returns *loop.ContextLimitError.

Proof: context tracker and threshold behavior tests.

Rearming

After an automatic attempt is durably committed or rejected, the tracker marks that basis as exhausted for automatic triggering. While pressure remains at or above RearmBelow, later measurements do not open another attempt for the same basis. Once occupancy falls below RearmBelow, the pressure returns to normal and a later rise can trigger a new automatic attempt with a new basis.

Manual attempts do not exhaust the automatic basis. This lets a manual request finish or reject while an eligible automatic attempt can still be coordinated.

%%{init: {"theme":"dark"}}%%
stateDiagram-v2
    [*] --> Normal
    Normal --> Compact: occupancy >= CompactAt
    Compact --> Compact: occupancy >= RearmBelow
    Compact --> Normal: occupancy < RearmBelow
    Compact --> HardLimit: occupancy >= FullScaleBasisPoints
    HardLimit --> Normal: occupancy < RearmBelow

Proof: rearm and automatic basis and automatic retry tests.

Counter policy table

CounterPolicyQuality admitted for automatic mode
CounterPolicyRequireExactProvider-exact or local-exact only.
CounterPolicyAllowConservativeExact or heuristic estimate, with nonzero SafetyMargin for heuristic quality.
CounterPolicyUnknownRejected by policy validation.

CountTimeout bounds the context counter call. Timeout, invalid count model, quality mismatch, and malformed measurement are typed count failures; they do not silently trigger with a guessed limit.

Proof: policy validation and context count tests.

Source and proof

← back to documentation