Skip to documentation
Documentation navigation

Documentation navigation

Documentation / guides

Overview

Normalize model token usage and keep it distinct from preflight context counts.

developer

Inference usage is postflight provider-reported consumption. The public inference/usage.Usage type is an alias of core/content.Usage; codecs fill it from native response fields and streams expose it only after clean EOF.

Model usage

type Usage struct {
	InputTokens         content.TokenCount
	OutputTokens        content.TokenCount
	CacheReadTokens     content.TokenCount
	CacheCreationTokens content.TokenCount
	ReasoningTokens     content.TokenCount
}

InputTokens excludes cache subsets when a provider reports gross input; ContextTokens derives input plus cache-read and cache-creation values; TotalTokens adds output. Missing provider usage is represented by nil on a response or stream result.

Separation

contextcount.ContextCounter estimates preflight request occupancy. It has no response or output field. Usage is authoritative provider accounting after a call. Do not use a zero Usage to stand in for a failed count or an absent provider report.

Source and proof

Run go test ./usage ./stream ./codec/....

← back to documentation