Sampling overrides

Request.Override is an optional pointer to per-call model.Sampling. A nil pointer means the codec should use Request.Model.Sampling; a non-nil value replaces those defaults for this request.

API surface

temperature := 0.2
request := inference.Request{
	Model: model.CustomModel(
		model.ProviderName("acme"), model.APIFormatOpenAI,
		"https://api.example.test", "chat-1",
		model.WithSampling(model.Sampling{MaxTokens: intPtr(1024)}),
	),
	Override: &model.Sampling{Temperature: &temperature},
}

The request owns the pointer value for the duration of encoding, so do not mutate it concurrently. CustomModel and WithSampling deep-copy model defaults; use Sampling.Clone if a caller needs an independent override before changing pointer or slice fields.

func intPtr(value int) *int { return &value }

Numeric ranges, stop-sequence limits, and provider-specific combinations are codec policy. The model package only supplies the shape and copy behavior.

Proof

• Source: inference/client.go, inference/model/sampling.go, inference/model/model.go
• Tests: inference/model/sampling_test.go, inference/model/model_test.go

Related: Sampling, Model selection.