Overview

Inference is the provider-neutral boundary between a model-serving client and the rest of Looprig. inference.Client exposes Invoke for one complete response and Stream for incremental content.Chunk values. A request carries a secret-free model.Model, system instructions, content.AgenticMessages, tools, an optional structured-output contract, tool choice, and sampling overrides.

Provider-neutral contract

Build the request from Core content and a validated model. Keep credentials in the provider client or its authenticator; model.Model describes routing and local capability limits but does not carry an API key.

package example

import (
	"context"

	"github.com/looprig/core/content"
	"github.com/looprig/inference"
	"github.com/looprig/inference/model"
)

func invoke(ctx context.Context, client inference.Client, selected model.Model) (*inference.Response, error) {
	if err := selected.Validate(); err != nil {
		return nil, err
	}
	req := inference.Request{
		Model: selected,
		Messages: content.AgenticMessages{
			&content.UserMessage{Message: content.Message{
				Role: content.RoleUser,
				Blocks: []content.Block{&content.TextBlock{Text: "Hello"}},
			}},
		},
	}
	if err := inference.ValidateRequestFeatures(req); err != nil {
		return nil, err
	}
	return client.Invoke(ctx, req)
}

The same request shape can be sent through client.Stream. The returned stream.StreamReader owns the response body: consume Next until it returns false, inspect Result, and call Close when the caller stops early. Structured output and tool calls remain request features, so validation happens before a provider codec attempts transport.

One request lifecycle

%%{init: {"theme":"dark"}}%%
flowchart LR
    M["model.Model: secret-free"] --> R[inference.Request]
    C["Core messages, tools, and output"] --> R
    R --> V[ValidateRequestFeatures]
    V --> I[Client.Invoke]
    V --> S[Client.Stream]
    I --> P[provider codec]
    P --> A[Response]
    S --> Q[StreamReader]
    Q --> T[Next and Result]
    Q --> X[Close]

Choose a path

Use the nested guides as the canonical developer path:

Keep the client lifecycle separate from the Harness lifecycle: Inference owns model requests, provider codecs, responses, and streams; Harness owns turns, steps, tool execution, gates, and durable session events.

Source

• inference.Client, Request, and Response
• model.Model validation and secret-free identity
• stream.StreamReader
• OutputSchema and request feature validation

Proof

• Client request and response tests
• Model validation tests
• Stream reader tests