Overview

The llm module owns provider policy above the neutral inference request contract. Each constructor below validates the provider and API format before network I/O, resolves an empty model base URL when the package has a reviewed default, and binds the credential scheme used by its transport. The individual pages link directly to the provider package and its tests.

Choose a provider

The format names are the values in inference/model: openai is Chat Completions, openai-responses is Responses, anthropic is Messages, gemini is Google generateContent, and bedrock-converse is the native Bedrock Converse envelope. A row with more than one format selects its route and codec from model.Model.APIFormat.

Construction and routing

Every model-backed constructor takes a model.Model, validates Provider, APIFormat, model name, and endpoint shape through llm.ValidateModel, then binds a transport endpoint. Empty BaseURL is not a universal promise: dynamic providers require the resource, account, project, location, deployment, or region that their page names.

When a provider becomes a Loop’s selected model, continue with Harness runtime model selection. Bounded auxiliary model selection for Hustles is covered by Hustle model selection.

%%{init: {"theme":"base","themeVariables":{"background":"#0b1020","primaryColor":"#172554","primaryTextColor":"#f8fafc","primaryBorderColor":"#60a5fa","lineColor":"#94a3b8","secondaryColor":"#1e293b","tertiaryColor":"#111827","fontFamily":"ui-sans-serif,system-ui"}}}%%
flowchart LR
    M[model.Model] --> V[llm.ValidateModel]
    V --> F{APIFormat}
    F --> C[provider constructor]
    C --> E[bound endpoint]
    C --> A[auth.Authenticator]
    E --> T[inference transport or native client]
    A --> T
    T --> R[Invoke or Stream]
    classDef dark fill:#172554,stroke:#60a5fa,color:#f8fafc;
    class M,V,F,C,E,A,T,R dark;

For the compatibility-backed packages, internal/compat.NewProvider selects the bundled OpenAI Chat, OpenAI Responses, or Anthropic codec and the format-specific path. Bespoke packages retain their own route where the provider puts the model in the URL, signs a request, performs token exchange, or encrypts the body.

Wire and capability boundaries

Streaming, tools, structured output, images, thinking, and usage are request features, not marketing labels. A provider page claims a feature only when its source selects a codec that encodes it, adds a provider-specific patch, or has a focused test. Shared compatibility providers do not invent a capability mask: request validation and the selected codec remain authoritative. The provider registry rejects an unsupported format before I/O.

Secret references and retries

Provider constructors receive already-resolved credential material such as auth.APIKey, bedrock.SigV4Credentials, an OAuth access token, or an SAP ServiceKey. They do not parse a generic secret reference. Applications can resolve a credentials.Descriptor and lease through credentials and secrets, then pass the resulting value to the constructor; labels and reference IDs must not be placed in model.Model.

The generic transport reports validation, network, HTTP, codec, and stream failures without a provider-local retry loop. Wrap idempotent calls in the inference retry package only after checking the provider page. GitLab retries once after invalidating its short-lived direct-access token, SAP refreshes OAuth during discovery, and Chutes re-attests when its cached session expires or nonces are exhausted. Those are authentication lifecycle behaviors, not a blanket retry guarantee.

Source and proof

• llm/provider.go defines provider identities, supported API formats, empty-base policy, and required auth.
• llm/validate.go proves fail-closed model validation.
• llm/authpolicy.go binds provider, transport, credential scheme, issuer, and audience.
• inference/transport and inference/retry define the shared request and retry boundaries.