Skip to documentation
Documentation navigation

Documentation navigation

Documentation / guides

Streaming inference

Open a pull-based chunk stream and always close it on every exit path.

developer

Client.Stream returns a *stream.StreamReader[content.Chunk]. The reader separates incremental content from the terminal result, so a caller can render text while retaining authoritative usage, model, finish reason, and attempt metadata at clean EOF.

API surface

Stream(ctx context.Context, req Request) (*stream.StreamReader[content.Chunk], error)
reader, err := client.Stream(ctx, req)
if err != nil {
	return err
}
defer reader.Close()

for {
	chunk, err := reader.Next()
	if errors.Is(err, io.EOF) {
		break
	}
	if err != nil {
		return err
	}
	switch typed := chunk.(type) {
	case *content.TextChunk:
		fmt.Print(typed.Text)
	case *content.ThinkingChunk:
		// Keep reasoning display policy separate from answer display policy.
	case *content.ToolUseChunk:
		// Buffer tool argument fragments until the call is complete.
	}
}
if result, ok := reader.Result(); ok {
	_ = result.FinishReason
}

Next serializes calls to the underlying reader. io.EOF is the only clean terminal signal; any other error permanently fails the reader and suppresses its result. Close is idempotent and should be deferred immediately after a successful Stream call.

Proof

Related: StreamReader, Terminal stream results, Close streams.

← back to documentation