Models

Model ports, provider models and usage helpers. Concepts: Model ports, Tool loop.

agent_adapter.models

Model ports: provider adapters, a host-function port, fallback, instrumentation, usage.

The provider ports import their SDK only when used (pip install "agent-adapter[openai]", [gemini], [anthropic]).

CapabilityLookup = Callable[[str], ModelCaps] module-attribute

DEFAULT_FALL_ON = (ModelRateLimited, ModelUnavailable, ModelTimeout) module-attribute

StreamEvent = TextDelta | ToolCallDelta | PartialOutput | StreamDone module-attribute

AnthropicModel

Bases: ProviderPort

Anthropic as an LLMPort (anthropic>=2.3; pip install "agent-adapter[anthropic]").

Parameters:

Name	Type	Description	Default
model	str	Model id sent when the request names none.	required
client	Any | None	An AsyncAnthropic the host owns; else one is created per event loop.	None
client_kwargs	Mapping[str, Any] | None	AsyncAnthropic(...) kwargs, e.g. {"max_retries": 0} for fast failover.	None
cache_plugin	PromptCachePlugin | None	Prompt-cache plug-in that edits the SDK kwargs.	None
max_tokens	int	max_tokens when the request sets no max_output_tokens (required by the API).	4096
defaults	Mapping[str, Any] | None	SDK kwargs applied under the request fields.	None

Raises:

Type	Description
MissingDependency	no client and the anthropic SDK is not installed.

GeminiModel

Bases: ProviderPort

Gemini as an LLMPort (google-genai>=2.30; pip install "agent-adapter[gemini]").

defaults and request.extra are GenerateContentConfig fields (merged into config).

Parameters:

Name	Type	Description	Default
model	str	Model id sent when the request names none.	required
client	Any | None	A genai.Client the host owns; else one is created per event loop.	None
client_kwargs	Mapping[str, Any] | None	genai.Client(...) kwargs (api_key, vertexai, http_options).	None
cache_plugin	PromptCachePlugin | None	Prompt-cache plug-in that edits the SDK kwargs (it gets the client).	None
thinking	Mapping[str, Any] | None	ThinkingConfig fields; request.reasoning sets thinking_level over them.	None
defaults	Mapping[str, Any] | None	GenerateContentConfig fields applied under the request fields.	None

Raises:

Type	Description
MissingDependency	no client and google-genai is not installed.

InstrumentedModel

An LLMPort that runs model with hooks, a response cache, ledger entries and events.

The model's own ledger / sink win; without them the call context's are used. The response cache is used only when cache is set (executor.model(m) passes the executor's). Each call records one kind="llm" ledger entry.

Raises:

Type	Description
ToolUsageError	a hook that is not a Hook.

bind(ctx)

A copy whose calls run in ctx (run id, ledger entry ids, ports).

cache_key_fields()

The wrapped port's LLM-cache key material.

generate(request) async

One instrumented call: hooks, response cache, the port, ledger entry, events.

Raises:

Type	Description
ModelError	the port's error when no on_llm_error hook recovers it.
ToolUsageError	a hook returned something other than an LLMResponse.
ConfigError	request.compression is invalid or could not run.

stream(request) async

The port's stream with the same hooks, ledger entry and events (no response cache).

Raises:

Type	Description
ModelError	as generate; a failure after the first event is not offered to hooks.
ConfigError	request.compression is invalid or could not run.

LLMPort

Bases: Protocol

A model provider; generate raises ModelError subclasses.

ModelCaps

Bases: BaseModel

Capability flags of one model; the defaults disable every gated feature.

prompt_window()

Tokens a prompt may use: context_window less max_output_tokens; None if unknown.

OpenAIModel

Bases: ProviderPort

OpenAI as an LLMPort (openai>=3.41; pip install "agent-adapter[openai]").

Parameters:

Name	Type	Description	Default
model	str	Model id sent when the request names none.	required
api	Literal['responses', 'chat']	"responses" (default) or "chat" (Chat Completions).	'responses'
client	Any | None	An AsyncOpenAI the host owns; else one is created per event loop.	None
client_kwargs	Mapping[str, Any] | None	AsyncOpenAI(...) kwargs, e.g. {"max_retries": 0} for fast failover.	None
cache_plugin	PromptCachePlugin | None	Prompt-cache plug-in that edits the SDK kwargs.	None
strict_tools	bool	Render tools strict when their ToolDef.strict allows it.	True
defaults	Mapping[str, Any] | None	SDK kwargs applied under the request fields.	None

Raises:

Type	Description
ConfigError	api is not "responses" or "chat".
MissingDependency	no client and the openai SDK is not installed.

OutputSchema

Bases: BaseModel

Structured-output schema for a request; ports apply provider transforms.

PartialOutput

Bases: BaseModel

The structured output parsed so far (streams with output_schema only); not validated.

PromptCachePlugin

Bases: Protocol

Edits provider SDK kwargs to use that provider's prompt cache.

StreamDone

Bases: BaseModel

The last stream item: the assembled response.

TextDelta

Bases: BaseModel

A chunk of streamed text.

ToolCallDelta

Bases: BaseModel

A chunk of a streamed tool call.

normalise_usage(provider, raw, *, api='')

Dispatch to the normaliser of provider (and OpenAI api: "chat" or "responses").

Raises:

Type	Description
ConfigError	no normaliser for this provider or API.

usage_anthropic(raw)

Usage of an Anthropic Message; input_tokens already excludes cache reads and writes.

usage_gemini(raw)

Usage of a Gemini GenerateContentResponse (snake_case or camelCase).

usage_openai_chat(raw)

Usage of a Chat Completions response (CompletionUsage).

usage_openai_responses(raw)

Usage of a Responses API response (ResponseUsage).

usage_voyage(raw)

Usage of a Voyage embed/rerank result; units is millions of tokens (price unit).

Also exported here, documented under the canonical path: CallableModel, FallbackModel.

agent_adapter.models.openai

OpenAIModel: the OpenAI Chat Completions and Responses APIs as an LLMPort.

build_chat_request(request, *, model, strict_tools=True)

chat.completions.create kwargs for request (without stream).

build_responses_request(request, *, model, strict_tools=True)

responses.create kwargs for request (without stream).

Assistant messages that carry Responses raw items (reasoning, function calls) are echoed back verbatim; seed and stop have no Responses equivalent and are dropped.

map_openai_error(exc, *, model)

The ModelError for an openai SDK exception; None for other exceptions.

Every openai.OpenAIError maps: errors sent inside a stream are typed by their body.

parse_chat_completion(raw)

A normalised response from a ChatCompletion (SDK object or dict); first choice only.

parse_response(raw)

A normalised response from a Responses Response (SDK object or dict).

Also exported here, documented under the canonical path: OpenAIModel.

agent_adapter.models.gemini

GeminiModel: the Gemini API (google-genai) as an LLMPort.

build_gemini_request(request, *, model)

models.generate_content kwargs: {"model", "contents", "config"} with a dict config.

Automatic function calling is always off (the adapter runs tools). Assistant messages with Gemini raw parts are echoed verbatim so thought signatures survive.

Raises:

Type	Description
ConfigError	a tool message whose tool name cannot be found.

map_gemini_error(exc, *, model)

The ModelError for a google-genai or httpx exception; None when unmapped.

parse_gemini_response(raw)

A normalised response from a GenerateContentResponse (SDK object or dict).

Also exported here, documented under the canonical path: GeminiModel.

agent_adapter.models.anthropic

AnthropicModel: the Anthropic Messages API as an LLMPort.

build_anthropic_request(request, *, model, max_tokens)

messages.create kwargs for request (without stream).

System text and message content are sent as blocks, so a cache plug-in can mark them. temperature, top_p and seed are dropped: the SDK's create has no such parameters.

map_anthropic_error(exc, *, model)

The ModelError for an anthropic SDK exception; None for other exceptions.

Every anthropic.AnthropicError maps: stream error events arrive as an APIStatusError with the stream's 200 status and are typed by their body.

parse_anthropic_message(raw)

A normalised response from an Anthropic Message (SDK object or dict).

Also exported here, documented under the canonical path: AnthropicModel.

agent_adapter.models.errors

Shared pieces of the provider error maps; each port maps its SDK classes.

The helpers are public on purpose: hosts writing their own LLMPort (or a CallableModel around their own client) use them to raise the same typed errors as the built-in ports.

class_for_error_type(code)

The ModelError subclass for a provider error type, e.g. one sent mid-stream.

class_for_status(status)

The ModelError subclass for an HTTP status the SDK has no dedicated class for.

error_type(body)

The error type and code of an error body ({"error": {...}} or flat), space-joined.

OpenAI puts the class in either field (type: "tokens", code: "rate_limit_exceeded").

is_context_overflow(text)

True when a provider error code or message says the input exceeds the context window.

model_error(cls, exc, *, provider, model, status_code=None, headers=None, request_id='')

Build cls from an SDK exception, copying status, Retry-After and request id.

retry_after(headers, *, now=None)

Seconds to wait from retry-after-ms or Retry-After (seconds or HTTP date).

with_overflow(cls, exc, body=None)

ModelContextOverflow for a bad request whose code or message names an overflow.

Also exported here, documented under the canonical path: ModelAuthError, ModelBadRequest, ModelContentFiltered, ModelContextOverflow, ModelError, ModelOutputInvalid, ModelRateLimited, ModelTimeout, ModelUnavailable.