Cache

Tool-result and model-response caches, keys and provider prompt caches. Concepts: Cache.

agent_adapter.cache

Caching: backends, keys, envelopes, the tool and LLM caches, and the prompt layout.

Provider prompt-cache plug-ins live in agent_adapter.cache.providers.

KEY_VERSION = 1 module-attribute

CachePort

Bases: Protocol

Byte cache. Implementations log backend failures and act as a miss; never raise.

Envelope dataclass

A decoded cache entry.

LLMResponseCache

Stores normalised LLMResponses under llm_key (provider, model, request, fields).

get(request, *, provider, model, fields) async

The cached response with cached=True and latency_ms=0, or None.

set(request, response, *, provider, model, fields) async

Store response; False for errors, cache hits, opt-outs or backend failures.

PromptLayout dataclass

Static prefix (system prompt, tool catalogue, few-shots) plus per-request text.

messages(user=None, *, history=())

System turn (cache breakpoint), history, then dynamic + user as one user turn.

prefix_digest()

Digest of system_text(); changes exactly when the cacheable prefix changes.

system_text()

static then every non-empty section, separated by blank lines.

with_section(name, text)

A copy with section name set (replaced in place, or appended).

with_static(text)

A copy with static replaced.

RedisCache

Byte cache on Redis; RedisError is logged at WARNING and treated as a miss.

Raises:

Type	Description
ConfigError	not exactly one of url / client_factory, a malformed URL, or bad retry settings.
MissingDependency	the redis package is not installed.

aclose() async

Close the running loop's client (a new one is made on next use).

delete(key) async

Remove key; a Redis error is logged and ignored.

get(key) async

The value, or None on a miss or a Redis error.

set(key, value, ttl=None) async

Store with ex=ceil(ttl); ttl <= 0 deletes the key; inf / nan never expire.

ToolResultCache

Stores tool results under tool_key, scoped to a run, a thread or globally.

get(tool, args, *, scope, ctx) async

The cached envelope, or None on a miss, an uncacheable call or a backend error.

set(tool, args, value, *, ttl, scope, ctx) async

Store value; False when it cannot be cached or the backend fails.

canonical_json(obj)

Deterministic JSON (sorted keys, no spaces, UTF-8).

Raises:

Type	Description
TypeError	obj holds a value JSON cannot represent (the caller treats it as uncacheable).

decode(data, kind)

The envelope in data; None for bad JSON, another version or another kind.

digest(obj)

First 32 hex characters of the SHA-256 of canonical_json(obj).

encode(kind, payload)

Wrap payload as {"v", "kind", "stored_at", "payload"} JSON bytes.

Raises:

Type	Description
TypeError	payload is not JSON-serialisable.

gcache_key(namespace, model, prefix_digest)

Name under which a Gemini explicit cache for one static prompt prefix is stored.

llm_key(namespace, request, *, provider, model, fields)

Key for an LLM response; fields is extra key material from the port and plug-ins.

Raises:

Type	Description
TypeError	request.extra or fields are not JSON-serialisable.

tool_key(namespace, tool, args, *, scope, run_id, thread_id)

Key for a tool result; "" when the call cannot be cached (no run or thread id).

Raises:

Type	Description
TypeError	args are not JSON-serialisable.
ToolUsageError	unknown scope.

Also exported here, documented under the canonical path: MemoryCache.

agent_adapter.cache.providers

Provider prompt-cache plug-ins; each edits SDK kwargs only where the model's caps allow.

Without caps=, the capability flags come from Prices.default().caps.

AnthropicPromptCache

Adds cache_control to the last system block, last tool and breakpoint messages (max 4).

apply(request, kwargs, *, model, client) async

Kwargs with cache_control markers, at most four in total.

key_fields()

Constructor options.

GeminiExplicitCache

Creates one Gemini cached content per (model, prefix), then sends its name instead.

apply(request, kwargs, *, model, client) async

Kwargs whose config uses the cached prefix; unchanged when not eligible.

key_fields()

Constructor options.

GeminiImplicitCache

Gemini 2.5+ caches repeated prefixes on its own; pair it with a PromptLayout.

apply(request, kwargs, *, model, client) async

The kwargs, unchanged.

key_fields()

No options.

OpenAIPromptCache

Bases: _KeyMixin

Sets prompt_cache_key / prompt_cache_retention where caps allow; keeps kwargs values.

apply(request, kwargs, *, model, client) async

Kwargs with the cache parameters the model's capabilities allow.

key_fields()

Constructor options (not the computed key).

OpenAIPromptCacheV2

Bases: _KeyMixin

Sets prompt_cache_options, prompt_cache_key and, in explicit mode, block breakpoints.

apply(request, kwargs, *, model, client) async

Kwargs with prompt_cache_options (and breakpoints in explicit mode).

key_fields()

Constructor options (not the computed key).